36.5.1Metrics Endpoint

 

The Metrics endpoint is used to access runtime properties of the running node. It exposes functionality provided by the Dropwizard Metrics library.

36.5.2Fetch Local Metrics

 
This method requires the VIEW_METRICS permission.

This method will return the runtime metrics provided by a given module (and only those metrics that are specific to the selected module). This method provides only metrics for the local Node (i.e. the same node where the JSON Admin API is running). As a result it is very resilient and consumes very few resources, as it does not need to access any remote services or processes (e.g. it does not need to talk to a database).

36.5.2.1Parameters

Required:

  • format: (optional) The output format to use. Can be any of the following values:
    • DROPWIZARD – The default Dropwizard export format
    • PROMETHEUS – The Prometheus native client format
    • PROMETHEUS_V2 – The Prometheus native client format with additional labels (node_id, process_id, module_name)

To invoke:

36.5.2.1.1Dropwizard format

GET http://localhost:9000/metrics/local/metrics?format=DROPWIZARD

The server will produce a response resembling the following:

{
  "version": "4.0.0",
  "gauges": {
    "fhir_endpoint.db_num_active": {
      "value": 0
    },
    "fhir_endpoint.db_num_idle": {
      "value": 3
    },
    "fhir_endpoint.fhir_searches_active": {
      "value": 0
    },
    "fhir_endpoint.ft_index_remaining": {
      "value": 0
    }
  }
}

For brevity only a small subset of the metrics are shown here but a real response might contain many more.

36.5.2.1.2Prometheus format

GET http://localhost:9000/metrics/local/metrics?format=PROMETHEUS

The server will produce a response resembling the following:

# TYPE persistence_overall_transaction_metric summary
persistence_overall_transaction_metric{quantile="0.5",} 0.014
persistence_overall_transaction_metric{quantile="0.75",} 0.163
persistence_overall_transaction_metric{quantile="0.95",} 0.163
persistence_overall_transaction_metric{quantile="0.98",} 0.246
persistence_overall_transaction_metric{quantile="0.99",} 0.246
persistence_overall_transaction_metric{quantile="0.999",} 0.246
persistence_overall_transaction_metric_count 3603.0

For brevity only a small subset of the metrics are shown here but a real response might contain many more.

36.5.2.1.3Prometheus format with additional labels

GET http://localhost:9000/metrics/local/metrics?format=PROMETHEUS_V2

The server will produce a response resembling the following:

# TYPE overall_transaction_metric summary
overall_transaction_metric{quantile="0.5",node_id="Master",process_id="QueensQuay",module_name="persistence",} 0.014
overall_transaction_metric{quantile="0.75",node_id="Master",process_id="QueensQuay",module_name="persistence",} 0.153
overall_transaction_metric{quantile="0.95",node_id="Master",process_id="QueensQuay",module_name="persistence",} 0.163
overall_transaction_metric{quantile="0.98",node_id="Master",process_id="QueensQuay",module_name="persistence",} 0.163
overall_transaction_metric{quantile="0.99",node_id="Master",process_id="QueensQuay",module_name="persistence",} 0.163
overall_transaction_metric{quantile="0.999",node_id="Master",process_id="QueensQuay",module_name="persistence",} 0.246
overall_transaction_metric_count{node_id="Master",process_id="QueensQuay",module_name="persistence",} 3609.0

For brevity only a small subset of the metrics are shown here but a real response might contain many more.

36.5.2.2Filtering Metrics

The metrics endpoint allows two additional parameters that can be used to filter the metrics that are returned.

These two parameters are called whitelist and blacklist, and act as an inclusionary whitelist and an exclusionary blacklist respectively. Values for these parameters are treated as regular expressions and can match on partial strings. They are also permitted to repeat.

For example, given the following URL: http://localhost:9000/metrics/local/metrics?whitelist=fhir.*_endpoint&blacklist=4xx&blacklist=5xx

This will match fhir_endpoint.http_responses_2xx and fhirweb_endpoint.http_responses_2xx, but will not match fhir_endpoint.http_responses_4xx.

36.5.2.3Anonymous Access

In some architectures, a monitoring/management system that is not expecting to be authenticated requires access to the metrics endpoint. If you would like to enable anonymous access specifically for this type of request, see Enabling Anonymous Access. You will need to specifically grant your anonymous user the following permissions:

36.5.3Capture Thread Dump

 
This method requires the VIEW_METRICS permission.

This method will return a thread dump for the local node (i.e. the same Java process that the JSON Admin API is running on).


To invoke:

GET http://localhost:9000/metrics/local/threads

The server will produce a response resembling the following:

"ForkJoinPool.commonPool-worker-7" id=174 state=TIMED_WAITING
    - waiting on (a java.util.concurrent.ForkJoinPool)
    - locked (a java.util.concurrent.ForkJoinPool)
    at java.base@11.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@11.0.2/java.util.concurrent.locks.LockSupport.parkUntil(LockSupport.java:275)
    at java.base@11.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1619)
    at java.base@11.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)

For brevity only a small subset of the threads are shown here but a real response might contain many more.

36.5.3.1Parameters

Optional:

  • writeToLog: (optional) If this parameter is supplied a value of true, the thread dump will also be written to the Smile CDR logs.