Showing documentation for 2025.12.R03
On this page:
   Table of Contents

Evaluating Measures

Table of Contents

 

Measure Resource

 

The FHIR Clinical Reasoning Module defines the Measure resource and several associated operations. The Measure Resource represents a structured, computable definition of a health-related measure such as a clinical quality measure, public health indicator, or population analytics measure. These Measures can then be used for reporting, analytics, and data-exchange purposes.

Electronic Clinical Quality Measures (eCQMs) in FHIR are represented as a FHIR Measure resource containing metadata and terminology, a population criteria section, and at least one FHIR Library resource containing a data criteria section as well as the logic used to define the population criteria. The population criteria section typically contains initial population criteria, denominator criteria, and numerator criteria subcomponents, among others. This is elaborated upon in greater detail in the CQF Measures IG. An example of an eCQM as defined in FHIR looks like:

{
   "resourceType" : "Measure",
   "library" : [
      "http://hl7.org/fhir/us/cqfmeasures/Library/EXMLogic"
   ],
   "group" : [
      {
         "population" : [
            {
               "code" : {
                  "coding" : [
                     {
                        "code" : "initial-population"
                     }
                  ]
               },
               "criteria" : {
                  "language" : "text/cql-identifier",
                  "expression" : "Initial Population"
               }
            },
            {
               "code" : {
                  "coding" : [
                     {
                        "code" : "numerator"
                     }
                  ]
               },
               "criteria" : {
                  "language" : "text/cql-identifier",
                  "expression" : "Numerator"
               }
            },
            {
               "code" : {
                  "coding" : [
                     {
                        "code" : "denominator"
                     }
                  ]
               },
               "criteria" : {
                  "language" : "text/cql-identifier",
                  "expression" : "Denominator"
               }
            }
         ]
      }
   ]
}

Measures are then scored according to whether a subjects (or subjects) are members of the various populations.

For example, a Measure for Breast Cancer screening might define an Initial Population (via CQL expressions) of "all women", a Denominator of "women over 35", and a Numerator of "women over 35 who have had breast cancer screenings in the past year". If the Measure is evaluated against a population of 100 women, 50 are over 35, and of those 25 have had breast cancer screenings in the past year, the final score would be 50%1 (total number in numerator / total number in the denominator).

  1. There are several methods for scoring Measures, this is meant only as an example.

Measure Evaluating Options

 

The FHIR Measure specification defines several types of Measures and various parameters for controlling the Measure evaluation. This section describes the features supported by Smile CDR.

Populations

Smile CDR uses the populations defined by the CQF Measures IG for each scoring type. A matrix of the supported populations is shown in the Criteria Names section of the CQF Measures IG.

Population Criteria

The logical criteria used for determining each Measure population is defined by the Measure.group.population.criteria element. The Measure specification allows population criteria to be defined using FHIR Path, CQL, or other languages as appropriate. Smile CDR currently only supports using CQL. The relationship between a Measure Population and CQL is illustrated in the Population Criteria section of the CQF Measures IG.

An example Measure resource with a population criteria referencing a CQL identifier looks like:

{
   "resourceType": "Measure",
   "group": [ {
      "population": [ {
         "code": {
            "coding": [ {
               "system": "http://terminology.hl7.org/CodeSystem/measure-population",
               "code": "initial-population",
               "display": "Initial Population"
            } ]
         },
         "criteria": {
            "language": "text/cql-identifier",
            "expression": "Initial Population"
         }
      }]
   }]
}

Population Basis

The population basis specifies the type of elements represented in the MeasureReport populations (example: "Numerator").

For measures that have a population basis that is different from a subject, this element specifies the type of the population basis.

This done with an extension element that can be added to a measure, see examples below.

Scoring BasisSupportedDescription
Boolean✓Population criteria define True/False values for each individual in the population
Encounter✓Population criteria return lists of encounters for a subject in the population
other resources✓Population criteria return lists of "resources" like an encounter for a subject in population

Example Population Basis of Encounter

{
	"extension": [ {
"url": "http://hl7.org/fhir/us/cqfmeasures/StructureDefinition/cqfm-populationBasis",
"valueCode": "Encounter"
} ]
}

Example Population Basis of Boolean

{
  "extension": [
    {
      "url": "http://hl7.org/fhir/us/cqfmeasures/StructureDefinition/cqfm-populationBasis",
      "valueCode": "boolean"
    }
  ]
}

Scoring Methods

The Measure scoring method determines how a Measure score is calculated. It is set with the scoring element on the Measure resource.

The Smile CDR implementation conforms to the requirements defined by the CQF Measures IG. A more detailed description of each scoring method is linked in the table below.

Scoring MethodSupportedDescription
proportion✓Proportion Measures
ratio✓Ratio Measures
continuous-variable✓Continuous Variable
cohort✓*Cohort
composite See below
  • The cohort Measure scoring support is partial. The Smile CDR implementation does not yet return the required Measure observations

An example Measure resource with scoring defined looks like:

{
  "resourceType": "Measure",
  "scoring": {
    "coding": [ {
      "system": "http://terminology.hl7.org/CodeSystem/measure-scoring",
      "code": "proportion",
      "display": "Proportion"
    } ]
  }
}

CDR also supports setting measure-scoring` on a Group level instead of just on the Measure level.

{
   "group": [
      {
         "id": "64f0daaf56d636294b157ea1",
         "extension": [
            {
               "url": "http://hl7.org/fhir/us/cqfmeasures/StructureDefinition/cqfm-scoring",
               "valueCodeableConcept": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-scoring",
                        "code": "proportion",
                        "display": "Proportion"
                     }
                  ]
               }
            },
            {
               "url": "http://hl7.org/fhir/us/cqfmeasures/StructureDefinition/cqfm-populationBasis",
               "valueCode": "Encounter"
            },
            {
               "url": "http://hl7.org/fhir/us/cqfmeasures/StructureDefinition/cqfm-type",
               "valueCodeableConcept": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-type",
                        "code": "process",
                        "display": "Process"
                     }
                  ]
               }
            }
         ]
      }
   ]
}

Composite Scoring

A composite Measure is scored by combining and/or aggregating the results of other Measures. The compositeScoring element is used to control how composite Measures are scored. Smile CDR does not currently support any composite scoring method.

Composite Scoring MethodSupportedDescription
opportunity Combines Numerators and Denominators for each component Measure
all-or-nothing Includes individuals that are in the numerator for all component Measures
linear Gives an individual score based on the number of numerators in which they appear
weighted Gives an individual a cored based on a weighted factor for each numerator in which they appear

Criteria Expression Type

Expression TypeSupported
CQL✓
FHIR Path

Supplemental Data Elements

Supplemental Data Elements are used to report additional information about the subjects that may not be included in the Population criteria definitions. For example, it may be of interest to report the gender of all subjects for informational purposes. Supplemental data elements are defined by the Measure.supplementalData element, and are reported as Observations in the evaluatedResources of the MeasureReport.

Supplemental Data Elements can be specified as either CQL definitions or FHIR Path expressions.

Expression TypeSupported
CQL✓
FHIR Path

An example Measure resource with some supplemental data elements set looks like:

{
"resourceType": "Measure",
   "supplementalData": [ {
      "code": {
         "text": "sde-ethnicity"
      },
      "criteria": {
         "language": "text/cql-identifier",
         "expression": "SDE Ethnicity"
      }
   }]
}

Stratifiers

 

  • Stratifiers are used divide Measure populations into segments of interest. For example, it may be of interest to compare the Measure score between different age groups or genders. Each stratum within a stratification is scored the same way as the overall population. Stratifiers are defined using the Measure.group.stratifier element.

  • An example Measure resource with a stratifier set looks like:

{
  "resourceType": "Measure",
  "group": [ {
      "stratifier": [ {
         "code": {
            "text": "Stratum 1"
         },
         "criteria": {
            "language": "text/cql-identifier",
            "expression": "Stratification 1"
         }
      }]
   }]
}

Stratifier Expression Support

As with Populations and Supplemental Data Elements the criteria used for Stratification may be defined with CQL or FHIR Path.

Expression TypeSupported
CQL✓
FHIR Path

Stratifier Component Support

The Measure specification also supports multidimensional stratification, for cases where more than one data element is needed.

Stratifier TypeSupported
Single Component✓
Multi Component✓

Criteria-Based Stratifiers

  • Must be defined on the field "criteria", and cannot be defined under stratifier.component.
  • Criteria Stratifier must be of the same basis as group.populations to allow proper intersection of results
  • No Component criteria stratifier capability, this can be done with one expression, given there is no combination of values visible on final reports
  • Intersection logic is key to criteria stratifiers: Measure evaluation will compute, per subject, the intersection between a given population's evaluation results (ex: Encounters) with that of the stratifier, and include only those in computing the final counts and scores in the stratum, aggregated or not based on the report type (ex individual vs. summary)

Example

  • An example of criteria-based stratification is a stratifier driven off a CQL expression: in-progress encounters, which filters encounters by those whose period is within the measurement period.
  • This means for all the populations captured by the measure group (initial population, denominator, numerator), we'll return counts only for those that meet the stratifier criteria.
  • This results in multiple stratum each mirroring their group population equivalent, but with the counts above:

Measure

"stratifier": [
  {
	 "id": "stratifier-1",
	 "criteria": {
		"language": "text/cql.identifier",
		"expression": "in-progress encounters"
	 }
  }
]

Measure Report

    "stratifier": [ {
      "id": "stratifier-1",
      "code": [ {
        "text": "Encounters in Period"
      } ],
      "stratum": [ {
        "population": [ {
          "id": "initial-population",
          "code": {
            "coding": [ {
              "system": "http://terminology.hl7.org/CodeSystem/measure-population",
              "code": "initial-population",
              "display": "Initial Population"
            } ]
          },
          "count": 3
        }, {
          "id": "denominator",
          "code": {
            "coding": [ {
              "system": "http://terminology.hl7.org/CodeSystem/measure-population",
              "code": "denominator",
              "display": "Denominator"
            } ]
          },
          "count": 2
        }, {
          "id": "numerator",
          "code": {
            "coding": [ {
              "system": "http://terminology.hl7.org/CodeSystem/measure-population",
              "code": "numerator",
              "display": "Numerator"
            } ]
          },
          "count": 1
        } ],
        "measureScore": {
          "value": 0.5
        }
      } ]
    } ]

Value-Based Stratifiers

  • Must be defined on Stratifier.component
  • Value based stratifiers must define at minimum one criterion, but can be up to N different criteria values

Grouping Logic

  • In some cases, value-based stratifiers can be quite complex.

Example

  • An example value-based stratifier is based on a gender stratification, where results are segmented based on a patient's gender
  • The initial population of 10 is split in two stratum, one for 5 males, and the other for 5 females.

Measure

"stratifier": [
  {
	 "id": "stratifier-1",
	 "code": {
		"coding": [
		  {
			 "system": "http://loinc.org",
			 "code": "72143-1",
			 "display": "Sex [HL7.v3]"
		  }
		],
		"text": "stratifier-sex"
	 },
	 "component": [
		{
		  "id": "stratifier-1-comp-1",
		  "code": {
			 "coding": [
				{
				  "system": "http://loinc.org",
				  "code": "72143-1",
				  "display": "Sex [HL7.v3]"
				}
			 ],
			 "text": "stratifier-sex"
		  },
		  "criteria": {
			 "language": "text/cql.identifier",
			 "expression": "Gender Stratification"
		  }
		}
	 ]
  }
]

Measure Report

"stratifier": [ {
	"id": "stratifier-1",
	"code": [ {
	  "coding": [ {
		 "system": "http://loinc.org",
		 "code": "72143-1",
		 "display": "Sex [HL7.v3]"
	  } ],
	  "text": "stratifier-sex"
	} ],
	"stratum": [ {
	  "value": {
		 "text": "F"
	  },
	  "population": [ {
		 "id": "initial-population",
		 "code": {
			"coding": [ {
			  "system": "http://terminology.hl7.org/CodeSystem/measure-population",
			  "code": "initial-population",
			  "display": "Initial Population"
			} ]
		 },
		 "count": 5
	  } ]
	}, {
	  "value": {
		 "text": "M"
	  },
	  "population": [ {
		 "id": "initial-population",
		 "code": {
			"coding": [ {
			  "system": "http://terminology.hl7.org/CodeSystem/measure-population",
			  "code": "initial-population",
			  "display": "Initial Population"
			} ]
		 },
		 "count": 5
	  } ]
	} ]
} ]

Example 2

  • In the case of multiple stratifiers, in this case Gender and Age, measure evaluation will group results

Measure

"stratifier": [
  {
	 "id": "stratifier-1",
	 "code" : {
		"text": "Gender and Age"
	 },
	 "component": [
		{
		  "id": "stratifier-comp-1",
		  "code" : {
			 "text": "Gender"
		  },
		  "criteria": {
			 "language": "text/cql.identifier",
			 "expression": "Gender Stratification"
		  }
		},
		{
		  "id": "stratifier-comp-2",
		  "code" : {
			 "text": "Age"
		  },
		  "criteria": {
			 "language": "text/cql.identifier",
			 "expression": "Age"
		  }
		}
	 ]
  }
]
  • The first step is to gather all results by subject:
Subject (String)CriteriaResult (ValueWrapper)StratifierComponentDef
subject-aMgender
subject-bFgender
subject-cMgender
subject-dFgender
subject-eFgender
subject-fFgender
subject-gMgender
subject-hMgender
subject-iMgender
subject-a38age
subject-b35age
subject-c35age
subject-d38age
subject-e35age
subject-f38age
subject-g35age
subject-h38age
subject-i38age
  • And then group them as follows:
SetList<Subjects(String)>
<'F',35>[subject-e]
<'M',35>[subject-c,subject-g]
<'F',38>[subject-b,subject-d,subject-f]
<'M',38>[subject-a,subject-c,subject-h,subject-i]

MeasureReport

    "stratifier": [ {
      "id": "stratifier-1",
      "code": [ {
        "text": "Gender and Age"
      } ],
      "stratum": [ {
        "component": [ {
          "code": {
            "text": "Age"
          },
          "value": {
            "text": "35"
          }
        }, {
          "code": {
            "text": "Gender"
          },
          "value": {
            "text": "F"
          }
        } ],
        "population": [ {
          "id": "initial-population",
          "code": {
            "coding": [ {
              "system": "http://terminology.hl7.org/CodeSystem/measure-population",
              "code": "initial-population",
              "display": "Initial Population"
            } ]
          },
          "count": 1
        } ]
      }, {
        "component": [ {
          "code": {
            "text": "Gender"
          },
          "value": {
            "text": "M"
          }
        }, {
          "code": {
            "text": "Age"
          },
          "value": {
            "text": "35"
          }
        } ],
        "population": [ {
          "id": "initial-population",
          "code": {
            "coding": [ {
              "system": "http://terminology.hl7.org/CodeSystem/measure-population",
              "code": "initial-population",
              "display": "Initial Population"
            } ]
          },
          "count": 2
        } ]
      }, {
        "component": [ {
          "code": {
            "text": "Gender"
          },
          "value": {
            "text": "F"
          }
        }, {
          "code": {
            "text": "Age"
          },
          "value": {
            "text": "38"
          }
        } ],
        "population": [ {
          "id": "initial-population",
          "code": {
            "coding": [ {
              "system": "http://terminology.hl7.org/CodeSystem/measure-population",
              "code": "initial-population",
              "display": "Initial Population"
            } ]
          },
          "count": 3
        } ]
      }, {
        "component": [ {
          "code": {
            "text": "Gender"
          },
          "value": {
            "text": "M"
          }
        }, {
          "code": {
            "text": "Age"
          },
          "value": {
            "text": "38"
          }
        } ],
        "population": [ {
          "id": "initial-population",
          "code": {
            "coding": [ {
              "system": "http://terminology.hl7.org/CodeSystem/measure-population",
              "code": "initial-population",
              "display": "Initial Population"
            } ]
          },
          "count": 4
        } ]
      }
     ]
    } 
]

Evaluated Resources

A FHIR MeasureReport permits referencing the Resources used when evaluating in the MeasureReport.evaluatedResource element. Smile CDR includes these resources when generating subject reports for a single Patient. Evaluated resources for population or subject-list reports are not included. For large populations this could quickly become an extremely large number of resources.

The evaluated resources will not include every resource on the Smile CDR server for a given subject. Rather, it includes only the resources that were retrieved from the server by the CQL logic that was evaluated. This corresponds to the data-requirements for a given Measure. As an example, consider the following CQL:

valueset "Example Value Set" : 'http://fhir.org/example-value-set'

define "Example Observations":
   [Observation : "Example Value Set"]

That CQL will only select Observation Resources that have a code in the "Example Value Set". Those Observations will be reported in the Evaluated Resources while any others will not.

Extensions

A number of extensions to Measure evaluation defined by various IGs are supported. They are described briefly in the table below.

ExtensionDescription
http://hl7.org/fhir/us/cqframework/cqfmeasures/StructureDefinition/cqfm-productLineUsed to evaluate different product lines (e.g. Medicare, Private, etc.)
http://hl7.org/fhir/StructureDefinition/cqf-measureInfoUsed to denote a Measure Observation
http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-populationReferenceUsed to specify the population that triggered a particular evaluatedResource

There's not currently a way to configure which extensions are enabled. All supported extensions are always enabled.

Operation

 

Smile CDR implements the $evaluate-measure operation into three different implementations and use cases.

FeatureStandard Evaluate-MeasureDistributed Evaluate-MeasureSpark Evaluate-Measure
Processing TypeLow Latency ResponseBulk ProcessingBig Data Processing
Population Size<1,000>=1000 to 10's of Millions100's of Millions to Billions
Integrated into CDR✓✓
FHIR R5
FHIR R4✓✓✓
FHIR DSTU3✓
Available in open-source✓
Creates MeasureReport✓✓✓
Persists MeasureReport ✓✓
Horizontally Scalable ✓✓
Populates Reporter Field ✓✓
Evaluation Type 'Subject-List'✓ Coming Soon

Operation Parameters

 
ParameterSupportedDescription
Id✓the id of the Measure to evaluate
periodStart✓The start of the reporting period
periodEnd✓The end of the reporting period
reportType✓The type of measure report: subject, subject-list, or population. If not specified, a default value of subject will be used if the subject parameter is supplied, otherwise, population will be used
subject✓Subject for which the measure will be calculated. If not specified, the measure will be calculated for all subjects that meet the requirements of the measure. If specified, the measure will only be calculated for the referenced subject(s)
practitioner✓Practitioner for which the measure will be calculated. If specified, the measure will be calculated only for subjects that have a primary relationship to the identified practitioner
lastReceivedOn✓The date the results of this measure were last received. This parameter is only valid for patient-level reports and is used to indicate when the last time a result for this patient was received. This information can be used to limit the set of resources returned for a patient-level report
productLine✓the productLine (e.g. Medicare, Medicaid, etc) to use for the evaluation. This is a non-standard parameter
additionalData✓the data bundle containing additional data
terminologyEndpoint
parameters✓Any input parameters for the evaluation. Parameters defined in this input will be made available by name to the CQL expression. Parameter types are mapped to CQL as specified in the Using CQL topic of the Clinical Reasoning Module. If a parameter appears more than once in the input Parameters resource, it is represented with a List in the input CQL. If a parameter has parts, it is represented as a Tuple in the input CQL.

Reporting Period

The periodStart and periodEnd parameters are used to control the Reporting Period for which a report is generated. This corresponds to Measurement Period defined in the CQL logic, as defined by the conformance requirements in the CQF Measures IG. Both periodStart and periodEnd must be used or neither must be used.

If neither are used the default reporting period specified in the CQL logic is used, as shown here

parameter "Measurement Period" Interval<DateTime>
  default Interval[@2019-01-01T00:00:00.0, @2020-01-01T00:00:00.0)

If neither are used and there is no default reporting period in the CQL logic an error is thrown.

A request using periodStart and periodEnd looks like:

GET{fhir-endpoint}/Measure/<MeasureId>/$evaluate-measure?periodStart=2019-01-01&periodEnd=2019-12-31

periodStart and periodEnd support Dates (YYYY, YYYY-MM, or YYYY-MM-DD) and DateTimes (YYYY-MM-DDThh:mm:ss). DateTime formats of YYYY-MM-DDThh:mm:ss+zz no longer accepted. To pass in timezones to period queries, please see the Headers section below:

Headers with FHIR Requests

 

The behaviour of both periodStart and periodEnd depend on the value of the Timezone header. The measure report will be queried according to the period range, as denoted by that timezone, NOT, the server timezone.

Accepted values for this header are documented here: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones

ex: Timezone:American/Denver will set the timezone to Mountain Time.

If the client provides nothing in this header, the timezone will default to UTC.

Please consult the below table for examples of various combinations of start, end, and timezone, as well as the resulting queried periods:

request timezonestartendconverted startconverted end
(unset)(unset)(unset)N/AN/A
(unset)202020212020-01-01T00:00:00Z2021-12-31T23:59:59Z
Z202020212020-01-01T00:00:00Z2021-12-31T23:59:59Z
UTC202020212020-01-01T00:00:00Z2021-12-31T23:59:59Z
America/St_Johns202020212020-01-01T00:00:00-03:302021-12-31T23:59:59-03:30
America/Toronto202020212020-01-01T00:00:00-05:002021-12-31T23:59:59-05:00
America/Denver202020212020-01-01T00:00:00-07:002021-12-31T23:59:59-07:00
(unset)2022-022022-082022-02-01T00:00:00Z2022-08-31T23:59:59Z
UTC2022-022022-082022-02-01T00:00:00Z2022-08-31T23:59:59Z
America/St_Johns2022-022022-082022-02-01T00:00:00-03:302022-08-31T23:59:59-02:30
America/Toronto2022-022022-082022-02-01T00:00:00-05:002022-08-31T23:59:59-04:00
America/Denver2022-022022-082022-02-01T00:00:00-07:002022-08-31T23:59:59-06:00
(unset)2024-02-252024-02-262024-02-25T00:00:00Z2024-02-26T23:59:59Z
UTC2024-02-252024-02-262024-02-25T00:00:00Z2024-02-26T23:59:59Z
America/St_Johns2024-02-252024-02-262024-02-25T00:00:00-03:302024-02-26T23:59:59-03:30
America/Toronto2024-02-252024-02-262024-02-25T00:00:00-05:002024-02-26T23:59:59-05:00
America/Denver2024-02-252024-02-262024-02-25T00:00:00-07:002024-02-26T23:59:59-07:00
(unset)2024-09-252024-09-262024-09-25T00:00:00Z2024-09-26T23:59:59Z
UTC2024-09-252024-09-262024-09-25T00:00:00Z2024-09-26T23:59:59Z
America/St_Johns2024-09-252024-09-262024-09-25T00:00:00-02:302024-09-26T23:59:59-02:30
America/Toronto2024-09-252024-09-262024-09-25T00:00:00-04:002024-09-26T23:59:59-04:00
America/Denver2024-09-252024-09-262024-09-25T00:00:00-06:002024-09-26T23:59:59-06:00
(unset)2024-09-25T12:00:002024-09-26T12:00:002024-09-25T12:00:00-06:002024-09-26T11:59:59-06:00
Z2024-09-25T12:00:002024-09-26T12:00:002024-09-25T12:00:00-06:002024-09-26T11:59:59-06:00
UTC2024-09-25T12:00:002024-09-26T12:00:002024-09-25T12:00:00-06:002024-09-26T11:59:59-06:00
America/St_Johns2024-09-25T12:00:002024-09-26T12:00:002024-09-25T12:00:00-02:302024-09-26T11:59:59-02:30
America/Toronto2024-09-25T12:00:002024-09-26T12:00:002024-09-25T12:00:00-04:002024-09-26T11:59:59-04:00
America/Denver2024-09-25T12:00:002024-09-26T12:00:002024-09-25T12:00:00-06:002024-09-26T11:59:59-06:00

Report Types

Measure report types determine what data is returned from the evaluation. This is controlled with the reportType parameter on the $evaluate-measure Operation

Report TypeSupportedDescription
subject*✓Measure report for a single subject (e.g. one patient). Includes additional detail, such as evaluatedResources
subject-list✓Measure report including the list of subjects in each population (e.g. all the patients in the "numerator")
population✓Summary measure report for a population

NOTE:

A request using reportType looks like:

GET{fhir-endpoint}/Measure/<MeasureId>/$evaluate-measure?reportType=subject-list

Measure Subject Types

 

The subject of a measure evaluation is controlled with the subject (R4+) and patient (DSTU3) operation parameters. Currently, the only subject type supported by Smile CDR is Patient. This means that all Measure evaluation and reporting happens with respect to a Patient or set of Patient resources.

Subject TypeSupportedDescription
Patient✓A Patient
Practitioner✓A Practitioner
Organization✓An Organization
Location A Location
Device A Device
Group✓A set of subjects

Selecting Measure Subject for a set of Patients

The set of Patients for Measure evaluation is defined by either the subject or practitioner to act as a reference to a subset of Patients defined by various Patient attribution models or if left empty, the entire set of Patients. This section and it's sub-sections will detail how Patients are attributed to the accepted subject references provided to measure evaluation operations.

The set of Patients used for Measure evaluation is controlled with the subject (R4+) or patient (DSTU3), and practitioner parameters. The two parameters are mutually exclusive.

ParameterSupported CDRSupported Open-SourceDescription
Not specified✓✓All Patients on the server
subject=XXX or subject=Patient/XXX✓✓A single Patient
practitioner=XXX or practitioner=Practitioner/XXX✓✓All Patients whose generalPractitioner is the referenced Practitioner
practitioner=Group/XXX✓✓All Patients whose generalPractitioner is the referenced one of the practitioners in Group
subject=Group/XXX✓✓All Patients who is a member of referenced Group of type='person'
subject=Organization/XXX✓ All Patients whose managingOrganization is the referenced Organization

Set of Patients by Patient Reference

When users build a request with parameter subject=Patient/XXX the reference links to Patient resources directly by the Patient.id field.

diagram

Set of Patients by Person Group Reference

When users build a request with parameter subject=Group/XXX and Group.type=person the reference links to Patient resources from Group.memberEntity references to a Patient.id.

diagram

Set of Patients by Practitioner Reference

When users build a request with parameter subject=Practitioner/XXX or practitioner=Practitioner/XXX this links to Patient resources by the Patient.generalPractitioner field.

diagram

Set of Patients by Practitioner Group Reference

When users build a request with parameter subject=Group/XXX or practitioner=Group/XXX, and Group.type=pratitioner the reference links to Practitioner resources from Group.memberEntity references to a Practitioner.id and to Patient resources by the Patient.generalPractitioner field.

diagram

Set of Patients by Organization Reference

When users build a request with parameter subject=Organization/XXX the reference links to Patient resources by the Patient.managingOrganization field.

When DQM Module configuration for 'Include Patients from Related Managing Organizations' is set to true, an additional collection of Organization resources is done by Organizations that have a matching reference to subject parameter by the Organization.partOf field. This then connects to additional Patient resources by the Patient.managingOrganization field.

diagram

Set of Patients by Empty Reference

When users build a request with parameter subject= the empty reference will not filter for specific Patients and collect all that are available in the system.

ReportType, Subject, Practitioner Matrix

The following table shows the combinations of the subject (or patient), practitioner and reportType parameters that are valid

subject reportTypesubject-list reportTypepopulation reportType
subject parameter✓✓ 1,2✓ 1,2
practitioner parameter3✓✓
  1. Including the subject parameter restricts the Measure evaluation to a single Patient. Omit the subject (or patient) parameter to get report for multiple Patients. The subject-list and population report types have less detail than a subject report.
  2. A Group subject with a subject-list or population reportType will be a valid combination once Group support is implemented.
  3. A practitioner have may zero, one, or many patients so a practitioner report always assumes a set.

Last Received On

The lastReceivedOn parameter is the date the Measure was evaluated and reported. It is used to limit the number of resources reported in the Measure report for individual reports. It is currently not supported by Smile CDR.

Parameters

Any input parameters for the evaluation. Parameters defined in this input will be made available by name to the CQL expression. Parameter types are mapped to CQL as specified in the Using CQL topic of the Clinical Reasoning Module. If a parameter appears more than once in the input Parameters resource, it is represented with a List in the input CQL. If a parameter has parts, it is represented as a Tuple in the input CQL.

example of 'parameters' parameter passed into the body of an $evaluate-measure operation. This specifies the FHIR 'id' of an encounter resource into a CQL parameter.

{
	"resourceType": "Parameters",
	"parameter": [
		{
			"name": "encounter",
			"valueString": "2d0ecfb4-9dec-4daa-a261-e37e426d0d7b"
		}
	]
}

Measure Report Resource

 

The outcome of running $evaluate-measure returns a Measure Report resource with the results outputting in the structure defined in the Measure resource.

Example Measure Report Individual

Example requests and outputs with parameter reportType=subject.

Example Individual Request

GET http://your-server-base/fhir/Measure/ColorectalCancerScreeningsFHIR/$evaluate-measure?periodStart=2023-01-01&periodEnd=2023-12-31&reportType=subject&subject=Patient/numer-EXM130-Patient-1

Example Individual Response

{
   "resourceType": "MeasureReport",
   "language": "en",
   "contained": [
      {
         "resourceType": "Observation",
         "id": "70e1db58-acc7-4d06-b9d1-1139164bfb0d",
         "extension": [
            {
               "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
               "extension": [
                  {
                     "url": "measure",
                     "valueCanonical": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR"
                  }
               ]
            }
         ],
         "status": "final",
         "code": {
            "coding": [
               {
                  "system": "http://terminology.hl7.org/CodeSystem/measure-data-usage",
                  "code": "supplemental-data"
               }
            ]
         },
         "valueCodeableConcept": {
            "coding": [
               {
                  "code": "M"
               }
            ]
         }
      },
      {
         "resourceType": "Observation",
         "id": "3f773d33-d40f-4b76-9de9-dd5af88f8924",
         "extension": [
            {
               "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
               "extension": [
                  {
                     "url": "measure",
                     "valueCanonical": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR"
                  }
               ]
            }
         ],
         "status": "final",
         "code": {
            "coding": [
               {
                  "system": "http://terminology.hl7.org/CodeSystem/measure-data-usage",
                  "code": "supplemental-data"
               }
            ]
         },
         "valueCodeableConcept": {
            "coding": [
               {
                  "code": "2135-2"
               }
            ]
         }
      },
      {
         "resourceType": "Observation",
         "id": "cd1924a6-15e5-448f-9b90-3de1bd1f4efe",
         "extension": [
            {
               "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
               "extension": [
                  {
                     "url": "measure",
                     "valueCanonical": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR"
                  }
               ]
            }
         ],
         "status": "final",
         "code": {
            "coding": [
               {
                  "system": "http://terminology.hl7.org/CodeSystem/measure-data-usage",
                  "code": "supplemental-data"
               }
            ]
         },
         "valueCodeableConcept": {
            "coding": [
               {
                  "code": "2028-9"
               }
            ]
         }
      }
   ],
   "extension": [
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.population.description",
         "valueString": "Percentage of adults 50-75 years of age who had appropriate screening for colorectal cancer"
      },
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.supplementalDataElement.reference",
         "valueReference": {
            "extension": [
               {
                  "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
                  "valueString": "5C3ACF2A-15D2-44A0-A83D-F4FCC3E2F27A"
               }
            ],
            "reference": "#cd1924a6-15e5-448f-9b90-3de1bd1f4efe"
         }
      },
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.supplementalDataElement.reference",
         "valueReference": {
            "extension": [
               {
                  "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
                  "valueString": "9CB0299E-74B0-4425-8758-52D6FC0F979C"
               }
            ],
            "reference": "#3f773d33-d40f-4b76-9de9-dd5af88f8924"
         }
      },
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.supplementalDataElement.reference",
         "valueReference": {
            "extension": [
               {
                  "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
                  "valueString": "8114BAD7-FAC8-4E22-91CB-33BC3DDB9986"
               }
            ],
            "reference": "#70e1db58-acc7-4d06-b9d1-1139164bfb0d"
         }
      }
   ],
   "status": "complete",
   "type": "individual",
   "measure": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR|0.0.003",
   "subject": {
      "reference": "Patient/numer-EXM130-Patient-1"
   },
   "date": "2024-03-05T19:30:24+00:00",
   "period": {
      "start": "2023-01-01T00:00:00+00:00",
      "end": "2023-12-31T23:59:59+00:00"
   },
   "improvementNotation": {
      "coding": [
         {
            "system": "http://terminology.hl7.org/CodeSystem/measure-improvement-notation",
            "code": "increase"
         }
      ]
   },
   "group": [
      {
         "extension": [
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-cqfm-denominator-membership",
               "valueString": "1"
            },
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-cqfm-numerator-membership",
               "valueString": "1"
            }
         ],
         "population": [
            {
               "id": "178DA8D8-0694-4B88-8FFE-42CE671EEE35",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "initial-population",
                        "display": "Initial Population"
                     }
                  ]
               },
               "count": 1
            },
            {
               "id": "0AC3911A-2ADC-4DA4-BEBF-545FF8D6D819",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "denominator",
                        "display": "Denominator"
                     }
                  ]
               },
               "count": 1
            },
            {
               "id": "67EABB9C-ADCF-4593-A8DA-35FF25DA594C",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "denominator-exclusion",
                        "display": "Denominator Exclusion"
                     }
                  ]
               },
               "count": 0
            },
            {
               "id": "14B66980-07F4-4872-83AF-C425C379B971",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "numerator",
                        "display": "Numerator"
                     }
                  ]
               },
               "count": 1
            }
         ],
         "measureScore": {
            "value": 1.0
         }
      }
   ],
   "evaluatedResource": [
      {
         "extension": [
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
               "valueString": "178DA8D8-0694-4B88-8FFE-42CE671EEE35"
            },
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
               "valueString": "0AC3911A-2ADC-4DA4-BEBF-545FF8D6D819"
            },
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
               "valueString": "67EABB9C-ADCF-4593-A8DA-35FF25DA594C"
            },
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
               "valueString": "9CB0299E-74B0-4425-8758-52D6FC0F979C"
            },
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
               "valueString": "5C3ACF2A-15D2-44A0-A83D-F4FCC3E2F27A"
            },
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
               "valueString": "8114BAD7-FAC8-4E22-91CB-33BC3DDB9986"
            }
         ],
         "reference": "Patient/numer-EXM130-Patient-1"
      },
      {
         "extension": [
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
               "valueString": "178DA8D8-0694-4B88-8FFE-42CE671EEE35"
            },
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
               "valueString": "0AC3911A-2ADC-4DA4-BEBF-545FF8D6D819"
            },
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
               "valueString": "67EABB9C-ADCF-4593-A8DA-35FF25DA594C"
            }
         ],
         "reference": "Encounter/numer-EXM130-Encounter-1"
      },
      {
         "extension": [
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
               "valueString": "14B66980-07F4-4872-83AF-C425C379B971"
            }
         ],
         "reference": "Procedure/numer-EXM130-1"
      }
   ]
}

Example Measure Report Summary

Example requests and outputs with parameter reportType=population.

Example Summary Request

GET http://your-server-base/fhir/Measure/BreastCancerScreeningFHIR/$evaluate-measure?periodStart=2019-01-01&periodEnd=2019-12-31&subject=numerator&reportType=population

Example Summary Response

{
   "resourceType": "MeasureReport",
   "language": "en",
   "contained": [
      {
         "resourceType": "Observation",
         "id": "5203d419-bb0b-4fe5-b5a7-83f761b0f13e",
         "extension": [
            {
               "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
               "extension": [
                  {
                     "url": "measure",
                     "valueCanonical": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR"
                  }
               ]
            }
         ],
         "status": "final",
         "code": {
            "coding": [
               {
                  "code": "F"
               }
            ]
         },
         "valueInteger": 2
      },
      {
         "resourceType": "Observation",
         "id": "ccf81b4d-4440-44ef-91ff-101b0758eded",
         "extension": [
            {
               "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
               "extension": [
                  {
                     "url": "measure",
                     "valueCanonical": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR"
                  }
               ]
            }
         ],
         "status": "final",
         "code": {
            "coding": [
               {
                  "code": "2135-2"
               }
            ]
         },
         "valueInteger": 6
      },
      {
         "resourceType": "Observation",
         "id": "258b869e-b4da-441c-93d4-fc64c71bb49a",
         "extension": [
            {
               "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
               "extension": [
                  {
                     "url": "measure",
                     "valueCanonical": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR"
                  }
               ]
            }
         ],
         "status": "final",
         "code": {
            "coding": [
               {
                  "code": "2028-9"
               }
            ]
         },
         "valueInteger": 6
      },
      {
         "resourceType": "Observation",
         "id": "23995bb2-bc5e-4ff5-8509-17dacfe6b375",
         "extension": [
            {
               "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
               "extension": [
                  {
                     "url": "measure",
                     "valueCanonical": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR"
                  }
               ]
            }
         ],
         "status": "final",
         "code": {
            "coding": [
               {
                  "code": "M"
               }
            ]
         },
         "valueInteger": 4
      }
   ],
   "extension": [
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.population.description",
         "valueString": "Percentage of adults 50-75 years of age who had appropriate screening for colorectal cancer"
      },
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.supplementalDataElement.reference",
         "valueReference": {
            "extension": [
               {
                  "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
                  "valueString": "5C3ACF2A-15D2-44A0-A83D-F4FCC3E2F27A"
               }
            ],
            "reference": "#258b869e-b4da-441c-93d4-fc64c71bb49a"
         }
      },
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.supplementalDataElement.reference",
         "valueReference": {
            "extension": [
               {
                  "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
                  "valueString": "8114BAD7-FAC8-4E22-91CB-33BC3DDB9986"
               }
            ],
            "reference": "#23995bb2-bc5e-4ff5-8509-17dacfe6b375"
         }
      },
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.supplementalDataElement.reference",
         "valueReference": {
            "extension": [
               {
                  "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
                  "valueString": "9CB0299E-74B0-4425-8758-52D6FC0F979C"
               }
            ],
            "reference": "#ccf81b4d-4440-44ef-91ff-101b0758eded"
         }
      },
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.supplementalDataElement.reference",
         "valueReference": {
            "extension": [
               {
                  "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
                  "valueString": "8114BAD7-FAC8-4E22-91CB-33BC3DDB9986"
               }
            ],
            "reference": "#5203d419-bb0b-4fe5-b5a7-83f761b0f13e"
         }
      }
   ],
   "status": "complete",
   "type": "summary",
   "measure": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR|0.0.003",
   "date": "2024-03-05T19:33:25+00:00",
   "period": {
      "start": "2023-01-01T00:00:00+00:00",
      "end": "2023-12-31T23:59:59+00:00"
   },
   "improvementNotation": {
      "coding": [
         {
            "system": "http://terminology.hl7.org/CodeSystem/measure-improvement-notation",
            "code": "increase"
         }
      ]
   },
   "group": [
      {
         "extension": [
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-cqfm-denominator-membership",
               "valueString": "2"
            },
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-cqfm-numerator-membership",
               "valueString": "1"
            }
         ],
         "population": [
            {
               "id": "178DA8D8-0694-4B88-8FFE-42CE671EEE35",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "initial-population",
                        "display": "Initial Population"
                     }
                  ]
               },
               "count": 2
            },
            {
               "id": "0AC3911A-2ADC-4DA4-BEBF-545FF8D6D819",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "denominator",
                        "display": "Denominator"
                     }
                  ]
               },
               "count": 2
            },
            {
               "id": "67EABB9C-ADCF-4593-A8DA-35FF25DA594C",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "denominator-exclusion",
                        "display": "Denominator Exclusion"
                     }
                  ]
               },
               "count": 0
            },
            {
               "id": "14B66980-07F4-4872-83AF-C425C379B971",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "numerator",
                        "display": "Numerator"
                     }
                  ]
               },
               "count": 1
            }
         ],
         "measureScore": {
            "value": 0.5
         }
      }
   ]
}

Example Measure Report Subject-List

Example requests and outputs with parameter reportType=subject-list.

Example Subject-List Request

GET http://your-server-base/fhir/Measure/ColorectalCancerScreeningsFHIR/$evaluate-measure?periodStart=2023-01-01&periodEnd=2023-12-31&reportType=subject-list&subject=Patient/numer-EXM130-Patient-1

Example Subject-List Response

{
   "resourceType": "MeasureReport",
   "language": "en",
   "contained": [
      {
         "resourceType": "Observation",
         "id": "9ed35e34-9270-49e2-bf71-ca979fb2bcbf",
         "extension": [
            {
               "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
               "extension": [
                  {
                     "url": "measure",
                     "valueCanonical": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR"
                  }
               ]
            }
         ],
         "status": "final",
         "code": {
            "coding": [
               {
                  "code": "2135-2"
               }
            ]
         },
         "valueInteger": 1
      },
      {
         "resourceType": "List",
         "id": "d11fe780-66e2-4ab5-aff6-63028665114a",
         "entry": [
            {
               "item": {
                  "reference": "numer-EXM130-Patient-1"
               }
            }
         ]
      },
      {
         "resourceType": "Observation",
         "id": "428128d9-9e4f-4fe5-999f-b2748032c5dd",
         "extension": [
            {
               "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
               "extension": [
                  {
                     "url": "measure",
                     "valueCanonical": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR"
                  }
               ]
            }
         ],
         "status": "final",
         "code": {
            "coding": [
               {
                  "code": "2028-9"
               }
            ]
         },
         "valueInteger": 1
      },
      {
         "resourceType": "List",
         "id": "ca82c41b-e668-477d-80ae-b545986e013f",
         "entry": [
            {
               "item": {
                  "reference": "numer-EXM130-Patient-1"
               }
            }
         ]
      },
      {
         "resourceType": "List",
         "id": "80873504-6c9b-4d85-a0de-2b8c36962327",
         "entry": [
            {
               "item": {
                  "reference": "numer-EXM130-Patient-1"
               }
            }
         ]
      },
      {
         "resourceType": "Observation",
         "id": "56a28519-939e-41bd-99c9-1e6fddf600da",
         "extension": [
            {
               "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
               "extension": [
                  {
                     "url": "measure",
                     "valueCanonical": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR"
                  }
               ]
            }
         ],
         "status": "final",
         "code": {
            "coding": [
               {
                  "code": "M"
               }
            ]
         },
         "valueInteger": 1
      }
   ],
   "extension": [
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.population.description",
         "valueString": "Percentage of adults 50-75 years of age who had appropriate screening for colorectal cancer"
      },
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.supplementalDataElement.reference",
         "valueReference": {
            "extension": [
               {
                  "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
                  "valueString": "8114BAD7-FAC8-4E22-91CB-33BC3DDB9986"
               }
            ],
            "reference": "#56a28519-939e-41bd-99c9-1e6fddf600da"
         }
      },
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.supplementalDataElement.reference",
         "valueReference": {
            "extension": [
               {
                  "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
                  "valueString": "9CB0299E-74B0-4425-8758-52D6FC0F979C"
               }
            ],
            "reference": "#9ed35e34-9270-49e2-bf71-ca979fb2bcbf"
         }
      },
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.supplementalDataElement.reference",
         "valueReference": {
            "extension": [
               {
                  "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
                  "valueString": "5C3ACF2A-15D2-44A0-A83D-F4FCC3E2F27A"
               }
            ],
            "reference": "#428128d9-9e4f-4fe5-999f-b2748032c5dd"
         }
      }
   ],
   "status": "complete",
   "type": "subject-list",
   "measure": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR|0.0.003",
   "date": "2024-03-05T19:31:47+00:00",
   "period": {
      "start": "2023-01-01T00:00:00+00:00",
      "end": "2023-12-31T23:59:59+00:00"
   },
   "improvementNotation": {
      "coding": [
         {
            "system": "http://terminology.hl7.org/CodeSystem/measure-improvement-notation",
            "code": "increase"
         }
      ]
   },
   "group": [
      {
         "extension": [
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-cqfm-denominator-membership",
               "valueString": "1"
            },
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-cqfm-numerator-membership",
               "valueString": "1"
            }
         ],
         "population": [
            {
               "id": "178DA8D8-0694-4B88-8FFE-42CE671EEE35",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "initial-population",
                        "display": "Initial Population"
                     }
                  ]
               },
               "count": 1,
               "subjectResults": {
                  "reference": "#d11fe780-66e2-4ab5-aff6-63028665114a"
               }
            },
            {
               "id": "0AC3911A-2ADC-4DA4-BEBF-545FF8D6D819",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "denominator",
                        "display": "Denominator"
                     }
                  ]
               },
               "count": 1,
               "subjectResults": {
                  "reference": "#80873504-6c9b-4d85-a0de-2b8c36962327"
               }
            },
            {
               "id": "67EABB9C-ADCF-4593-A8DA-35FF25DA594C",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "denominator-exclusion",
                        "display": "Denominator Exclusion"
                     }
                  ]
               },
               "count": 0
            },
            {
               "id": "14B66980-07F4-4872-83AF-C425C379B971",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "numerator",
                        "display": "Numerator"
                     }
                  ]
               },
               "count": 1,
               "subjectResults": {
                  "reference": "#ca82c41b-e668-477d-80ae-b545986e013f"
               }
            }
         ],
         "measureScore": {
            "value": 1.0
         }
      }
   ]
}

Measure Report Populations And Scores

 

When looking at single-rate or Multi-rate (more than one group) measures that have been evaluated it is important to understand what you are looking at in the counts for each population.

We learned in population-basis that this could represent a patient resource or represent resources of a subject like an encounter, but each population is not simply a raw expression output of CQL, each scoring-method defines the structure of each Measure Report as well as how population membership is evaluated and scored.

These scoring algorithms are defined in the following IG.

The intent of the MeasureReport population counts is to display how each population was evaluated to the scoring algorithm for the specified scoring type. Allowing end users the ability to compute the measure "score" using the values from population counts.

If a user ever needed to understand the raw output of a population expression, as a means to audit the algorithm, they could use the available Library $evaluate operation.

The tables below express in set logic how each scoring method would evaluate from raw CQL expression output to eventual "score".

  • MeasureReport Populations - population names found in Measure Report
  • Criteria Expression Result Set - Set representation of criteria expression output, raw results
  • Evaluated Population Membership Set - Set representation of evaluator algorithm
  • Criteria Expression Result Count - Count representation of criteria expression output, raw results
  • Evaluated Population Membership Count representation of evaluator algorithm, the values shown on report

Proportion & Ratio Measure

MeasureReport PopulationsCriteria Expression Result SetEvaluated Population Membership SetCriteria Expression Result CountEvaluated Population Membership Count
Initial PopulationA,B,C,D,EA,B,C,D,E55
DenominatorA,B,C,D,EB,E52
Denominator ExclusionAA11
Denominator Exception*B,CC21
Numerator ExclusionDD11
NumeratorB,D,EB,E32
MeasureScore 1.0
  • Denominator Exception does not apply to Ratio measures

Continuous Variable Measure

MeasureReport PopulationsCriteria Expression Result SetEvaluated Population Membership SetCriteria Expression Result CountEvaluated Population Membership Count
InitialPopulationA,B,C,D,EA,B,C,D,E55
MeasurePopulationA,B,C,D,EB,C,D,E54
MeasurePopulationExclusionAA11
Observation*NAB11
TotalMeasurePopulation 4

Cohort Measure

MeasureReport PopulationsCriteria Expression Result SetEvaluated Population Membership SetCriteria Expression Result CountEvaluated Population Membership Count
InitialPopulationA,B,C,D,EA,B,C,D,E55
MeasureScore NA

Multiple Measure Evaluation

 

The $evaluate-measures operation is used to execute more than one measure resource per selected subject population. This behaves similarly to the $evalaute-measure operation in that it has a "Distributed" and "Standard" mode for low latency and batch processing, but returns a Parameters resource containing one or more Bundle resources each containing one of more MeasureReports instead of a single MeasureReport resource.

For population/summary and subject-list evaluation types, there will be a single contained Bundle container for all Measure Reports. For a subject/individual evaluation type, there will be one Bundle per resolved subject. In the case of a subject/individual measure evaluation that resolves to 5 subjects, there will be 5 Bundles within the returned Parameters resource, each Bundle containing all Measure Reports associated with that subject.

This entire mechanism is designed to assist any users that will perform the measure evaluations for the same population against multiple measures.

  • One use case could be evaluating a provider's population against a specialty measurement-set. Instead of multiple requests of $evaluate-measure, one for each measure, you can now have a single request to $evaluate-measures that contains all run results in a contained Parameters resource.

Multiple Measure Operation Parameters

 
ParameterSupportedDescription
measureId✓the IDs of the Measure to evaluate
measureIdentifier✓the Identifier System & value or just value that resolves to the Measure resource
measureUrl✓the canonical url of the Measure to evaluate
periodStart✓The start of the reporting period
periodEnd✓The end of the reporting period
reportType✓The type of measure report: subject, subject-list, or population. If not specified, a default value of subject will be used if the subject parameter is supplied, otherwise, population will be used.
subject✓Subject for which the measure will be calculated. If not specified, the measure will be calculated for all subjects that meet the requirements of the measure. If specified, the measure will only be calculated for the referenced subject(s)
productLine✓the productLine (e.g. Medicare, Medicaid, etc) to use for the evaluation. This is a non-standard parameter
additional Data✓the data bundle containing additional data
terminology Endpoint
parameters✓Any input parameters for the evaluation. Parameters defined in this input will be made available by name to the CQL expression. Parameter types are mapped to CQL as specified in the Using CQL topic of the Clinical Reasoning Module. If a parameter appears more than once in the input Parameters resource, it is represented with a List in the input CQL. If a parameter has parts, it is represented as a Tuple in the input CQL.
reporter✓optional parameter to pass in a reference to a resource that the user would like to have populated on Measure Report reporter field.

Parameters of Bundles of Measure Reports

For each measureId parameter added to the operation request, a subsequent Measure Report resource will be added to the returned Parameters resource created by $evaluate-measures request.

For example, in order to evaluate six measure resources for a selected population (subject or practitioner parameters), then the operation will evaluate that same population against each of those measures and return a single Measure Report back for each passed in Measure.id.

Measure Report Reporter Parameter

This feature also allows for an additional parameter to the standard $evaluate-measure, allowing callers to pass in a compliant resource for Measure Report reporter.

This can be used instead of dQM module configurations for default reporter and/or use Group Managing Entity.

  • The reporter parameter will override these configurations (if set), and persist the parameter value to each Measure Report created.

Multiple Measure Evaluation Example Request

In each example, the subject is Group/XYZ. This group resolves to 3 patients, Patient/ABC, Patient/DEF, and Patient/GHI.

Subject Report Type

  • "Standard" Mode Type Request
GET{fhir-endpoint}/Measure/$evaluate-measures?reportType=subject&subject=Group/XYZ&reporter=Organization/123&measureId=<MeasureId-1>&measureId=<MeasureId-2>&measureId=<MeasureId-3>
  • "Distributed" Mode Type Request
POST{fhir-endpoint}/Measure/$evaluate-measures
Prefer: respond-async


{
	"resourceType": "Parameters",
	"parameter": [
		{
			"name": "reportType",
			"valueString": "subject"
		},
		{
			"name": "subject",
			"valueString": "Group/XYZ"
		},
		{
			"name": "reporter",
			"valueString": "Organization/123"
		},
		{
			"name": "measureId",
			"valueString": "<MeasureId-1>"
		},
		{
			"name": "measureId",
			"valueString": "<MeasureId-2>"
		},
		{
			"name": "measureId",
			"valueString": "<MeasureId-3>"
		}
	]
}

Population Report Type

  • "Standard" Mode Type Request
GET{fhir-endpoint}/Measure/$evaluate-measures?reportType=population&subject=Group/XYZ&reporter=Organization/123&measureId=<MeasureId-1>&measureId=<MeasureId-2>&measureId=<MeasureId-3>
  • "Distributed" Mode Type Request
POST{fhir-endpoint}/Measure/$evaluate-measures
Prefer: respond-async


{
	"resourceType": "Parameters",
	"parameter": [
		{
			"name": "reportType",
			"valueString": "population"
		},
		{
			"name": "subject",
			"valueString": "Group/XYZ"
		},
		{
			"name": "reporter",
			"valueString": "Organization/123"
		},
		{
			"name": "measureId",
			"valueString": "<MeasureId-1>"
		},
		{
			"name": "measureId",
			"valueString": "<MeasureId-2>"
		},
		{
			"name": "measureId",
			"valueString": "<MeasureId-3>"
		}
	]
}

Subject-List Report Type

  • "Standard" Mode Type Request
GET{fhir-endpoint}/Measure/$evaluate-measures?reportType=subject-list&subject=Group/XYZ&reporter=Organization/123&measureId=<MeasureId-1>&measureId=<MeasureId-2>&measureId=<MeasureId-3>
  • "Distributed" Mode Type Request
POST{fhir-endpoint}/Measure/$evaluate-measures
Prefer: respond-async


{
	"resourceType": "Parameters",
	"parameter": [
		{
			"name": "reportType",
			"valueString": "subject-list"
		},
		{
			"name": "subject",
			"valueString": "Group/XYZ"
		},
		{
			"name": "reporter",
			"valueString": "Organization/123"
		},
		{
			"name": "measureId",
			"valueString": "<MeasureId-1>"
		},
		{
			"name": "measureId",
			"valueString": "<MeasureId-2>"
		},
		{
			"name": "measureId",
			"valueString": "<MeasureId-3>"
		}
	]
}

Multiple Measure Evaluation Example Response

Subject Report Type

  • "Distributed" Mode Response - pending
{
   "resourceType": "Parameters",
   "parameter": [
      {
         "name": "return",
         "resource": {
            "resourceType": "Bundle",
            "id": "bundle-id",
            "identifier": {
               "value": "measure-eval-job-id-<job-id>"
            },
            "type": "collection",
            "entry": [
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-1>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-1-Title>"
                        }
                     ],
                     "status": "pending",
                     "measure": "url/Measure/<MeasureId-1>"
                  }
               },
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-2>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-2-Title>"
                        }
                     ],
                     "status": "pending",
                     "measure": "url/Measure/<MeasureId-2>"
                  }
               },
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-3>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-3-Title>"
                        }
                     ],
                     "status": "pending",
                     "measure": "url/Measure/<MeasureId-3>"
                  }
               }
            ]
         }
      }
   ]
}
  • "Distributed" Mode Response - complete
    • bundle and report(s) do persist
{
   "resourceType": "Bundle",
   "id": "bundle-id",
   "identifier": {
      "value": "measure-eval-job-id-<job-id>"
   },
   "type": "collection",
   "entry": [
      {
         "fullUrl": "http://localhost:8000/MeasureReport/<id>",
         "resource": {
            "resourceType": "MeasureReport",
            "id": "<id>",
            "extension": [
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                  "valueString": "<MeasureId-1>"
               },
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                  "valueString": "<MeasureId-1-Title>"
               }
            ],
            "status": "complete",
            "measure": "url/Measure/<MeasureId-1>",
            "date": "2024-05-16T11:51:02-06:00",
            "reporter": {
               "reference": "Organization/123"
            },
            "subject": {
               "reference": "Patient/ABC"
            },
            "period": {
               "start": "2023-01-01T00:00:00-07:00",
               "end": "2023-12-31T23:59:59-07:00"
            }
         }
      },
      {
         "fullUrl": "http://localhost:8000/MeasureReport/<id>",
         "resource": {
            "resourceType": "MeasureReport",
            "id": "<id>",
            "extension": [
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                  "valueString": "<MeasureId-1>"
               },
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                  "valueString": "<MeasureId-1-Title>"
               }
            ],
            "status": "complete",
            "measure": "url/Measure/<MeasureId-1>",
            "date": "2024-05-16T11:51:02-06:00",
            "reporter": {
               "reference": "Organization/123"
            },
            "subject": {
               "reference": "Patient/ABC"
            },
            "period": {
               "start": "2023-01-01T00:00:00-07:00",
               "end": "2023-12-31T23:59:59-07:00"
            }
         }
      }
   ]
}
  • "Standard" Mode Response - complete
    • bundle and report(s) do not persist, returns Bundle Type SearchSet instead of Collection
{
   "resourceType": "Parameters",
   "parameter": [
      {
         "name": "return",
         "resource": {
            "resourceType": "Bundle",
            "id": "bundle-id",
            "identifier": {
               "value": "measure-eval-job-id-<job-id>"
            },
            "type": "collection",
            "entry": [
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-1>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-1-Title>"
                        }
                     ],
                     "status": "complete",
                     "type": "individual",
                     "measure": "url/Measure/<MeasureId-1>",
                     "subject": {
                        "reference": "Patient/<subject1>"
                     }
                  }
               },
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-2>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-2-Title>"
                        }
                     ],
                     "status": "complete",
                     "type": "individual",
                     "measure": "url/Measure/<MeasureId-2>",
                     "subject": {
                        "reference": "Patient/<subject1>"
                     }
                  }
               },
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-3>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-3-Title>"
                        }
                     ],
                     "status": "pending",
                     "measure": "url/Measure/<MeasureId-3>",
                     "subject": {
                        "reference": "Patient/<subject1>"
                     }
                  }
               }
            ]
         }
      },
      {
         "name": "return",
         "resource": {
            "resourceType": "Bundle",
            "id": "bundle-id",
            "identifier": {
               "value": "measure-eval-job-id-<job-id>"
            },
            "type": "collection",
            "entry": [
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-1>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-1-Title>"
                        }
                     ],
                     "status": "complete",
                     "type": "individual",
                     "measure": "url/Measure/<MeasureId-1>",
                     "subject": {
                        "reference": "Patient/<subject2>"
                     }
                  }
               },
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-2>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-2-Title>"
                        }
                     ],
                     "status": "complete",
                     "type": "individual",
                     "measure": "url/Measure/<MeasureId-2>",
                     "subject": {
                        "reference": "Patient/<subject2>"
                     }
                  }
               },
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-3>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-3-Title>"
                        }
                     ],
                     "status": "pending",
                     "measure": "url/Measure/<MeasureId-3>",
                     "subject": {
                        "reference": "Patient/<subject2>"
                     }
                  }
               }
            ]
         }
      },
      {
         "name": "return",
         "resource": {
            "resourceType": "Bundle",
            "id": "bundle-id",
            "identifier": {
               "value": "measure-eval-job-id-<job-id>"
            },
            "type": "collection",
            "entry": [
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-1>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-1-Title>"
                        }
                     ],
                     "status": "complete",
                     "type": "individual",
                     "measure": "url/Measure/<MeasureId-1>",
                     "subject": {
                        "reference": "Patient/<subject3>"
                     }
                  }
               },
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-2>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-2-Title>"
                        }
                     ],
                     "status": "complete",
                     "type": "individual",
                     "measure": "url/Measure/<MeasureId-2>",
                     "subject": {
                        "reference": "Patient/<subject3>"
                     }
                  }
               },
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-3>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-3-Title>"
                        }
                     ],
                     "status": "pending",
                     "measure": "url/Measure/<MeasureId-3>",
                     "subject": {
                        "reference": "Patient/<subject3>"
                     }
                  }
               }
            ]
         }
      }
   ]
}

Population Report Type

  • "Distributed" Mode Response - pending
{
   "resourceType": "Parameters",
   "parameter": [
      {
         "name": "return",
         "resource": {
            "resourceType": "Bundle",
            "id": "bundle-id",
            "identifier": {
               "value": "measure-eval-job-id-<job-id>"
            },
            "type": "collection",
            "entry": [
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-1>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-1-Title>"
                        }
                     ],
                     "status": "pending",
                     "measure": "url/Measure/<MeasureId-1>",
                     "date": "2025-11-27T18:31:37+03:30"
                  }
               },
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-2>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-2-Title>"
                        }
                     ],
                     "status": "pending",
                     "measure": "url/Measure/<MeasureId-2>"
                  }
               },
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-3>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-3-Title>"
                        }
                     ],
                     "status": "pending",
                     "measure": "url/Measure/<MeasureId-3>",
                     "date": "2025-11-27T18:31:37+03:30"
                  }
               }
            ]
         }
      }
   ]
}
  • "Distributed" Mode Response - complete
    • bundle and report(s) do persist
{
   "resourceType": "Bundle",
   "id": "bundle-id",
   "identifier": {
      "value": "measure-eval-job-id-<job-id>"
   },
   "type": "collection",
   "entry": [
      {
         "fullUrl": "http://localhost:8000/MeasureReport/<id>",
         "resource": {
            "resourceType": "MeasureReport",
            "id": "<id>",
            "extension": [
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                  "valueString": "<MeasureId-1>"
               },
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                  "valueString": "<MeasureId-1-Title>"
               }
            ],
            "status": "complete",
            "measure": "url/Measure/<MeasureId-1>",
            "date": "2024-05-16T11:51:02-06:00",
            "reporter": {
               "reference": "Organization/123"
            },
            "subject": {
               "reference": "Practitioner/XYZ"
            },
            "period": {
               "start": "2023-01-01T00:00:00-07:00",
               "end": "2023-12-31T23:59:59-07:00"
            }
         }
      },
      {
         "fullUrl": "http://localhost:8000/MeasureReport/<id>",
         "resource": {
            "resourceType": "MeasureReport",
            "id": "<id>",
            "extension": [
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                  "valueString": "<MeasureId-2>"
               },
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                  "valueString": "<MeasureId-2-Title>"
               }
            ],
            "status": "complete",
            "type": "summary",
            "measure": "url/Measure/<MeasureId-2>",
            "date": "2024-05-16T11:51:02-06:00",
            "reporter": {
               "reference": "Organization/123"
            },
            "subject": {
               "reference": "Practitioner/XYZ"
            },
            "period": {
               "start": "2023-01-01T00:00:00-07:00",
               "end": "2023-12-31T23:59:59-07:00"
            }
         }
      },
      {
         "fullUrl": "http://localhost:8000/MeasureReport/<id>",
         "resource": {
            "resourceType": "MeasureReport",
            "id": "<id>",
            "extension": [
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                  "valueString": "<MeasureId-3>"
               },
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                  "valueString": "<MeasureId-3-Title>"
               }
            ],
            "status": "complete",
            "type": "summary",
            "measure": "url/Measure/<MeasureId-3>",
            "date": "2024-05-16T11:51:02-06:00",
            "reporter": {
               "reference": "Organization/123"
            },
            "subject": {
               "reference": "Practitioner/XYZ"
            },
            "period": {
               "start": "2023-01-01T00:00:00-07:00",
               "end": "2023-12-31T23:59:59-07:00"
            }
         }
      }
   ]
}
  • "Standard" Mode Response - complete
    • bundle and report(s) do not persist, returns Bundle Type SearchSet instead of Collection
{
   "resourceType": "Parameters",
   "parameter": [
      {
         "name": "return",
         "resource": {
            "resourceType": "Bundle",
            "id": "bundle-id",
            "identifier": {
               "value": "measure-eval-job-id-<job-id>"
            },
            "type": "collection",
            "entry": [
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-1>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-1-Title>"
                        }
                     ],
                     "status": "pending",
                     "measure": "url/Measure/<MeasureId-1>",
                     "date": "2025-11-27T10:33:27-05:00"
                  }
               },
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-2>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-2-Title>"
                        }
                     ],
                     "status": "pending",
                     "measure": "url/Measure/<MeasureId-2>",
                     "date": "2025-11-27T10:33:27-05:00"
                  }
               },
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-3>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-3-Title>"
                        }
                     ],
                     "status": "pending",
                     "measure": "url/Measure/<MeasureId-3>",
                     "date": "2025-11-27T10:33:27-05:00"
                  }
               }
            ]
         }
      }
   ]
}

Subject-List Report Type

  • "Distributed" Mode Response - pending
{
   "resourceType": "Parameters",
   "parameter": [
      {
         "name": "return",
         "resource": {
            "resourceType": "Bundle",
            "id": "bundle-id",
            "identifier": {
               "value": "measure-eval-job-id-<job-id>"
            },
            "type": "collection",
            "entry": [
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-1>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-1-Title>"
                        }
                     ],
                     "status": "pending",
                     "measure": "url/Measure/<MeasureId-1>"
                  }
               },
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-2>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-2-Title>"
                        }
                     ],
                     "status": "pending",
                     "measure": "url/Measure/<MeasureId-2>"
                  }
               },
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-3>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-3-Title>"
                        }
                     ],
                     "status": "pending",
                     "measure": "url/Measure/<MeasureId-3>"
                  }
               }
            ]
         }
      }
   ]
}
  • "Distributed" Mode Response - complete
    • bundle and report(s) do persist
{
   "resourceType": "Bundle",
   "id": "bundle-id",
   "identifier": {
      "value": "measure-eval-job-id-<job-id>"
   },
   "type": "collection",
   "entry": [
      {
         "fullUrl": "http://localhost:8000/MeasureReport/<id>",
         "resource": {
            "resourceType": "MeasureReport",
            "id": "<id>",
            "extension": [
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                  "valueString": "<MeasureId-1>"
               },
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                  "valueString": "<MeasureId-1-Title>"
               }
            ],
            "status": "complete",
            "type": "subject-list",
            "measure": "url/Measure/<MeasureId-1>",
            "date": "2024-05-16T11:51:02-06:00",
            "reporter": {
               "reference": "Organization/123"
            },
            "subject": {
               "reference": "Practitioner/XYZ"
            },
            "period": {
               "start": "2023-01-01T00:00:00-07:00",
               "end": "2023-12-31T23:59:59-07:00"
            }
         }
      },
      {
         "fullUrl": "http://localhost:8000/MeasureReport/<id>",
         "resource": {
            "resourceType": "MeasureReport",
            "id": "<id>",
            "extension": [
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                  "valueString": "<MeasureId-2>"
               },
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                  "valueString": "<MeasureId-2-Title>"
               }
            ],
            "status": "complete",
            "type": "subject-list",
            "measure": "url/Measure/<MeasureId-2>",
            "date": "2024-05-16T11:51:02-06:00",
            "reporter": {
               "reference": "Organization/123"
            },
            "subject": {
               "reference": "Practitioner/XYZ"
            },
            "period": {
               "start": "2023-01-01T00:00:00-07:00",
               "end": "2023-12-31T23:59:59-07:00"
            }
         }
      },
      {
         "fullUrl": "http://localhost:8000/MeasureReport/<id>",
         "resource": {
            "resourceType": "MeasureReport",
            "id": "<id>",
            "extension": [
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                  "valueString": "<MeasureId-3>"
               },
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                  "valueString": "<MeasureId-3-Title>"
               }
            ],
            "status": "complete",
            "type": "subject-list",
            "measure": "url/Measure/<MeasureId-3>",
            "date": "2024-05-16T11:51:02-06:00",
            "reporter": {
               "reference": "Organization/123"
            },
            "period": {
               "start": "2023-01-01T00:00:00-07:00",
               "end": "2023-12-31T23:59:59-07:00"
            }
         }
      }
   ]
}
  • "Standard" Mode Response - complete
    • bundle and report(s) do not persist, returns Bundle Type SearchSet instead of Collection
{
   "resourceType": "Parameters",
   "parameter": [
      {
         "name": "return",
         "resource": {
            "resourceType": "Bundle",
            "id": "bundle-id",
            "identifier": {
               "value": "measure-eval-job-id-<job-id>"
            },
            "type": "collection",
            "entry": [
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-1>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-1-Title>"
                        }
                     ],
                     "status": "complete",
                     "type": "individual",
                     "measure": "url/Measure/<MeasureId-1>",
                     "subject": {
                        "reference": "Patient/<subject1>"
                     }
                  }
               },
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-2>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-2-Title>"
                        }
                     ],
                     "status": "complete",
                     "type": "individual",
                     "measure": "url/Measure/<MeasureId-2>",
                     "subject": {
                        "reference": "Patient/<subject1>"
                     }
                  }
               },
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-3>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-3-Title>"
                        }
                     ],
                     "status": "pending",
                     "measure": "url/Measure/<MeasureId-3>",
                     "subject": {
                        "reference": "Patient/<subject1>"
                     }
                  }
               }
            ]
         }
      },
      {
         "name": "return",
         "resource": {
            "resourceType": "Bundle",
            "id": "bundle-id",
            "identifier": {
               "value": "measure-eval-job-id-<job-id>"
            },
            "type": "collection",
            "entry": [
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-1>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-1-Title>"
                        }
                     ],
                     "status": "complete",
                     "type": "individual",
                     "measure": "url/Measure/<MeasureId-1>",
                     "subject": {
                        "reference": "Patient/<subject2>"
                     }
                  }
               },
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-2>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-2-Title>"
                        }
                     ],
                     "status": "complete",
                     "type": "individual",
                     "measure": "url/Measure/<MeasureId-2>",
                     "subject": {
                        "reference": "Patient/<subject2>"
                     }
                  }
               },
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-3>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-3-Title>"
                        }
                     ],
                     "status": "pending",
                     "measure": "url/Measure/<MeasureId-3>",
                     "subject": {
                        "reference": "Patient/<subject2>"
                     }
                  }
               }
            ]
         }
      },
      {
         "name": "return",
         "resource": {
            "resourceType": "Bundle",
            "id": "bundle-id",
            "identifier": {
               "value": "measure-eval-job-id-<job-id>"
            },
            "type": "collection",
            "entry": [
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-1>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-1-Title>"
                        }
                     ],
                     "status": "complete",
                     "type": "individual",
                     "measure": "url/Measure/<MeasureId-1>",
                     "subject": {
                        "reference": "Patient/<subject3>"
                     }
                  }
               },
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-2>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-2-Title>"
                        }
                     ],
                     "status": "complete",
                     "type": "individual",
                     "measure": "url/Measure/<MeasureId-2>",
                     "subject": {
                        "reference": "Patient/<subject3>"
                     }
                  }
               },
               {
                  "fullUrl": "http://localhost:8000/MeasureReport/<id>",
                  "resource": {
                     "resourceType": "MeasureReport",
                     "id": "<id>",
                     "extension": [
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                           "valueString": "<MeasureId-3>"
                        },
                        {
                           "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                           "valueString": "<MeasureId-3-Title>"
                        }
                     ],
                     "status": "pending",
                     "measure": "url/Measure/<MeasureId-3>",
                     "subject": {
                        "reference": "Patient/<subject3>"
                     }
                  }
               }
            ]
         }
      }
   ]
}

Standard Evaluate Measure

 

The $evaluate-measure operation is used to execute a Measure as specified by the relevant FHIR Resources against a subject or set of subjects. This implementation currently focuses primarily on supporting the narrower evaluation requirements defined by the CQF Measures IG. Some support for extensions defined by other IGs is included as well, and the implementation aims to support a wider range of functionality in the future.

Standard Evaluate Measure Example Request

Several example Measures are available in the ecqm-content-r4 IG. Full Bundles with all the required supporting resources are available here. You can download a Bundle and load it on your server as a transaction:

POST http://your-server-base/fhir BreastCancerScreeningFHIR-bundle.json

These Bundles also include example Patient clinical data so once posted Measure evaluation can be invoked with:

GET http://your-server-base/fhir/Measure/BreastCancerScreeningFHIR/$evaluate-measure?periodStart=2019-01-01&periodEnd=2019-12-31&subject=numerator&reportType=subject

Distributed Evaluate Measure

 

Overview

CDR offers an exclusive enhancement to the evaluate-measure operation to allow it to work at scale with significantly larger patient populations. This enhanced version is called "Distributed" mode that leans on CDR's ability to perform parallel processing of larger workloads.

The open-source available "standard" mode allows for performant low-latency response on smaller populations.

To take advantage this functionality you need to have a clustered Smile CDR deployment (see Clustering for more information) and a premium license for the DQM module (see Licensing for more information).

This capability extends to both $evaluate-measure and $evaluate-measures operations.

Note that subject-list mode reports are not supported in distributed mode, unlike in standard evaluate measure, where they are supported.

Please consider which of subject or population mode reports are best to fulfill your needs in distributed mode.

Requirements

  • Distributed evaluate-measure has requirements to merge Measure Reports that get generated in parallel, and in order to perform this, there are different profile requirements to allow this to happen.
  • Multi-Rate Measures with >1 Groups or Stratifiers require an id be populated so merge logic can uniquely relate data elements together. If your resource does not have this, the operation will give you an error.

Distributed Mode Configurations

  • Distributed mode breaks workloads into manageable pieces for parallel processing and exposes options to users to configure optimally to their deployment model.

Distributed Workflow

diagram

Batch 2 Work Chunks

A Batch2 'work chunk' is a task that is processed as a part of a Batch2 job. The distributed measure evaluation operation sits on top of batch2 job structure to efficiently process and distribute a task work load.

  1. build and distribute work-chunks
  2. evaluate work-chunks
  3. aggregate work-chunk results

There are several dQM module configurations that help end users optimally configure how much processing each 'work chunk' will perform.

  • $evaluate-measure in 'distributed mode', allows for sizing 'subject chunk' to define how many patients each 'work chunk' will process for a measure evaluation.
  • $evaluate-meaasures in 'distributed mode', allows for sizing 'subject-chunk' in addition to 'measure-chunk' to define a 'work chunk', since it can evaluate more than one Measure at a time, it means one 'subject chunk' could be split into multiple 'work chunks'.

Subject Chunk Size

  • Controls the size of the subject population that will be processed in a 'work chunk', we call this a "subject chunk".
  • If the configuration was set to 1000 and the total population of subjects retrieved was 10,000. Then 10 'subject chunks' would be created for processing.

See here for Dqm Module Configuration Categories

Measure Chunk Size

  • Controls the number of measure resources each 'work chunk' will evaluate against.
  • A 'Measure Chunk' is processed against a 'subject chunk' to optimize how many
  • If the configuration was set to 4 and the total number of measures was 9, then there would be three 'measure chunks' created
    • measure chunk 1: 1, 2, 3, 4
    • measure chunk 2: 5, 6, 7, 8
    • measure chunk 3: 9

See here for Dqm Module Configuration Categories

User Interface for Requested Jobs

  • Once a 'Distributed Evaluation' is requested from the server, it is viewable via CDR Admin Web Console, under the RunTime/Batch Jobs menu.
  • This will allow you to view progress, kill an ongoing request, and view metadata from the request.

Example Distributed Request

  • The format follows the same parameter convention as Standard mode, but requires a POST request with a JSON Parameters body for the request parameters and an additional header of Prefer: respond-async. If done correctly, you will receive back a report in "Pending" status.
POST {{fhir-endpoint}}/Measure/$evaluate-measures
Prefer: respond-async

{
	"resourceType": "Parameters",
	"parameter": [
		{
			"name": "periodStart",
			"valueString": "2024-01-01"
		},
		{
			"name": "periodEnd",
			"valueString": "2024-12-31"
		},
		{
			"name": "measureId",
			"valueString": "GSDReporting"
		},
		{
			"name": "reportType",
			"valueString": "population"
		},
		{
			"name": "reporter",
			"valueString": "Organization/organization-group-1"
		}
	]
}

Example In-Progress Response

  • Measure Report returned
{
   "resourceType": "MeasureReport",
   "id": "c365bc0a-7b76-4869-a99f-71c8c5a1ffb3",
   "meta": {
      "versionId": "1",
      "lastUpdated": "2024-03-04T09:35:08.802-07:00",
      "source": "#pvlOpkXdI88TcUUV"
   },
   "extension": [
      {
         "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.jobId",
         "valueString": "2849ef30-66cb-4f75-add0-110479e2f365"
      },
      {
         "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
         "valueString": "Measure/ColorectalCancerScreeningsFHIR"
      },
      {
         "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
         "valueString": "Colorectal Cancer ScreeningFHIR"
      }
   ],
   "status": "pending",
   "date": "2024-03-04T09:35:08-07:00"
}

Example Retrieve Completed Report

  • Once the report is completed, it will be available on the MeasureReport.id provided in the 'In-Progress' state. The id is also visible via CDR Admin Web Console Batch jobs screen.
    • Here is an example request with the MeasureReport.id populated.
GET {fhir-endpoint}/MeasureReport/c365bc0a-7b76-4869-a99f-71c8c5a1ffb3
  • Generic Request
GET {fhir-endpoint}/MeasureReport/<MeasureReport.id>

Measure Evaluation Processing Errors

Measure Evaluation Processing Errors Example

This is an example of what a 'Measure Report' resource that was returned to a user when it had a processing error. Note the 'status' changed to 'Error' & 'Operation Outcome' shown in contained resources.

{
   "resourceType": "MeasureReport",
   "id": "measurereport-contained-oo-1",
   "contained": [ {
      "resourceType": "Observation",
      "id": "89af9a11-2bc5-4235-aa89-2338019e7175",
      "extension": [ {
         "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
         "extension": [ {
            "url": "measure",
            "valueCanonical": "http://content.alphora.com/fhir/dqm/Measure/CMS138v11"
         } ]
      } ],
      "status": "final",
      "code": {
         "coding": [ {
            "code": "1586-7"
         } ]
      },
      "valueInteger": 4
   }, {
      "resourceType": "Observation",
      "id": "2ca8fa92-4bb2-407f-8232-887998c34c84",
      "extension": [ {
         "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
         "extension": [ {
            "url": "measure",
            "valueCanonical": "http://content.alphora.com/fhir/dqm/Measure/CMS138v11"
         } ]
      } ],
      "status": "final",
      "code": {
         "coding": [ {
            "code": "1002-5"
         } ]
      },
      "valueInteger": 1
   },{
      "resourceType" : "OperationOutcome",
      "id" : "1",
      "issue" : [ {
         "severity" : "error",
         "code" : "exception",
         "diagnostics" : "Exception for subjectId: Patient/patient-3, Message: multi-component stratifiers are not yet supported."
      } ]
   }],
   "extension": [
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.supplementalDataElement.reference",
         "valueReference": {
            "extension": [ {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
               "valueString": "sde-race"
            } ],
            "reference": "#89af9a11-2bc5-4235-aa89-2338019e7175"
         }
      },
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.supplementalDataElement.reference",
         "valueReference": {
            "extension": [ {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
               "valueString": "sde-race"
            } ],
            "reference": "#2ca8fa92-4bb2-407f-8232-887998c34c84"
         }
      },
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.population.description",
         "valueString": "Percentage of patients aged 18 years and older who were screened for tobacco use one or more times during the measurement period AND who received tobacco cessation intervention during the measurement period or in the six months prior to the measurement period if identified as a tobacco user"
      } ],
   "status": "error",
   "type": "summary",
   "measure": "http://content.alphora.com/fhir/dqm/Measure/CMS138v11|0.0.001",
   "date": "2024-06-25T16:25:25+00:00",
   "reporter": {
      "reference": "Organization/organization-group-1"
   },
   "period": {
      "start": "2023-01-01T00:00:00+00:00",
      "end": "2023-12-31T23:59:59+00:00"
   },
   "improvementNotation": {
      "coding": [ {
         "system": "http://terminology.hl7.org/CodeSystem/measure-improvement-notation",
         "version": "0.1.0",
         "code": "increase",
         "display": "Increased score indicates improvement"
      } ]
   },
   "group": [ {
      "id": "screened",
      "population": [ {
         "id": "group-1-initial-population",
         "code": {
            "coding": [ {
               "system": "http://terminology.hl7.org/CodeSystem/measure-population",
               "code": "initial-population",
               "display": "Initial Population"
            } ]
         },
         "count": 0
      }, {
         "id": "group-1-denominator",
         "code": {
            "coding": [ {
               "system": "http://terminology.hl7.org/CodeSystem/measure-population",
               "code": "denominator",
               "display": "Denominator"
            } ]
         },
         "count": 0
      }, {
         "id": "group-1-numerator",
         "code": {
            "coding": [ {
               "system": "http://terminology.hl7.org/CodeSystem/measure-population",
               "code": "numerator",
               "display": "Numerator"
            } ]
         },
         "count": 0
      }, {
         "id": "group-1-denominator-exclusion",
         "code": {
            "coding": [ {
               "system": "http://terminology.hl7.org/CodeSystem/measure-population",
               "code": "denominator-exclusion",
               "display": "Denominator Exclusion"
            } ]
         },
         "count": 0
      } ]
   } ]
}

Measure Report Reporter

  • Measure Report reporter field indicates who created the report. When using Standard mode it will not be populated, but in Distributed mode it can be populated from passed in operation resources or by defaulting it on module configuration.

The following diagram illustrates the different ways via $evaluate-measures you can set the MeasureReport.reporter field using module configuration settings or operation parameter.

diagram

Defaulting Measure Report Reporter

Measure Report default reporter configuration in the DQM Module is meant to be set as a reference to an Organization Resource that you want to show on your Measure Reports.

To populate this, just add the Organization.id value to the configuration field in the Web Admin Console while in the DQM Module.

See here for Dqm Module Configuration Categories

Use Group Managing Entity for Measure Report Reporter

If configuration is set to true, MeasureReport.reporter field will attempt to source MeasureReport.reporter from evaluate-measure subject or practitioner parameter.

If a Group resource is used for the parameter and has a populated managingEntity reference to an Organization. It will use this value instead of the value saved to Measure Report default reporter configuration.

See here for Dqm Module Configuration Categories

Include Patients from Related Managing Organizations

If set to true, evaluate-measure subject parameter will collect Patient resources with a matching Patient.managingOrganization reference in addition to Organizations related by the Organization.partOf field.

see Set of Patients by Organization Reference for diagram and how Organization reference is used.

See here for Dqm Module Configuration Categories

Batch Job Analytics Operation

 

The $job-analytics operation is a tool to be used with distributed $evaluate-measure operation, and allows for real time progress of batch2 jobs per 'instanceId' created when distributed $evaluate-measure operation is called. The operation will provide an aggregated JSON response body containing a breakdown of batch job 'steps' and the distribution of work-chunks 'status' and durations within each step. This allows end-users to get deeper level of insight into where in the process jobs are progressing.

When a batch job completes processing and reaches status COMPLETED, it will purge all work chunk data associated with it. This means that the $job-analytics operation will not show results for "steps" when complete.

Batch Job Analytics Parameters

ParameterDescription
instanceIdThe Batch Job instanceId generated when creating a job

Batch Job Analytics Distributed Evaluate Measure Steps

Distributed $evaluate-measure operation has a specific sequence of steps it processes as a part of the job. The table below outlines the steps of the job for purposes of understanding progress.

StepsDescription
create-subject-chunksFirst step of the job to distribute chunks of patients to be processed
evaluate-subject-chunkSecond step of the job to evaluate each patient chunk for selected Measure resource
aggregate-subject-chunksFinal step of the job to reduce or aggregate all produced Measure Reports created from step 2.

Batch Job Analytics Work Chunk Status

Work Chunk StatusDescription
QUEUEDThe work is waiting to be picked up for processing
GATE_WAITINGThe work is staged for reduction step and won't process until all other prior steps are completed
IN_PROGRESSThe work chunk is currently being processed
COMPLETEDThe work chunk finished processing
FAILEDThe work chunk encountered an error and stopped processing

Batch Job Analytics Job Status

Batch Job StatusDescription
QUEUEDThe Job is waiting to begin
IN_PROGRESSThe job has begun
COMPLETEDThe job has finished
CANCELLEDthe job was manually stopped by a user
FAILEDthe job encountered an error and stopped processing

Batch Job Analytics Measure Report Response with InstanceId

When triggering a distributed $evaluate-measure request a user gets response back in the form of a Parameters resource containing one or more Bundles of Measure Reports or a single Measure Report that contains the instanceId value needed for $job-analytics request. This can be found in an extension of the Measure Report. Example shown below.

  • Example extension with instanceId
{
  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.jobId",
  "valueString": "2ecb9110-2c31-4f3e-aed6-67b5fe84f0a3"
}
  • Example full response body with instanceId
{
   "resourceType": "MeasureReport",
   "id": "8cfbc000-8330-4d11-b554-12c6e8dc246a",
   "meta": {
      "versionId": "1",
      "lastUpdated": "2024-08-02T10:38:27.935-06:00",
      "source": "#eysjHPPneaNBExsX"
   },
   "extension": [
      {
         "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
         "valueString": "CMS130"
      },
      {
         "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
         "valueString": "Measure - CMS-130: Colorectal Cancer Screening"
      },
      {
         "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.jobId",
         "valueString": "2ecb9110-2c31-4f3e-aed6-67b5fe84f0a3"
      }
   ],
   "status": "pending",
   "measure": "http://ecqi.healthit.gov/ecqms/Measure/CMS130|0.0.003",
   "date": "2024-08-02T10:38:27-06:00"
}

Batch Job Analytics Example Request

GET {fhir-endpoint}/$job-analytics?instanceId={Batch2Job InstanceId}

Batch Job Analytics Example Response

{
   "jobId": "{Batch2Job InstanceId}",
   "jobStatus": "IN_PROGRESS",
   "jobStart": "2024-08-02 10:38:30.893",
   "requestTimeStamp": "2024-08-02T16:40:53.799954Z",
   "steps": [
      {
         "stepChunkCount": 165,
         "stepStart": "2024-08-02 10:40:01.436",
         "stepId": "aggregate-subject-chunks",
         "stepStatus": "GATE_WAITING"
      },
      {
         "stepChunkCount": 1,
         "stepStart": "2024-08-02 10:38:30.893",
         "stepAvg": "INTERVAL '0 00:00:01.317' DAY TO SECOND",
         "stepId": "create-subject-chunks",
         "stepStatus": "COMPLETED",
         "stepEnd": "2024-08-02 10:38:32.21"
      },
      {
         "stepChunkCount": 165,
         "stepStart": "2024-08-02 10:39:30.957",
         "stepAvg": "INTERVAL '0 00:00:03.704533333' DAY TO SECOND",
         "stepId": "evaluate-subject-chunk",
         "stepStatus": "COMPLETED",
         "stepEnd": "2024-08-02 10:40:53.286"
      },
      {
         "stepChunkCount": 8,
         "stepStart": "2024-08-02 10:40:47.01",
         "stepId": "evaluate-subject-chunk",
         "stepStatus": "IN_PROGRESS"
      },
      {
         "stepChunkCount": 37,
         "stepStart": "2024-08-02 10:38:31.849",
         "stepId": "evaluate-subject-chunk",
         "stepStatus": "QUEUED"
      }
   ]
}

Spark Evaluate-Measure

 

Spark Evaluate-Measure provides a premium Big Data processing solution for Quality Measure Evaluations that relies on scaled out parallel processing using a spark compute cluster. This solution is meant to compliment Distributed Measure-Evaluation by giving a larger ceiling of processing power for population scale Quality Measure queries with 100's of Millions to Billions of resources.

It also provides the ability to perform multi-measure processing where users have the option to process as many measures as needed with the same patient population.

CMS Quality Reporting

 

In order to bridge capability between generating Quality Measure Reporting in FHIR and being able to submit Measure Reports to CMS, we have a built in operation that transforms FHIR into Quality Payment Program (QPP) JSON format.

For more information see QPP Reporting.

Measure Data Requirements Operation

 

The data-requirements operation aggregates and returns the parameters and data requirements for the measure and all its dependencies as a single module definition.

Data Requirements Parameters

ParameterSupportedDescription
Id✓the id of the Measure to evaluate
PeriodStart✓The start of the reporting period
PeriodEnd✓The end of the reporting period

Example Data Requirements Request

GET {fhir-endpoint}/Measure/<measureID>/$data-requirements?periodStart=2014&periodEnd=2014

Example Data Requirements Response

{
   "resourceType": "Library",
   "type": {
      "coding": [
         {
            "code": "module-definition"
         }
      ]
   },
   "relatedArtifact": [
      {
         "type": "depends-on",
         "resource": "Library/library-quick-model-definition"
      }
   ],
   "dataRequirement": [
      {
         "type": "Condition",
         "codeFilter": [
            {
               "path": "code",
               "valueSetString": "Other Female Reproductive Conditions"
            }
         ]
      },
      {
         "type": "Patient"
      },
      {
         "type": "Condition",
         "codeFilter": [
            {
               "path": "category",
               "valueCode": [
                  "diagnosis"
               ]
            },
            {
               "path": "clinicalStatus",
               "valueCode": [
                  "confirmed"
               ]
            },
            {
               "path": "code",
               "valueSetString": "2.16.840.1.113883.3.464.1003.102.12.1011"
            }
         ]
      },
      {
         "type": "Condition",
         "codeFilter": [
            {
               "path": "category",
               "valueCode": [
                  "diagnosis"
               ]
            },
            {
               "path": "clinicalStatus",
               "valueCode": [
                  "confirmed"
               ]
            },
            {
               "path": "code",
               "valueSetString": "2.16.840.1.113883.3.464.1003.102.12.1012"
            }
         ]
      },
      {
         "type": "Encounter",
         "codeFilter": [
            {
               "path": "status",
               "valueCode": [
                  "finished"
               ]
            },
            {
               "path": "class",
               "valueCode": [
                  "ambulatory"
               ]
            },
            {
               "path": "type",
               "valueSetString": "2.16.840.1.113883.3.464.1003.101.12.1061"
            }
         ]
      },
      {
         "type": "DiagnosticReport",
         "codeFilter": [
            {
               "path": "diagnosis",
               "valueSetString": "2.16.840.1.113883.3.464.1003.198.12.1012"
            }
         ]
      },
      {
         "type": "Medication",
         "codeFilter": [
            {
               "path": "code",
               "valueSetString": "2.16.840.1.113883.3.464.1003.196.12.1001"
            }
         ]
      },
      {
         "type": "MedicationRequest",
         "codeFilter": [
            {
               "path": "status",
               "valueCode": [
                  "active"
               ]
            },
            {
               "path": "medication.code",
               "valueSetString": "2.16.840.1.113883.3.464.1003.196.12.1001"
            }
         ]
      },
      {
         "type": "MedicationStatement",
         "codeFilter": [
            {
               "path": "status",
               "valueCode": [
                  "completed"
               ]
            },
            {
               "path": "medication.code",
               "valueSetString": "2.16.840.1.113883.3.464.1003.196.12.1001"
            }
         ]
      }
   ]
}

Measure Collect Data Operation

 

The collect-data operation is used to collect the data-of-interest for the given measure.

Collect Data Parameters

ParameterSupportedDescription
Id✓the id of the Measure
PeriodStart✓The start of the reporting period
PeriodEnd✓The end of the reporting period
Subject✓Subject for which the measure will be calculated. If not specified, the measure will be calculated for all subjects that meet the requirements of the measure. If specified, the measure will only be calculated for the referenced subject(s)
Practitioner✓Practitioner for which the measure will be calculated. If specified, the measure will be calculated only for subjects that have a primary relationship to the identified practitioner
LastReceivedOn The date the results of this measure were last received. This parameter is only valid for patient-level reports and is used to indicate when the last time a result for this patient was received. This information can be used to limit the set of resources returned for a patient-level report

Example Collect Data Request

  • Practitioner example
GET{fhir-endpoint}/Measure/<MeasureId>/$collect-data?practitioner=Practitioner/XYZ&periodStart=2023-01-01&periodEnd=2023-12-31
  • Patient Subject example
GET{fhir-endpoint}/Measure/<MeasureId>/$collect-data?subject=Patient/XYZ&periodStart=2023-01-01&periodEnd=2023-12-31
  • Group Subject example
GET{fhir-endpoint}/Measure/<MeasureId>/$collect-data?subject=Group/XYZ&periodStart=2023-01-01&periodEnd=2023-12-31

Example Collect Data Response

  • example response
{
   "resourceType": "Parameters",
   "parameter": [
      {
         "name": "measureReport",
         "resource": {
            "resourceType": "MeasureReport",
            "status": "complete",
            "type": "data-collection",
            "measure": "Measure/measure-mrp",
            "patient": {
               "reference": "Patient/Practitioner-2178"
            },
            "period": {
               "start": "2017-01-01T00:00:00+00:00",
               "end": "2017-12-31T00:00:00+00:00"
            },
            "evaluatedResource": [
               {
                  "reference": "Patient/Practitioner-2178"
               },
               {
                  "reference": "Encounter/Encounter-62912"
               },
               {
                  "reference": "Procedure/Procedure-89972"
               }
            ]
         }
      },
      {
         "name": "resource",
         "resource": {
            "resourceType": "Patient",
            "id": "Practitioner-2178",
            "meta": {
               "versionId": "8",
               "lastUpdated": "2018-05-31T07:15:08.000+00:00",
               "profile": [
                  "http://hl7.org/fhir/us/hedis/StructureDefinition/hedis-patient"
               ]
            },
            "text": {
               "status": "generated",
               "div": "<div xmlns=\"http://www.w3.org/1999/xhtml\"><div class=\"hapiHeaderText\">Natalie Huff <b>WALTER </b></div><table class=\"hapiPropertyTable\"><tbody><tr><td>Identifier</td><td>000002418</td></tr><tr><td>Address</td><td><span>623 SW French Place </span><br/><span>Westland </span><span>MI </span></td></tr><tr><td>Date of birth</td><td><span>26 July 1974</span></td></tr></tbody></table></div>"
            },
            "extension": [
               {
                  "url": "http://mihin.org/extension/copyright",
                  "valueString": "Copyright 2014-2017 Michigan Health Information Network Shared Services. Licensed under the Apache License, Version 2.0 (the 'License'); you can not use this file except in compliance with the License. You can obtain a copy of the License at  http://www.apache.org/licenses/LICENSE-2.0.  Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an 'AS IS' BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
               },
               {
                  "url": "http://hl7.org/fhir/us/core/StructureDefinition/us-core-race",
                  "valueCodeableConcept": {
                     "coding": [
                        {
                           "system": "http://hl7.org/fhir/v3/Race",
                           "code": "2106-3",
                           "display": "White"
                        }
                     ]
                  }
               },
               {
                  "url": "http://hl7.org/fhir/us/core/StructureDefinition/us-core-ethnicity",
                  "valueCodeableConcept": {
                     "coding": [
                        {
                           "system": "http://hl7.org/fhir/v3/Ethnicity",
                           "code": "2186-5",
                           "display": "Not Hispanic or Latino"
                        }
                     ]
                  }
               },
               {
                  "url": "http://hl7.org/fhir/us/core/StructureDefinition/us-core-religion",
                  "valueCodeableConcept": {
                     "coding": [
                        {
                           "system": "http://hl7.org/fhir/v3/ReligiousAffiliation",
                           "code": "1020",
                           "display": "Hinduism"
                        }
                     ]
                  }
               }
            ],
            "identifier": [
               {
                  "use": "official",
                  "type": {
                     "coding": [
                        {
                           "system": "http://terminology.hl7.org/CodeSystem/v2-0203",
                           "code": "SB",
                           "display": "Social Beneficiary Identifier"
                        }
                     ],
                     "text": "US Social Security Number"
                  },
                  "system": "http://hl7.org/fhir/sid/us-ssn",
                  "value": "000002418"
               }
            ],
            "active": true,
            "name": [
               {
                  "family": "Walter",
                  "given": [
                     "Natalie",
                     "Huff"
                  ]
               }
            ],
            "telecom": [
               {
                  "system": "phone",
                  "value": "248-555-3608",
                  "use": "home"
               },
               {
                  "system": "phone",
                  "value": "248-555-4649",
                  "use": "mobile"
               }
            ],
            "gender": "female",
            "birthDate": "1974-07-26",
            "address": [
               {
                  "use": "home",
                  "type": "postal",
                  "line": [
                     "623 SW French Place"
                  ],
                  "city": "Westland",
                  "district": "Wayne County",
                  "state": "MI",
                  "postalCode": "48185"
               }
            ]
         }
      },
      {
         "name": "resource",
         "resource": {
            "resourceType": "Encounter",
            "id": "Encounter-62912",
            "meta": {
               "versionId": "1",
               "lastUpdated": "2018-05-31T08:21:50.000+00:00",
               "profile": [
                  "http://hl7.org/fhir/us/hedis/StructureDefinition/hedis-encounter"
               ]
            },
            "extension": [
               {
                  "url": "http://mihin.org/extension/copyright",
                  "valueString": "Copyright 2014-2017 Michigan Health Information Network Shared Services. Licensed under the Apache License, Version 2.0 (the 'License'); you can not use this file except in compliance with the License. You can obtain a copy of the License at  http://www.apache.org/licenses/LICENSE-2.0.  Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an 'AS IS' BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
               }
            ],
            "status": "finished",
            "class": {
               "system": "http://hl7.org/fhir/encounter-class",
               "code": "inpatient",
               "display": "Inpatient"
            },
            "type": [
               {
                  "coding": [
                     {
                        "system": "http://snomed.info/sct",
                        "code": "56876005",
                        "display": "Drug rehabilitation and detoxification"
                     }
                  ]
               }
            ],
            "subject": {
               "reference": "Patient/Practitioner-2178",
               "display": "Natalie Huff Walter"
            },
            "period": {
               "start": "2018-05-29T11:00:00-04:00",
               "end": "2018-05-29T11:00:00-04:00"
            },
            "hospitalization": {
               "dischargeDisposition": {
                  "coding": [
                     {
                        "system": "http://hl7.org/fhir/discharge-disposition",
                        "code": "home",
                        "display": "Home"
                     }
                  ]
               }
            }
         }
      },
      {
         "name": "resource",
         "resource": {
            "resourceType": "Procedure",
            "id": "Procedure-89972",
            "meta": {
               "versionId": "1",
               "lastUpdated": "2018-05-30T23:56:19.000+00:00",
               "profile": [
                  "http://hl7.org/fhir/us/hedis/StructureDefinition/hedis-procedure"
               ]
            },
            "extension": [
               {
                  "url": "http://mihin.org/extension/copyright",
                  "valueString": "Copyright 2014-2017 Michigan Health Information Network Shared Services. Licensed under the Apache License, Version 2.0 (the 'License'); you can not use this file except in compliance with the License. You can obtain a copy of the License at  http://www.apache.org/licenses/LICENSE-2.0.  Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an 'AS IS' BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
               }
            ],
            "identifier": [
               {
                  "system": "mihin.org/fhir/sid/elementId",
                  "value": "ca097928-b18c-406c-a01b-6ad4a321fc28"
               }
            ],
            "status": "completed",
            "code": {
               "coding": [
                  {
                     "system": "http://www.ama-assn.org/go/cpt",
                     "code": "1111F",
                     "display": "Medication reconciliation postdischarge"
                  }
               ]
            },
            "subject": {
               "reference": "Patient/Practitioner-2178",
               "display": "Natalie Huff Walter"
            },
            "performedPeriod": {
               "start": "2018-06-02T14:00:00-05:00",
               "end": "2018-06-02T14:00:00-05:00"
            }
         }
      }
   ]
}

Partitioning

 

Smile CDR broadly (not dQM-specific) supports a number of partitioning schemes. The state of each of these partition schemes in the context of dQM measure evaluation is outlined below:

Tenant-Based

  • Tenant-based partitioning support is not explicitly supported.

Patient ID Based

  • Patient ID partitioning is NOT supported.

Other

  • Bucketed Patient, Request Header and Manual partitioning modes are NOT supported.

DQM Module

 

To enable both 'Standard' and 'Distributed' evaluate-measure, you will need to turn on the DQM Module on CDR.

See DQM for more details.

Performance

 

Retrieve Providers

The RetrieveProvider is the CQL engine's interface for fetching clinical data during expression evaluation. When CQL contains a retrieve expression like [Observation: "LDL Cholesterol"], the engine delegates to a Retrieve Provider to find matching FHIR resources. The configuration of the retrieve provider — particularly its terminology filtering strategy — has a major impact on evaluation performance.

Terminology Filter Modes

The retrieve_settings_terminology_filter_mode setting controls how ValueSet/code-based filtering is applied during CQL retrieves. There are four modes:

ModeBehaviorQuery Pattern
AUTODetects server capabilities and falls back appropriatelyDepends on detection
USE_VALUE_SET_URLUses the FHIR :in search modifier with the ValueSet URLObservation?code:in=<valueSetUrl>&subject=Patient/123
USE_INLINE_CODESExpands the ValueSet and adds every code as inline token search parametersObservation?code=system\|code1,system\|code2,...&subject=Patient/123
FILTER_IN_MEMORYSends no terminology search parameters; filters in Java after retrievalObservation?subject=Patient/123 (no code parameter)

The CDR property for this setting is: module.cql.config.retrieve.retrieve_settings_terminology_filter_mode

In the Web Admin Console, this is the Terminology Filter Mode dropdown in the CQL module configuration.

USE_INLINE_CODES

The USE_INLINE_CODES mode expands the ValueSet via the terminology provider, then adds every code from the expansion as inline token search parameters. The FHIR server then returns only resources matching one of the specified codes, enabling server-side filtering without requiring :in modifier support.

When to use USE_INLINE_CODES

This mode is recommended when CQL evaluations are slow (e.g., more than a full second per measure) and the FHIR server does not support the :in search modifier. The performance advantage over FILTER_IN_MEMORY scales with selectivity x volume x retrieve count x population size:

  • Resource volume per patient: A CQL retrieve like [Observation: "HbA1c"] with a ValueSet containing ~5 codes will cause USE_INLINE_CODES to return only the 2-3 matching Observations via indexed search, while FILTER_IN_MEMORY returns all Observations for the patient (potentially 10,000+), each of which must be deserialized, FHIRPath-evaluated, and checked against the terminology provider.
  • CMS measure structure: CMS 2024 measures typically have 5-10 retrieve expressions per measure, each with a selective ValueSet. Multiple retrieves often target the same resource type. Each retrieve independently suffers the FILTER_IN_MEMORY penalty.
  • Population size: Every per-patient inefficiency multiplies across all subjects in a population measure.

When FILTER_IN_MEMORY may be preferable

  • Huge ValueSet expansions with low selectivity: If a ValueSet expands to thousands of codes and most resources match anyway, the expansion cost is wasted and the query URL becomes very large.
  • Tiny datasets: If a patient has only a handful of resources of a given type, the fixed cost of ValueSet expansion exceeds the savings from filtering.
  • FHIR server poorly optimized for multi-token OR: Some servers handle long code=a,b,c,... queries poorly compared to a simple compartment scan.

Configuration

To set USE_INLINE_CODES via CDR properties:

module.cql.config.retrieve.retrieve_settings_terminology_filter_mode=USE_INLINE_CODES

If using the Web Admin Console, set Terminology Filter Mode to USE_INLINE_CODES in the dropdown.

Troubleshooting

 

Q: Measure evaluation is not showing enough logging, and I need to troubleshoot a performance issue which I suspect is related to CQL evaluation.

A: Enable troubleshooting logs for CQL evaluation via the admin JSON endpoint (port 9000). The CQIS shadow JAR relocates the clinical-reasoning packages, so the relocated package name must be used. Debug logs will appear in smile-troubleshooting.log.

PUT {admin-json-endpoint}/troubleshooting-log/configuration/ca.jd.cdr.cr.vendor.org.opencds.cqf.fhir.cr.measure?loggerLevel=DEBUG

Q: I have a performance issue with CQL evaluation in a measure evaluation. Debug logs show me CQL evaluations taking more than a full second per measure.

A: See the Performance section below for guidance on tuning CQL evaluation performance, particularly the Retrieve Providers and USE_INLINE_CODES sections.

FAQs

 

Q: I get an error saying Smile CDR can't locate my library, and I've verified it's on the server.

A: Smile CDR follows the Library conformance requirements defined by the CQF Measures IG, meaning the Library must have a logic-library type, the name and versions of the FHIR Library and CQL Library must match, and the url of the Library must end in the name of the Library.

FHIR Libraries generated from CQL via the IG Publisher follow these requirements automatically.

Q: Does Smile CDR with dQM support partitions for evaluation?

A: See the [Partitioning](Partition Section) above