23.3.1Table of Contents

• Measure Resource

• Measure Evaluating Options

  • Population Basis
  • Scoring Methods
  • Composite Scoring
  • Criteria Expression Type
  • Supplemental Data Elements
  • Stratifiers
    • Stratifier Expression Support
    • Stratifier Component Support
  • Evaluated Resources
  • Extensions

• Operation

• Operation Parameters

  • Reporting Period
  • Report Types
  • Subject Types
    • Selecting a set of Patients
  • ReportType, Subject, Practitioner Matrix
  • Last Received On
  • Parameters

• Measure Report Resource

  • Example Measure Report Individual
    • Example Individual Request
    • Example Individual Response
  • Example Measure Report Summary
    • Example Summary Request
    • Example Summary Response
  • Example Measure Report Subject-List
    • Example Subject-List Request
    • Example Subject-List Response

• Measure Report Populations And Scores

• Multiple Measure Evaluation

• Standard Evaluate Measure

  • Standard Evaluate Measure Example Request

• Distributed Evaluate Measure

  • Overview
  • Requirements
  • Distributed Mode Configurations
  • Distributed Workflow
    • Subject Chunk Size
    • Multi-Threading Options
      • Enable Threading
      • Number of Threads
      • Threaded Subject Chunk Size
  • User Interface for Requested Jobs
  • Example Distributed Request
  • Example In-Progress Response
  • Example Retrieve Completed Report
  • Measure Report Reporter
    • Defaulting Measure Report Reporter
    • Use Group managingEntity for Reporter

• Batch Job Analytics Operation

• CMS Quality Reporting

• Spark Evaluate-Measure

• Data Requirements Operation

• Collect Data Operation

• DQM Module

• FAQs

23.3.2Measure Resource

The FHIR Clinical Reasoning Module defines the Measure resource and several associated operations. The Measure Resource represents a structured, computable definition of a health-related measure such as a clinical quality measure, public health indicator, or population analytics measure. These Measures can then be used for reporting, analytics, and data-exchange purposes.

Electronic Clinical Quality Measures (eCQMs) in FHIR are represented as a FHIR Measure resource containing metadata and terminology, a population criteria section, and at least one FHIR Library resource containing a data criteria section as well as the logic used to define the population criteria. The population criteria section typically contains initial population criteria, denominator criteria, and numerator criteria subcomponents, among others. This is elaborated upon in greater detail in the CQF Measures IG. An example of an eCQM as defined in FHIR looks like:

{
   "resourceType" : "Measure",
   "library" : [
      "http://hl7.org/fhir/us/cqfmeasures/Library/EXMLogic"
   ],
   "group" : [
      {
         "population" : [
            {
               "code" : {
                  "coding" : [
                     {
                        "code" : "initial-population"
                     }
                  ]
               },
               "criteria" : {
                  "language" : "text/cql-identifier",
                  "expression" : "Initial Population"
               }
            },
            {
               "code" : {
                  "coding" : [
                     {
                        "code" : "numerator"
                     }
                  ]
               },
               "criteria" : {
                  "language" : "text/cql-identifier",
                  "expression" : "Numerator"
               }
            },
            {
               "code" : {
                  "coding" : [
                     {
                        "code" : "denominator"
                     }
                  ]
               },
               "criteria" : {
                  "language" : "text/cql-identifier",
                  "expression" : "Denominator"
               }
            }
         ]
      }
   ]
}

Measures are then scored according to whether a subjects (or subjects) are members of the various populations.

For example, a Measure for Breast Cancer screening might define an Initial Population (via CQL expressions) of "all women", a Denominator of "women over 35", and a Numerator of "women over 35 who have had breast cancer screenings in the past year". If the Measure is evaluated against a population of 100 women, 50 are over 35, and of those 25 have had breast cancer screenings in the past year, the final score would be 50%¹ (total number in numerator / total number in the denominator).

1. There are several methods for scoring Measures, this is meant only as an example.

23.3.3Measure Evaluating Options

The FHIR Measure specification defines several types of Measures and various parameters for controlling the Measure evaluation. This section describes the features supported by Smile CDR.

23.3.3.1Populations

Smile CDR uses the populations defined by the CQF Measures IG for each scoring type. A matrix of the supported populations is shown in the Criteria Names section of the CQF Measures IG.

23.3.3.2Population Criteria

The logical criteria used for determining each Measure population is defined by the Measure.group.population.criteria element. The Measure specification allows population criteria to be defined using FHIR Path, CQL, or other languages as appropriate. Smile CDR currently only supports using CQL. The relationship between a Measure Population and CQL is illustrated in the Population Criteria section of the CQF Measures IG.

An example Measure resource with a population criteria referencing a CQL identifier looks like:

{
   "resourceType": "Measure",
   "group": [ {
      "population": [ {
         "code": {
            "coding": [ {
               "system": "http://terminology.hl7.org/CodeSystem/measure-population",
               "code": "initial-population",
               "display": "Initial Population"
            } ]
         },
         "criteria": {
            "language": "text/cql-identifier",
            "expression": "Initial Population"
         }
      }]
   }]
}

23.3.3.3Population Basis

The population basis specifies the type of elements represented in the MeasureReport populations (example: "Numerator").

For measures that have a population basis that is different from a subject, this element specifies the type of the population basis.

This done with an extension element that can be added to a measure, see examples below.

Example Population Basis of Encounter

{
	"extension": [ {
"url": "http://hl7.org/fhir/us/cqfmeasures/StructureDefinition/cqfm-populationBasis",
"valueCode": "Encounter"
} ]
}

Example Population Basis of Boolean

{
  "extension": [
    {
      "url": "http://hl7.org/fhir/us/cqfmeasures/StructureDefinition/cqfm-populationBasis",
      "valueCode": "boolean"
    }
  ]
}

23.3.3.4Scoring Methods

The Measure scoring method determines how a Measure score is calculated. It is set with the scoring element on the Measure resource.

The Smile CDR implementation conforms to the requirements defined by the CQF Measures IG. A more detailed description of each scoring method is linked in the table below.

• The cohort Measure scoring support is partial. The Smile CDR implementation does not yet return the required Measure observations

An example Measure resource with scoring defined looks like:

{
  "resourceType": "Measure",
  "scoring": {
    "coding": [ {
      "system": "http://terminology.hl7.org/CodeSystem/measure-scoring",
      "code": "proportion",
      "display": "Proportion"
    } ]
  }
}

CDR also supports setting measure-scoring` on a Group level instead of just on the Measure level.

{
   "group": [
      {
         "id": "64f0daaf56d636294b157ea1",
         "extension": [
            {
               "url": "http://hl7.org/fhir/us/cqfmeasures/StructureDefinition/cqfm-scoring",
               "valueCodeableConcept": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-scoring",
                        "code": "proportion",
                        "display": "Proportion"
                     }
                  ]
               }
            },
            {
               "url": "http://hl7.org/fhir/us/cqfmeasures/StructureDefinition/cqfm-populationBasis",
               "valueCode": "Encounter"
            },
            {
               "url": "http://hl7.org/fhir/us/cqfmeasures/StructureDefinition/cqfm-type",
               "valueCodeableConcept": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-type",
                        "code": "process",
                        "display": "Process"
                     }
                  ]
               }
            }
         ]
      }
   ]
}

A composite Measure is scored by combining and/or aggregating the results of other Measures. The compositeScoring element is used to control how composite Measures are scored. Smile CDR does not currently support any composite scoring method.

23.3.3.5Criteria Expression Type

23.3.3.6Supplemental Data Elements

Supplemental Data Elements are used to report additional information about the subjects that may not be included in the Population criteria definitions. For example, it may be of interest to report the gender of all subjects for informational purposes. Supplemental data elements are defined by the Measure.supplementalData element, and are reported as Observations in the evaluatedResources of the MeasureReport.

Supplemental Data Elements can be specified as either CQL definitions or FHIR Path expressions.

An example Measure resource with some supplemental data elements set looks like:

{
"resourceType": "Measure",
   "supplementalData": [ {
      "code": {
         "text": "sde-ethnicity"
      },
      "criteria": {
         "language": "text/cql-identifier",
         "expression": "SDE Ethnicity"
      }
   }]
}

23.3.3.7Stratifiers

• Stratifiers are used divide Measure populations into segments of interest. For example, it may be of interest to compare the Measure score between different age groups or genders. Each stratum within a stratification is scored the same way as the overall population. Stratifiers are defined using the Measure.group.stratifier element.

• An example Measure resource with a stratifier set looks like:

{
  "resourceType": "Measure",
  "group": [ {
      "stratifier": [ {
         "code": {
            "text": "Stratum 1"
         },
         "criteria": {
            "language": "text/cql-identifier",
            "expression": "Stratification 1"
         }
      }]
   }]
}

23.3.3.8Evaluated Resources

A FHIR MeasureReport permits referencing the Resources used when evaluating in the MeasureReport.evaluatedResource element. Smile CDR includes these resources when generating subject reports for a single Patient. Evaluated resources for population or subject-list reports are not included. For large populations this could quickly become an extremely large number of resources.

The evaluated resources will not include every resource on the Smile CDR server for a given subject. Rather, it includes only the resources that were retrieved from the server by the CQL logic that was evaluated. This corresponds to the data-requirements for a given Measure. As an example, consider the following CQL:

valueset "Example Value Set" : 'http://fhir.org/example-value-set'

define "Example Observations":
   [Observation : "Example Value Set"]

That CQL will only select Observation Resources that have a code in the "Example Value Set". Those Observations will be reported in the Evaluated Resources while any others will not.

23.3.3.9Extensions

A number of extensions to Measure evaluation defined by various IGs are supported. They are described briefly in the table below.

There's not currently a way to configure which extensions are enabled. All supported extensions are always enabled.

23.3.4Operation

Smile CDR implements the $evaluate-measure operation into three different implementations and use cases.

23.3.5Operation Parameters

parameter "Measurement Period" Interval<DateTime>
  default Interval[@2019-01-01T00:00:00.0, @2020-01-01T00:00:00.0)

GET{fhir-endpoint}/Measure/<MeasureId>/$evaluate-measure?periodStart=2019-01-01&periodEnd=2019-12-31

23.3.6Headers with FHIR Requests

The behaviour of both periodStart and periodEnd depend on the value of the Timezone header. The measure report will be queried according to the period range, as denoted by that timezone, NOT, the server timezone.

Accepted values for this header are documented here: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones

ex: Timezone:American/Denver will set the timezone to Mountain Time.

If the client provides nothing in this header, the timezone will default to UTC.

Please consult the below table for examples of various combinations of start, end, and timezone, as well as the resulting queried periods:

• There's an open issue on the FHIR specification to align these names to the MeasureReportType value set.
• ReportType Subject, only when called from $evaluate-measures, where the subject type reference returns multiple patients, it will produce one 'Measure Report' per patient, per Measure requested in the returned Bundle results.
  • example: 5 patients were queried with $evaluate-measures?reportType=subject&measureId=Measure/123 would generate a bundle with 5 'individual' type 'Measure Report' resources for 'Measure/123'.

A request using reportType looks like:

GET{fhir-endpoint}/Measure/<MeasureId>/$evaluate-measure?reportType=subject-list

23.3.7Measure Subject Types

The subject of a measure evaluation is controlled with the subject (R4+) and patient (DSTU3) operation parameters. Currently, the only subject type supported by Smile CDR is Patient. This means that all Measure evaluation and reporting happens with respect to a Patient or set of Patient resources.

23.3.7.1Selecting Measure Subject for a set of Patients

The set of Patients for Measure evaluation is defined by either the subject or practitioner to act as a reference to a subset of Patients defined by various Patient attribution models or if left empty, the entire set of Patients. This section and it's sub-sections will detail how Patients are attributed to the accepted subject references provided to measure evaluation operations.

The set of Patients used for Measure evaluation is controlled with the subject (R4+) or patient (DSTU3), and practitioner parameters. The two parameters are mutually exclusive.

{
	"resourceType": "Parameters",
	"parameter": [
		{
			"name": "encounter",
			"valueString": "2d0ecfb4-9dec-4daa-a261-e37e426d0d7b"
		}
	]
}

23.3.8Measure Report Resource

The outcome of running $evaluate-measure returns a Measure Report resource with the results outputting in the structure defined in the Measure resource.

GET http://your-server-base/fhir/Measure/ColorectalCancerScreeningsFHIR/$evaluate-measure?periodStart=2023-01-01&periodEnd=2023-12-31&reportType=subject&subject=Patient/numer-EXM130-Patient-1

{
   "resourceType": "MeasureReport",
   "language": "en",
   "contained": [
      {
         "resourceType": "Observation",
         "id": "70e1db58-acc7-4d06-b9d1-1139164bfb0d",
         "extension": [
            {
               "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
               "extension": [
                  {
                     "url": "measure",
                     "valueCanonical": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR"
                  }
               ]
            }
         ],
         "status": "final",
         "code": {
            "coding": [
               {
                  "system": "http://terminology.hl7.org/CodeSystem/measure-data-usage",
                  "code": "supplemental-data"
               }
            ]
         },
         "valueCodeableConcept": {
            "coding": [
               {
                  "code": "M"
               }
            ]
         }
      },
      {
         "resourceType": "Observation",
         "id": "3f773d33-d40f-4b76-9de9-dd5af88f8924",
         "extension": [
            {
               "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
               "extension": [
                  {
                     "url": "measure",
                     "valueCanonical": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR"
                  }
               ]
            }
         ],
         "status": "final",
         "code": {
            "coding": [
               {
                  "system": "http://terminology.hl7.org/CodeSystem/measure-data-usage",
                  "code": "supplemental-data"
               }
            ]
         },
         "valueCodeableConcept": {
            "coding": [
               {
                  "code": "2135-2"
               }
            ]
         }
      },
      {
         "resourceType": "Observation",
         "id": "cd1924a6-15e5-448f-9b90-3de1bd1f4efe",
         "extension": [
            {
               "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
               "extension": [
                  {
                     "url": "measure",
                     "valueCanonical": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR"
                  }
               ]
            }
         ],
         "status": "final",
         "code": {
            "coding": [
               {
                  "system": "http://terminology.hl7.org/CodeSystem/measure-data-usage",
                  "code": "supplemental-data"
               }
            ]
         },
         "valueCodeableConcept": {
            "coding": [
               {
                  "code": "2028-9"
               }
            ]
         }
      }
   ],
   "extension": [
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.population.description",
         "valueString": "Percentage of adults 50-75 years of age who had appropriate screening for colorectal cancer"
      },
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.supplementalDataElement.reference",
         "valueReference": {
            "extension": [
               {
                  "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
                  "valueString": "5C3ACF2A-15D2-44A0-A83D-F4FCC3E2F27A"
               }
            ],
            "reference": "#cd1924a6-15e5-448f-9b90-3de1bd1f4efe"
         }
      },
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.supplementalDataElement.reference",
         "valueReference": {
            "extension": [
               {
                  "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
                  "valueString": "9CB0299E-74B0-4425-8758-52D6FC0F979C"
               }
            ],
            "reference": "#3f773d33-d40f-4b76-9de9-dd5af88f8924"
         }
      },
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.supplementalDataElement.reference",
         "valueReference": {
            "extension": [
               {
                  "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
                  "valueString": "8114BAD7-FAC8-4E22-91CB-33BC3DDB9986"
               }
            ],
            "reference": "#70e1db58-acc7-4d06-b9d1-1139164bfb0d"
         }
      }
   ],
   "status": "complete",
   "type": "individual",
   "measure": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR|0.0.003",
   "subject": {
      "reference": "Patient/numer-EXM130-Patient-1"
   },
   "date": "2024-03-05T19:30:24+00:00",
   "period": {
      "start": "2023-01-01T00:00:00+00:00",
      "end": "2023-12-31T23:59:59+00:00"
   },
   "improvementNotation": {
      "coding": [
         {
            "system": "http://terminology.hl7.org/CodeSystem/measure-improvement-notation",
            "code": "increase"
         }
      ]
   },
   "group": [
      {
         "extension": [
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-cqfm-denominator-membership",
               "valueString": "1"
            },
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-cqfm-numerator-membership",
               "valueString": "1"
            }
         ],
         "population": [
            {
               "id": "178DA8D8-0694-4B88-8FFE-42CE671EEE35",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "initial-population",
                        "display": "Initial Population"
                     }
                  ]
               },
               "count": 1
            },
            {
               "id": "0AC3911A-2ADC-4DA4-BEBF-545FF8D6D819",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "denominator",
                        "display": "Denominator"
                     }
                  ]
               },
               "count": 1
            },
            {
               "id": "67EABB9C-ADCF-4593-A8DA-35FF25DA594C",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "denominator-exclusion",
                        "display": "Denominator Exclusion"
                     }
                  ]
               },
               "count": 0
            },
            {
               "id": "14B66980-07F4-4872-83AF-C425C379B971",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "numerator",
                        "display": "Numerator"
                     }
                  ]
               },
               "count": 1
            }
         ],
         "measureScore": {
            "value": 1.0
         }
      }
   ],
   "evaluatedResource": [
      {
         "extension": [
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
               "valueString": "178DA8D8-0694-4B88-8FFE-42CE671EEE35"
            },
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
               "valueString": "0AC3911A-2ADC-4DA4-BEBF-545FF8D6D819"
            },
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
               "valueString": "67EABB9C-ADCF-4593-A8DA-35FF25DA594C"
            },
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
               "valueString": "9CB0299E-74B0-4425-8758-52D6FC0F979C"
            },
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
               "valueString": "5C3ACF2A-15D2-44A0-A83D-F4FCC3E2F27A"
            },
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
               "valueString": "8114BAD7-FAC8-4E22-91CB-33BC3DDB9986"
            }
         ],
         "reference": "Patient/numer-EXM130-Patient-1"
      },
      {
         "extension": [
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
               "valueString": "178DA8D8-0694-4B88-8FFE-42CE671EEE35"
            },
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
               "valueString": "0AC3911A-2ADC-4DA4-BEBF-545FF8D6D819"
            },
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
               "valueString": "67EABB9C-ADCF-4593-A8DA-35FF25DA594C"
            }
         ],
         "reference": "Encounter/numer-EXM130-Encounter-1"
      },
      {
         "extension": [
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
               "valueString": "14B66980-07F4-4872-83AF-C425C379B971"
            }
         ],
         "reference": "Procedure/numer-EXM130-1"
      }
   ]
}

GET http://your-server-base/fhir/Measure/BreastCancerScreeningFHIR/$evaluate-measure?periodStart=2019-01-01&periodEnd=2019-12-31&subject=numerator&reportType=population

{
   "resourceType": "MeasureReport",
   "language": "en",
   "contained": [
      {
         "resourceType": "Observation",
         "id": "5203d419-bb0b-4fe5-b5a7-83f761b0f13e",
         "extension": [
            {
               "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
               "extension": [
                  {
                     "url": "measure",
                     "valueCanonical": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR"
                  }
               ]
            }
         ],
         "status": "final",
         "code": {
            "coding": [
               {
                  "code": "F"
               }
            ]
         },
         "valueInteger": 2
      },
      {
         "resourceType": "Observation",
         "id": "ccf81b4d-4440-44ef-91ff-101b0758eded",
         "extension": [
            {
               "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
               "extension": [
                  {
                     "url": "measure",
                     "valueCanonical": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR"
                  }
               ]
            }
         ],
         "status": "final",
         "code": {
            "coding": [
               {
                  "code": "2135-2"
               }
            ]
         },
         "valueInteger": 6
      },
      {
         "resourceType": "Observation",
         "id": "258b869e-b4da-441c-93d4-fc64c71bb49a",
         "extension": [
            {
               "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
               "extension": [
                  {
                     "url": "measure",
                     "valueCanonical": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR"
                  }
               ]
            }
         ],
         "status": "final",
         "code": {
            "coding": [
               {
                  "code": "2028-9"
               }
            ]
         },
         "valueInteger": 6
      },
      {
         "resourceType": "Observation",
         "id": "23995bb2-bc5e-4ff5-8509-17dacfe6b375",
         "extension": [
            {
               "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
               "extension": [
                  {
                     "url": "measure",
                     "valueCanonical": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR"
                  }
               ]
            }
         ],
         "status": "final",
         "code": {
            "coding": [
               {
                  "code": "M"
               }
            ]
         },
         "valueInteger": 4
      }
   ],
   "extension": [
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.population.description",
         "valueString": "Percentage of adults 50-75 years of age who had appropriate screening for colorectal cancer"
      },
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.supplementalDataElement.reference",
         "valueReference": {
            "extension": [
               {
                  "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
                  "valueString": "5C3ACF2A-15D2-44A0-A83D-F4FCC3E2F27A"
               }
            ],
            "reference": "#258b869e-b4da-441c-93d4-fc64c71bb49a"
         }
      },
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.supplementalDataElement.reference",
         "valueReference": {
            "extension": [
               {
                  "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
                  "valueString": "8114BAD7-FAC8-4E22-91CB-33BC3DDB9986"
               }
            ],
            "reference": "#23995bb2-bc5e-4ff5-8509-17dacfe6b375"
         }
      },
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.supplementalDataElement.reference",
         "valueReference": {
            "extension": [
               {
                  "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
                  "valueString": "9CB0299E-74B0-4425-8758-52D6FC0F979C"
               }
            ],
            "reference": "#ccf81b4d-4440-44ef-91ff-101b0758eded"
         }
      },
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.supplementalDataElement.reference",
         "valueReference": {
            "extension": [
               {
                  "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
                  "valueString": "8114BAD7-FAC8-4E22-91CB-33BC3DDB9986"
               }
            ],
            "reference": "#5203d419-bb0b-4fe5-b5a7-83f761b0f13e"
         }
      }
   ],
   "status": "complete",
   "type": "summary",
   "measure": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR|0.0.003",
   "date": "2024-03-05T19:33:25+00:00",
   "period": {
      "start": "2023-01-01T00:00:00+00:00",
      "end": "2023-12-31T23:59:59+00:00"
   },
   "improvementNotation": {
      "coding": [
         {
            "system": "http://terminology.hl7.org/CodeSystem/measure-improvement-notation",
            "code": "increase"
         }
      ]
   },
   "group": [
      {
         "extension": [
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-cqfm-denominator-membership",
               "valueString": "2"
            },
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-cqfm-numerator-membership",
               "valueString": "1"
            }
         ],
         "population": [
            {
               "id": "178DA8D8-0694-4B88-8FFE-42CE671EEE35",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "initial-population",
                        "display": "Initial Population"
                     }
                  ]
               },
               "count": 2
            },
            {
               "id": "0AC3911A-2ADC-4DA4-BEBF-545FF8D6D819",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "denominator",
                        "display": "Denominator"
                     }
                  ]
               },
               "count": 2
            },
            {
               "id": "67EABB9C-ADCF-4593-A8DA-35FF25DA594C",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "denominator-exclusion",
                        "display": "Denominator Exclusion"
                     }
                  ]
               },
               "count": 0
            },
            {
               "id": "14B66980-07F4-4872-83AF-C425C379B971",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "numerator",
                        "display": "Numerator"
                     }
                  ]
               },
               "count": 1
            }
         ],
         "measureScore": {
            "value": 0.5
         }
      }
   ]
}

GET http://your-server-base/fhir/Measure/ColorectalCancerScreeningsFHIR/$evaluate-measure?periodStart=2023-01-01&periodEnd=2023-12-31&reportType=subject-list&subject=Patient/numer-EXM130-Patient-1

{
   "resourceType": "MeasureReport",
   "language": "en",
   "contained": [
      {
         "resourceType": "Observation",
         "id": "9ed35e34-9270-49e2-bf71-ca979fb2bcbf",
         "extension": [
            {
               "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
               "extension": [
                  {
                     "url": "measure",
                     "valueCanonical": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR"
                  }
               ]
            }
         ],
         "status": "final",
         "code": {
            "coding": [
               {
                  "code": "2135-2"
               }
            ]
         },
         "valueInteger": 1
      },
      {
         "resourceType": "List",
         "id": "d11fe780-66e2-4ab5-aff6-63028665114a",
         "entry": [
            {
               "item": {
                  "reference": "numer-EXM130-Patient-1"
               }
            }
         ]
      },
      {
         "resourceType": "Observation",
         "id": "428128d9-9e4f-4fe5-999f-b2748032c5dd",
         "extension": [
            {
               "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
               "extension": [
                  {
                     "url": "measure",
                     "valueCanonical": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR"
                  }
               ]
            }
         ],
         "status": "final",
         "code": {
            "coding": [
               {
                  "code": "2028-9"
               }
            ]
         },
         "valueInteger": 1
      },
      {
         "resourceType": "List",
         "id": "ca82c41b-e668-477d-80ae-b545986e013f",
         "entry": [
            {
               "item": {
                  "reference": "numer-EXM130-Patient-1"
               }
            }
         ]
      },
      {
         "resourceType": "List",
         "id": "80873504-6c9b-4d85-a0de-2b8c36962327",
         "entry": [
            {
               "item": {
                  "reference": "numer-EXM130-Patient-1"
               }
            }
         ]
      },
      {
         "resourceType": "Observation",
         "id": "56a28519-939e-41bd-99c9-1e6fddf600da",
         "extension": [
            {
               "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
               "extension": [
                  {
                     "url": "measure",
                     "valueCanonical": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR"
                  }
               ]
            }
         ],
         "status": "final",
         "code": {
            "coding": [
               {
                  "code": "M"
               }
            ]
         },
         "valueInteger": 1
      }
   ],
   "extension": [
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.population.description",
         "valueString": "Percentage of adults 50-75 years of age who had appropriate screening for colorectal cancer"
      },
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.supplementalDataElement.reference",
         "valueReference": {
            "extension": [
               {
                  "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
                  "valueString": "8114BAD7-FAC8-4E22-91CB-33BC3DDB9986"
               }
            ],
            "reference": "#56a28519-939e-41bd-99c9-1e6fddf600da"
         }
      },
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.supplementalDataElement.reference",
         "valueReference": {
            "extension": [
               {
                  "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
                  "valueString": "9CB0299E-74B0-4425-8758-52D6FC0F979C"
               }
            ],
            "reference": "#9ed35e34-9270-49e2-bf71-ca979fb2bcbf"
         }
      },
      {
         "url": "http://hl7.org/fhir/5.0/StructureDefinition/extension-MeasureReport.supplementalDataElement.reference",
         "valueReference": {
            "extension": [
               {
                  "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-criteriaReference",
                  "valueString": "5C3ACF2A-15D2-44A0-A83D-F4FCC3E2F27A"
               }
            ],
            "reference": "#428128d9-9e4f-4fe5-999f-b2748032c5dd"
         }
      }
   ],
   "status": "complete",
   "type": "subject-list",
   "measure": "http://ecqi.healthit.gov/ecqms/Measure/ColorectalCancerScreeningsFHIR|0.0.003",
   "date": "2024-03-05T19:31:47+00:00",
   "period": {
      "start": "2023-01-01T00:00:00+00:00",
      "end": "2023-12-31T23:59:59+00:00"
   },
   "improvementNotation": {
      "coding": [
         {
            "system": "http://terminology.hl7.org/CodeSystem/measure-improvement-notation",
            "code": "increase"
         }
      ]
   },
   "group": [
      {
         "extension": [
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-cqfm-denominator-membership",
               "valueString": "1"
            },
            {
               "url": "http://hl7.org/fhir/us/davinci-deqm/StructureDefinition/extension-cqfm-numerator-membership",
               "valueString": "1"
            }
         ],
         "population": [
            {
               "id": "178DA8D8-0694-4B88-8FFE-42CE671EEE35",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "initial-population",
                        "display": "Initial Population"
                     }
                  ]
               },
               "count": 1,
               "subjectResults": {
                  "reference": "#d11fe780-66e2-4ab5-aff6-63028665114a"
               }
            },
            {
               "id": "0AC3911A-2ADC-4DA4-BEBF-545FF8D6D819",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "denominator",
                        "display": "Denominator"
                     }
                  ]
               },
               "count": 1,
               "subjectResults": {
                  "reference": "#80873504-6c9b-4d85-a0de-2b8c36962327"
               }
            },
            {
               "id": "67EABB9C-ADCF-4593-A8DA-35FF25DA594C",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "denominator-exclusion",
                        "display": "Denominator Exclusion"
                     }
                  ]
               },
               "count": 0
            },
            {
               "id": "14B66980-07F4-4872-83AF-C425C379B971",
               "code": {
                  "coding": [
                     {
                        "system": "http://terminology.hl7.org/CodeSystem/measure-population",
                        "code": "numerator",
                        "display": "Numerator"
                     }
                  ]
               },
               "count": 1,
               "subjectResults": {
                  "reference": "#ca82c41b-e668-477d-80ae-b545986e013f"
               }
            }
         ],
         "measureScore": {
            "value": 1.0
         }
      }
   ]
}

23.3.9Measure Report Populations And Scores

When looking at single-rate or Multi-rate (more than one group) measures that have been evaluated it is important to understand what you are looking at in the counts for each population.

We learned in population-basis that this could represent a patient resource or represent resources of a subject like an encounter, but each population is not simply a raw expression output of CQL, each scoring-method defines the structure of each Measure Report as well as how population membership is evaluated and scored.

These scoring algorithms are defined in the following IG.

The intent of the MeasureReport population counts is to display how each population was evaluated to the scoring algorithm for the specified scoring type. Allowing end users the ability to compute the measure "score" using the values from population counts.

If a user ever needed to understand the raw output of a population expression, as a means to audit the algorithm, they could use the available Library $evaluate operation.

The tables below express in set logic how each scoring method would evaluate from raw CQL expression output to eventual "score".

• MeasureReport Populations - population names found in Measure Report
• Criteria Expression Result Set - Set representation of criteria expression output, raw results
• Evaluated Population Membership Set - Set representation of evaluator algorithm
• Criteria Expression Result Count - Count representation of criteria expression output, raw results
• Evaluated Population Membership Count representation of evaluator algorithm, the values shown on report

23.3.9.1Proportion & Ratio Measure

• Denominator Exception does not apply to Ratio measures
• TotalDenominator and TotalNumerator are also available extensions in CDR that show the final values used to calculate measureScore.

23.3.9.2Continuous Variable Measure

23.3.9.3Cohort Measure

23.3.10Multiple Measure Evaluation

The $evaluate-measures operation is used to execute more than one measure resource per selected subject population. This behaves similarly to the $evalaute-measure operation in that it has a "Distributed" and "Standard" mode for low latency and batch processing, but returns a Bundle resource instead of a MeasureReport resource.

This is designed to assist any users that will perform the measure evaluations for the same population against multiple measures.

• One use case could be evaluating a provider's population against a specialty measurement-set. Instead of multiple requests of $evaluate-measure, one for each measure, you can now have a single request to $evaluate-measures that contains all run results in a contained bundle resource.

23.3.11Multiple Measure Operation Parameters

23.3.11.1Bundle of Measure Reports

For each measureId parameter added to the operation request, a subsequent Measure Report resource will be added to the returned Bundle resource created by $evaluate-measures request.

For example, in order to evaluate six measure resources for a selected population (subject or practitioner parameters), then the operation will evaluate that same population against each of those measures and return a single Measure Report back for each passed in Measure.id.

23.3.11.2Measure Report Reporter Parameter

This feature also allows for an additional parameter to the standard $evaluate-measure, allowing callers to pass in a compliant resource for Measure Report reporter.

This can be used instead of dQM module configurations for default reporter and/or use Group Managing Entity.

• The reporter parameter will override these configurations (if set), and persist the parameter value to each Measure Report created.

23.3.11.3Multiple Measure Evaluation Example Request

• "Standard" Mode Request

GET{fhir-endpoint}/Measure/$evaluate-measures?subject=Practitioner/XYZ&reporter=Organization/123&measureId=<MeasureId-1>&measureId=<MeasureId-2>&measureId=<MeasureId-3>

• "Distributed" Mode Request

POST{fhir-endpoint}/Measure/$evaluate-measures?subject=Practitioner/XYZ&reporter=Organization/123&measureId=<MeasureId-1>&measureId=<MeasureId-2>&measureId=<MeasureId-3>
Prefer: respond-async

23.3.11.4Multiple Measure Evaluation Example Response

• "Distributed" Mode Response - pending

{
   "resourceType": "Bundle",
   "id": "bundle-id",
   "identifier": {
      "value": "measure-eval-job-id-<job-id>"
   },
   "type": "collection",
   "entry": [
      {
         "fullUrl": "http://localhost:8000/MeasureReport/<id>",
         "resource": {
            "resourceType": "MeasureReport",
            "id": "<id>",
            "extension": [
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                  "valueString": "<MeasureId-1>"
               },
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                  "valueString": "<MeasureId-1-Title>"
               }
            ],
            "status": "pending",
            "measure": "url/Measure/<MeasureId-1>"
         }
      },
      {
         "fullUrl": "http://localhost:8000/MeasureReport/<id>",
         "resource": {
            "resourceType": "MeasureReport",
            "id": "<id>",
            "extension": [
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                  "valueString": "<MeasureId-2>"
               },
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                  "valueString": "<MeasureId-2-Title>"
               }
            ],
            "status": "pending",
            "measure": "url/Measure/<MeasureId-2>"
         }
      },
      {
         "fullUrl": "http://localhost:8000/MeasureReport/<id>",
         "resource": {
            "resourceType": "MeasureReport",
            "id": "<id>",
            "extension": [
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                  "valueString": "<MeasureId-3>"
               },
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                  "valueString": "<MeasureId-3-Title>"
               }
            ],
            "status": "pending",
            "measure": "url/Measure/<MeasureId-3>"
         }
      }
   ]
}

• "Distributed" Mode Response - complete
  • bundle and report(s) do persist

{
   "resourceType": "Bundle",
   "id": "bundle-id",
   "identifier": {
      "value": "measure-eval-job-id-<job-id>"
   },
   "type": "collection",
   "entry": [
      {
         "fullUrl": "http://localhost:8000/MeasureReport/<id>",
         "resource": {
            "resourceType": "MeasureReport",
            "id": "<id>",
            "extension": [
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                  "valueString": "<MeasureId-1>"
               },
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                  "valueString": "<MeasureId-1-Title>"
               }
            ],
            "status": "complete",
            "measure": "url/Measure/<MeasureId-1>",
            "date": "2024-05-16T11:51:02-06:00",
            "reporter": {
               "reference": "Organization/123"
            },
            "subject": {
               "reference": "Practitioner/XYZ"
            },
            "period": {
               "start": "2023-01-01T00:00:00-07:00",
               "end": "2023-12-31T23:59:59-07:00"
            }
         }
      },
      {
         "fullUrl": "http://localhost:8000/MeasureReport/<id>",
         "resource": {
            "resourceType": "MeasureReport",
            "id": "<id>",
            "extension": [
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                  "valueString": "<MeasureId-2>"
               },
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                  "valueString": "<MeasureId-2-Title>"
               }
            ],
            "status": "complete",
            "measure": "url/Measure/<MeasureId-2>",
            "date": "2024-05-16T11:51:02-06:00",
            "reporter": {
               "reference": "Organization/123"
            },
            "subject": {
               "reference": "Practitioner/XYZ"
            },
            "period": {
               "start": "2023-01-01T00:00:00-07:00",
               "end": "2023-12-31T23:59:59-07:00"
            }
         }
      },
      {
         "fullUrl": "http://localhost:8000/MeasureReport/<id>",
         "resource": {
            "resourceType": "MeasureReport",
            "id": "<id>",
            "extension": [
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                  "valueString": "<MeasureId-3>"
               },
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                  "valueString": "<MeasureId-3-Title>"
               }
            ],

            "status": "complete",
            "measure": "url/Measure/<MeasureId-3>",
            "date": "2024-05-16T11:51:02-06:00",
            "reporter": {
               "reference": "Organization/123"
            },
            "subject": {
               "reference": "Practitioner/XYZ"
            },
            "period": {
               "start": "2023-01-01T00:00:00-07:00",
               "end": "2023-12-31T23:59:59-07:00"
            }
         }
      }
   ]
}

• "Standard" Mode Response - complete
  • bundle and report(s) do not persist, returns Bundle Type SearchSet instead of Collection

{
   "resourceType": "Bundle",
   "id": "bundle-id",
   "type": "searchset",
   "entry": [
      {
         "fullUrl": "http://localhost:8000/MeasureReport/<id>",
         "resource": {
            "resourceType": "MeasureReport",
            "id": "<id>",
            "extension": [
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                  "valueString": "<MeasureId-1>"
               },
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                  "valueString": "<MeasureId-1-Title>"
               }
            ],
            "status": "complete",
            "measure": "url/Measure/<MeasureId-1>",
            "date": "2024-05-16T11:51:02-06:00",
            "reporter": {
               "reference": "Organization/123"
            },
            "subject": {
               "reference": "Practitioner/XYZ"
            },
            "period": {
               "start": "2023-01-01T00:00:00-07:00",
               "end": "2023-12-31T23:59:59-07:00"
            }
         }
      },
      {
         "fullUrl": "http://localhost:8000/MeasureReport/<id>",
         "resource": {
            "resourceType": "MeasureReport",
            "id": "<id>",
            "extension": [
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                  "valueString": "<MeasureId-2>"
               },
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                  "valueString": "<MeasureId-2-Title>"
               }
            ],
            "status": "complete",
            "measure": "url/Measure/<MeasureId-2>",
            "date": "2024-05-16T11:51:02-06:00",
            "reporter": {
               "reference": "Organization/123"
            },
            "subject": {
               "reference": "Practitioner/XYZ"
            },
            "period": {
               "start": "2023-01-01T00:00:00-07:00",
               "end": "2023-12-31T23:59:59-07:00"
            }
         }
      },
      {
         "fullUrl": "http://localhost:8000/MeasureReport/<id>",
         "resource": {
            "resourceType": "MeasureReport",
            "id": "<id>",
            "extension": [
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
                  "valueString": "<MeasureId-3>"
               },
               {
                  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
                  "valueString": "<MeasureId-3-Title>"
               }
            ],

            "status": "complete",
            "measure": "url/Measure/<MeasureId-3>",
            "date": "2024-05-16T11:51:02-06:00",
            "reporter": {
               "reference": "Organization/123"
            },
            "subject": {
               "reference": "Practitioner/XYZ"
            },
            "period": {
               "start": "2023-01-01T00:00:00-07:00",
               "end": "2023-12-31T23:59:59-07:00"
            }
         }
      }
   ]
}

23.3.12Standard Evaluate Measure

The $evaluate-measure operation is used to execute a Measure as specified by the relevant FHIR Resources against a subject or set of subjects. This implementation currently focuses primarily on supporting the narrower evaluation requirements defined by the CQF Measures IG. Some support for extensions defined by other IGs is included as well, and the implementation aims to support a wider range of functionality in the future.

Several example Measures are available in the ecqm-content-r4 IG. Full Bundles with all the required supporting resources are available here. You can download a Bundle and load it on your server as a transaction:

POST http://your-server-base/fhir BreastCancerScreeningFHIR-bundle.json

These Bundles also include example Patient clinical data so once posted Measure evaluation can be invoked with:

GET http://your-server-base/fhir/Measure/BreastCancerScreeningFHIR/$evaluate-measure?periodStart=2019-01-01&periodEnd=2019-12-31&subject=numerator&reportType=subject

23.3.13Distributed Evaluate Measure

To take advantage this functionality you need to have a clustered Smile CDR deployment (see Clustering for more information) and a premium license for the DQM module (see Licensing for more information).

This capability extends to both $evaluate-measure and $evaluate-measures operations.

See here for Dqm Module Configuration Categories

See here for Dqm Module Configuration Categories

See here for Dqm Module Configuration Categories

See here for Dqm Module Configuration Categories

See here for Dqm Module Configuration Categories

See here for Dqm Module Configuration Categories

GET{fhir-endpoint}/Measure/<MeasureId>/$evaluate-measure?practitioner=Practitioner/XYZ
Prefer: response-async

{
   "resourceType": "MeasureReport",
   "id": "c365bc0a-7b76-4869-a99f-71c8c5a1ffb3",
   "meta": {
      "versionId": "1",
      "lastUpdated": "2024-03-04T09:35:08.802-07:00",
      "source": "#pvlOpkXdI88TcUUV"
   },
   "extension": [
      {
         "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.jobId",
         "valueString": "2849ef30-66cb-4f75-add0-110479e2f365"
      },
      {
         "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
         "valueString": "Measure/ColorectalCancerScreeningsFHIR"
      },
      {
         "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
         "valueString": "Colorectal Cancer ScreeningFHIR"
      }
   ],
   "status": "pending",
   "date": "2024-03-04T09:35:08-07:00"
}

GET {fhir-endpoint}/MeasureReport/c365bc0a-7b76-4869-a99f-71c8c5a1ffb3

GET {fhir-endpoint}/MeasureReport/<MeasureReport.id>

See here for Dqm Module Configuration Categories

See here for Dqm Module Configuration Categories

see Set of Patients by Organization Reference for diagram and how Organization reference is used.

See here for Dqm Module Configuration Categories

23.3.14Batch Job Analytics Operation

The $job-analytics operation is a tool to be used with distributed $evaluate-measure operation, and allows for real time progress of batch2 jobs per 'instanceId' created when distributed $evaluate-measure operation is called. The operation will provide an aggregated JSON response body containing a breakdown of batch job 'steps' and the distribution of work-chunks 'status' and durations within each step. This allows end-users to get deeper level of insight into where in the process jobs are progressing.

When a batch job completes processing and reaches status COMPLETED, it will purge all work chunk data associated with it. This means that the $job-analytics operation will not show results for "steps" when complete.

23.3.14.1Batch Job Analytics Parameters

23.3.14.2Batch Job Analytics Distributed Evaluate Measure Steps

Distributed $evaluate-measure operation has a specific sequence of steps it processes as a part of the job. The table below outlines the steps of the job for purposes of understanding progress.

23.3.14.3Batch Job Analytics Work Chunk Status

23.3.14.4Batch Job Analytics Job Status

23.3.14.5Batch Job Analytics Measure Report Response with InstanceId

When triggering a distributed $evaluate-measure request a user gets response back in the form of a Bundle of Measure Reports or a single Measure Report that contains the instanceId value needed for $job-analytics request. This can be found in an extension of the Measure Report. Example shown below.

• Example extension with instanceId

{
  "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.jobId",
  "valueString": "2ecb9110-2c31-4f3e-aed6-67b5fe84f0a3"
}

• Example full response body with instanceId

{
   "resourceType": "MeasureReport",
   "id": "8cfbc000-8330-4d11-b554-12c6e8dc246a",
   "meta": {
      "versionId": "1",
      "lastUpdated": "2024-08-02T10:38:27.935-06:00",
      "source": "#eysjHPPneaNBExsX"
   },
   "extension": [
      {
         "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureId",
         "valueString": "CMS130"
      },
      {
         "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.measureTitle",
         "valueString": "Measure - CMS-130: Colorectal Cancer Screening"
      },
      {
         "url": "http://smilecdr.com/fhir/StructureDefinition/extension-MeasureReport.jobId",
         "valueString": "2ecb9110-2c31-4f3e-aed6-67b5fe84f0a3"
      }
   ],
   "status": "pending",
   "measure": "http://ecqi.healthit.gov/ecqms/Measure/CMS130|0.0.003",
   "date": "2024-08-02T10:38:27-06:00"
}

23.3.14.6Batch Job Analytics Example Request

GET {fhir-endpoint}/$job-analytics?instanceId={Batch2Job InstanceId}

23.3.14.7Batch Job Analytics Example Response

{
   "jobId": "{Batch2Job InstanceId}",
   "jobStatus": "IN_PROGRESS",
   "jobStart": "2024-08-02 10:38:30.893",
   "requestTimeStamp": "2024-08-02T16:40:53.799954Z",
   "steps": [
      {
         "stepChunkCount": 165,
         "stepStart": "2024-08-02 10:40:01.436",
         "stepId": "aggregate-subject-chunks",
         "stepStatus": "GATE_WAITING"
      },
      {
         "stepChunkCount": 1,
         "stepStart": "2024-08-02 10:38:30.893",
         "stepAvg": "INTERVAL '0 00:00:01.317' DAY TO SECOND",
         "stepId": "create-subject-chunks",
         "stepStatus": "COMPLETED",
         "stepEnd": "2024-08-02 10:38:32.21"
      },
      {
         "stepChunkCount": 165,
         "stepStart": "2024-08-02 10:39:30.957",
         "stepAvg": "INTERVAL '0 00:00:03.704533333' DAY TO SECOND",
         "stepId": "evaluate-subject-chunk",
         "stepStatus": "COMPLETED",
         "stepEnd": "2024-08-02 10:40:53.286"
      },
      {
         "stepChunkCount": 8,
         "stepStart": "2024-08-02 10:40:47.01",
         "stepId": "evaluate-subject-chunk",
         "stepStatus": "IN_PROGRESS"
      },
      {
         "stepChunkCount": 37,
         "stepStart": "2024-08-02 10:38:31.849",
         "stepId": "evaluate-subject-chunk",
         "stepStatus": "QUEUED"
      }
   ]
}

23.3.15Spark Evaluate-Measure

Spark Evaluate-Measure provides a premium Big Data processing solution for Quality Measure Evaluations that relies on scaled out parallel processing using a spark compute cluster. This solution is meant to compliment Distributed Measure-Evaluation by giving a larger ceiling of processing power for population scale Quality Measure queries with 100's of Millions to Billions of resources.

It also provides the ability to perform multi-measure processing where users have the option to process as many measures as needed with the same patient population.

23.3.16CMS Quality Reporting

In order to bridge capability between generating Quality Measure Reporting in FHIR and being able to submit Measure Reports to CMS, we have a built in operation that transforms FHIR into Quality Payment Program (QPP) JSON format.

For more information see QPP Reporting.

23.3.17Measure Data Requirements Operation

The data-requirements operation aggregates and returns the parameters and data requirements for the measure and all its dependencies as a single module definition.

• FHIR spec on Data-Requirements

23.3.17.1Data Requirements Parameters

23.3.17.2Example Data Requirements Request

GET {fhir-endpoint}/Measure/<measureID>/$data-requirements?periodStart=2014&periodEnd=2014

23.3.17.3Example Data Requirements Response

{
   "resourceType": "Library",
   "type": {
      "coding": [
         {
            "code": "module-definition"
         }
      ]
   },
   "relatedArtifact": [
      {
         "type": "depends-on",
         "resource": "Library/library-quick-model-definition"
      }
   ],
   "dataRequirement": [
      {
         "type": "Condition",
         "codeFilter": [
            {
               "path": "code",
               "valueSetString": "Other Female Reproductive Conditions"
            }
         ]
      },
      {
         "type": "Patient"
      },
      {
         "type": "Condition",
         "codeFilter": [
            {
               "path": "category",
               "valueCode": [
                  "diagnosis"
               ]
            },
            {
               "path": "clinicalStatus",
               "valueCode": [
                  "confirmed"
               ]
            },
            {
               "path": "code",
               "valueSetString": "2.16.840.1.113883.3.464.1003.102.12.1011"
            }
         ]
      },
      {
         "type": "Condition",
         "codeFilter": [
            {
               "path": "category",
               "valueCode": [
                  "diagnosis"
               ]
            },
            {
               "path": "clinicalStatus",
               "valueCode": [
                  "confirmed"
               ]
            },
            {
               "path": "code",
               "valueSetString": "2.16.840.1.113883.3.464.1003.102.12.1012"
            }
         ]
      },
      {
         "type": "Encounter",
         "codeFilter": [
            {
               "path": "status",
               "valueCode": [
                  "finished"
               ]
            },
            {
               "path": "class",
               "valueCode": [
                  "ambulatory"
               ]
            },
            {
               "path": "type",
               "valueSetString": "2.16.840.1.113883.3.464.1003.101.12.1061"
            }
         ]
      },
      {
         "type": "DiagnosticReport",
         "codeFilter": [
            {
               "path": "diagnosis",
               "valueSetString": "2.16.840.1.113883.3.464.1003.198.12.1012"
            }
         ]
      },
      {
         "type": "Medication",
         "codeFilter": [
            {
               "path": "code",
               "valueSetString": "2.16.840.1.113883.3.464.1003.196.12.1001"
            }
         ]
      },
      {
         "type": "MedicationRequest",
         "codeFilter": [
            {
               "path": "status",
               "valueCode": [
                  "active"
               ]
            },
            {
               "path": "medication.code",
               "valueSetString": "2.16.840.1.113883.3.464.1003.196.12.1001"
            }
         ]
      },
      {
         "type": "MedicationStatement",
         "codeFilter": [
            {
               "path": "status",
               "valueCode": [
                  "completed"
               ]
            },
            {
               "path": "medication.code",
               "valueSetString": "2.16.840.1.113883.3.464.1003.196.12.1001"
            }
         ]
      }
   ]
}

23.3.18Measure Collect Data Operation

The collect-data operation is used to collect the data-of-interest for the given measure.

• FHIR spec on Collect-Data

23.3.18.1Collect Data Parameters

23.3.18.2Example Collect Data Request

• Practitioner example

GET{fhir-endpoint}/Measure/<MeasureId>/$collect-data?practitioner=Practitioner/XYZ&periodStart=2023-01-01&periodEnd=2023-12-31

• Patient Subject example

GET{fhir-endpoint}/Measure/<MeasureId>/$collect-data?subject=Patient/XYZ&periodStart=2023-01-01&periodEnd=2023-12-31

• Group Subject example

GET{fhir-endpoint}/Measure/<MeasureId>/$collect-data?subject=Group/XYZ&periodStart=2023-01-01&periodEnd=2023-12-31

23.3.18.3Example Collect Data Response

• example response

{
   "resourceType": "Parameters",
   "parameter": [
      {
         "name": "measureReport",
         "resource": {
            "resourceType": "MeasureReport",
            "status": "complete",
            "type": "data-collection",
            "measure": "Measure/measure-mrp",
            "patient": {
               "reference": "Patient/Practitioner-2178"
            },
            "period": {
               "start": "2017-01-01T00:00:00+00:00",
               "end": "2017-12-31T00:00:00+00:00"
            },
            "evaluatedResource": [
               {
                  "reference": "Patient/Practitioner-2178"
               },
               {
                  "reference": "Encounter/Encounter-62912"
               },
               {
                  "reference": "Procedure/Procedure-89972"
               }
            ]
         }
      },
      {
         "name": "resource",
         "resource": {
            "resourceType": "Patient",
            "id": "Practitioner-2178",
            "meta": {
               "versionId": "8",
               "lastUpdated": "2018-05-31T07:15:08.000+00:00",
               "profile": [
                  "http://hl7.org/fhir/us/hedis/StructureDefinition/hedis-patient"
               ]
            },
            "text": {
               "status": "generated",
               "div": "<div xmlns=\"http://www.w3.org/1999/xhtml\"><div class=\"hapiHeaderText\">Natalie Huff <b>WALTER </b></div><table class=\"hapiPropertyTable\"><tbody><tr><td>Identifier</td><td>000002418</td></tr><tr><td>Address</td><td><span>623 SW French Place </span><br/><span>Westland </span><span>MI </span></td></tr><tr><td>Date of birth</td><td><span>26 July 1974</span></td></tr></tbody></table></div>"
            },
            "extension": [
               {
                  "url": "http://mihin.org/extension/copyright",
                  "valueString": "Copyright 2014-2017 Michigan Health Information Network Shared Services. Licensed under the Apache License, Version 2.0 (the 'License'); you can not use this file except in compliance with the License. You can obtain a copy of the License at  http://www.apache.org/licenses/LICENSE-2.0.  Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an 'AS IS' BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
               },
               {
                  "url": "http://hl7.org/fhir/us/core/StructureDefinition/us-core-race",
                  "valueCodeableConcept": {
                     "coding": [
                        {
                           "system": "http://hl7.org/fhir/v3/Race",
                           "code": "2106-3",
                           "display": "White"
                        }
                     ]
                  }
               },
               {
                  "url": "http://hl7.org/fhir/us/core/StructureDefinition/us-core-ethnicity",
                  "valueCodeableConcept": {
                     "coding": [
                        {
                           "system": "http://hl7.org/fhir/v3/Ethnicity",
                           "code": "2186-5",
                           "display": "Not Hispanic or Latino"
                        }
                     ]
                  }
               },
               {
                  "url": "http://hl7.org/fhir/us/core/StructureDefinition/us-core-religion",
                  "valueCodeableConcept": {
                     "coding": [
                        {
                           "system": "http://hl7.org/fhir/v3/ReligiousAffiliation",
                           "code": "1020",
                           "display": "Hinduism"
                        }
                     ]
                  }
               }
            ],
            "identifier": [
               {
                  "use": "official",
                  "type": {
                     "coding": [
                        {
                           "system": "http://terminology.hl7.org/CodeSystem/v2-0203",
                           "code": "SB",
                           "display": "Social Beneficiary Identifier"
                        }
                     ],
                     "text": "US Social Security Number"
                  },
                  "system": "http://hl7.org/fhir/sid/us-ssn",
                  "value": "000002418"
               }
            ],
            "active": true,
            "name": [
               {
                  "family": "Walter",
                  "given": [
                     "Natalie",
                     "Huff"
                  ]
               }
            ],
            "telecom": [
               {
                  "system": "phone",
                  "value": "248-555-3608",
                  "use": "home"
               },
               {
                  "system": "phone",
                  "value": "248-555-4649",
                  "use": "mobile"
               }
            ],
            "gender": "female",
            "birthDate": "1974-07-26",
            "address": [
               {
                  "use": "home",
                  "type": "postal",
                  "line": [
                     "623 SW French Place"
                  ],
                  "city": "Westland",
                  "district": "Wayne County",
                  "state": "MI",
                  "postalCode": "48185"
               }
            ]
         }
      },
      {
         "name": "resource",
         "resource": {
            "resourceType": "Encounter",
            "id": "Encounter-62912",
            "meta": {
               "versionId": "1",
               "lastUpdated": "2018-05-31T08:21:50.000+00:00",
               "profile": [
                  "http://hl7.org/fhir/us/hedis/StructureDefinition/hedis-encounter"
               ]
            },
            "extension": [
               {
                  "url": "http://mihin.org/extension/copyright",
                  "valueString": "Copyright 2014-2017 Michigan Health Information Network Shared Services. Licensed under the Apache License, Version 2.0 (the 'License'); you can not use this file except in compliance with the License. You can obtain a copy of the License at  http://www.apache.org/licenses/LICENSE-2.0.  Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an 'AS IS' BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
               }
            ],
            "status": "finished",
            "class": {
               "system": "http://hl7.org/fhir/encounter-class",
               "code": "inpatient",
               "display": "Inpatient"
            },
            "type": [
               {
                  "coding": [
                     {
                        "system": "http://snomed.info/sct",
                        "code": "56876005",
                        "display": "Drug rehabilitation and detoxification"
                     }
                  ]
               }
            ],
            "subject": {
               "reference": "Patient/Practitioner-2178",
               "display": "Natalie Huff Walter"
            },
            "period": {
               "start": "2018-05-29T11:00:00-04:00",
               "end": "2018-05-29T11:00:00-04:00"
            },
            "hospitalization": {
               "dischargeDisposition": {
                  "coding": [
                     {
                        "system": "http://hl7.org/fhir/discharge-disposition",
                        "code": "home",
                        "display": "Home"
                     }
                  ]
               }
            }
         }
      },
      {
         "name": "resource",
         "resource": {
            "resourceType": "Procedure",
            "id": "Procedure-89972",
            "meta": {
               "versionId": "1",
               "lastUpdated": "2018-05-30T23:56:19.000+00:00",
               "profile": [
                  "http://hl7.org/fhir/us/hedis/StructureDefinition/hedis-procedure"
               ]
            },
            "extension": [
               {
                  "url": "http://mihin.org/extension/copyright",
                  "valueString": "Copyright 2014-2017 Michigan Health Information Network Shared Services. Licensed under the Apache License, Version 2.0 (the 'License'); you can not use this file except in compliance with the License. You can obtain a copy of the License at  http://www.apache.org/licenses/LICENSE-2.0.  Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an 'AS IS' BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
               }
            ],
            "identifier": [
               {
                  "system": "mihin.org/fhir/sid/elementId",
                  "value": "ca097928-b18c-406c-a01b-6ad4a321fc28"
               }
            ],
            "status": "completed",
            "code": {
               "coding": [
                  {
                     "system": "http://www.ama-assn.org/go/cpt",
                     "code": "1111F",
                     "display": "Medication reconciliation postdischarge"
                  }
               ]
            },
            "subject": {
               "reference": "Patient/Practitioner-2178",
               "display": "Natalie Huff Walter"
            },
            "performedPeriod": {
               "start": "2018-06-02T14:00:00-05:00",
               "end": "2018-06-02T14:00:00-05:00"
            }
         }
      }
   ]
}

23.3.19DQM Module

To enable both 'Standard' and 'Distributed' evaluate-measure, you will need to turn on the DQM Module on CDR.

See DQM for more details.

23.3.20FAQs

Q: I get an error saying Smile CDR can't locate my library, and I've verified it's on the server.

A: Smile CDR follows the Library conformance requirements defined by the CQF Measures IG, meaning the Library must have a logic-library type, the name and versions of the FHIR Library and CQL Library must match, and the url of the Library must end in the name of the Library.

FHIR Libraries generated from CQL via the IG Publisher follow these requirements automatically.

Q: Does Smile CDR support partitions for evaluation?

A: Yes, though the Measure and associated Resources must be in the same partition as the clinical data being used.