On this page:

11.1Realtime Export Rules Definition

 

The Realtime Export Rules Definition controls the behaviour of the Realtime Export module. It is composed of some settings, and a collection of transformers, and can be configured either via the Script Text, or the Script File. This is an example Rules Definition:

{
   "retainAllHistory": false,
   "concatenationDelimiter": " - ",
   "overflowStrategy": "TRUNCATE",
   "namedTransformers": {
      "simple-humanname-transformer": {
         "columns": [
            {
               "columnName": "name_use",
               "fhirPath": "use",
               "columnType": "STRING"
            },
            {
               "columnName": "given_names",
               "fhirPath": "given",
               "columnType": "STRING",
               "multiplePrimitiveStrategy": "CONCAT",
               "maximumSize": 20
            },
            {
               "columnName": "family_name",
               "fhirPath": "family",
               "columnType": "STRING"
            }
         ]
      }
   },
   "transformers": [
      {
         "resourceType": "Patient",
         "tableName": "rte_patient",
         "columns": [
            {
               "columnName": "id",
               "fhirPath": "Patient.id",
               "columnType": "STRING"
            },
            {
               "columnName": "first_name",
               "fhirPath": "Patient.name.given.first()",
               "columnType": "STRING"
            },
            {
               "columnName": "version",
               "fhirPath": "Patient.meta.versionId",
               "columnType": "INT"
            }
         ],
         "childTables": [
            {
               "fhirPath": "Patient.name",
               "tableName": "rte_patient_name",
               "childTransformerName": "simple-humanname-transformer"
            }
         ]
      }
   ]
}
  1. retainAllHistory – When disabled, only the current version of each resources is tracked in the remote database. When enabled, historical versions of every table are stored in tables suffixed with _history. For example, if history was enabled, and the remote table to be updated was rte_patient_table, then the RTE module would also try to update the table rte_patient_table_history. The history table never receives delete queries, and so will retain the full history of any resources exported.
  2. concatenationDelimiter – Any fields with multiplePrimitiveStrategy set to CONCAT will use the delimiter defined here.
  3. overflowStrategy – If the maximumSize is set on a column, and the value to be stored in it exceeds that length, then the overflow strategy is applied to the data before storage. The options are DROP and TRUNCATE.
  4. transformers – A collection of objects which define resource transformation rules.
  5. namedTransformers – A common set of transformers that can be used in multiple other transformers.

Transformers are the primary component that you will interact with when configuring the Realtime Export module. Each transformer contains a set of rules and metadata which inform the module as to how to process any given incoming resource type. The remainder of this page discusses transformers in detail.

11.1.1Transformers

 
{
   "resourceType": "Patient",
   "tableName": "remote_patient",
   "columns": [
      {
         "columnName": "id",
         "fhirPath": "Patient.id",
         "columnType": "STRING"
      },
      {
         "columnName": "family_name",
         "fhirPath": "Patient.name.family.first()",
         "columnType": "STRING",
         "maximumSize": 40
      },
      {
         "columnName": "gender",
         "fhirPath": "Patient.gender",
         "columnType": "STRING"
      }
   ]
}

Transformers are configured as part of the Realtime Export Rules Definition and must each minimally have:

  1. resourceType – This is the type of resource which the transformer is applied to. When a message comes in on a channel matching the resource type, the transformer will be applied. Multiple transformers may each refer to the same resource type.
  2. tableName – The name of the table in the remote database where the transformed data will be exported to. Note that each transformer can move data into exactly one remote table.
  3. columns – A list of columns in the remote table which need to be populated by this resource.

Here is the anatomy of the column element:

  1. columnName – The name of the column in the remote table.
  2. fhirPath – The FHIRPath expression to be applied to the incoming resource, in order to retrieve the needed information.
  3. columnType – The database column type in the remote table. This is currently limited to the values defined in ColumnTypeEnum.
  4. maximumSize: This should be set to correspond to the maximum length of the column set in the database for STRING fields. This field is optional, but if set, will use the top-level overflowStrategy defined in the rules.

Let's use this transformer to show a small example of how Realtime Export will operate if a Patient resource is created in the database.

Take for example this Patient:

{
   "resourceType":"Patient",
   "id":"123",
   "meta":{
      "versionId":"1",
      "lastUpdated":"2020-09-15T18:38:07.182+00:00"
   },
   "identifier":[
      {
         "system": "http://example.org/mrn",
         "value": "123"
      },
      {
         "system": "http://example.org/person-id",
         "value": "456"
      }
   ],
   "gender":"male",
   "name":[
      {
         "family":"Scott",
         "given":[
            "Michael",
            "Gary"
         ]
      },
      {
         "family":"Levinson",
         "given":[
            "Astrid"
         ]
      }
   ],
   "address":[
      {
         "use":"home",
         "line":[
            "#4 Privet Drive",
            "Under the staircase"
         ],
         "city":"Little Whinging",
         "district":"Surrey",
         "postalCode":"ABC 123"
      }
   ]
}

When this Patient is received in the Realtime Export module, the following will happen:

  1. First, the module will check all known transformers against the incoming resource.
  2. Finding that our above transformer matches this resource type (the resource is a Patient, and the resourceType of the transformer is set to Patient), processing will begin for this transformer.
  3. For each column in the column list, apply the fhirPath expression against the resource. For example, for the column named family_name, the fhirPath expression Patient.name.family.first() will be applied. This will return the value Scott.
  4. After this has occurred for each of the columns, the module will generate a sql statement to insert the found data into the remote database, which will look something like the following:
INSERT INTO remote_patient (id, first_name, gender, source_resource_id, version) VALUES ('Patient/123', 'Scott', 'male', 'Patient/123', 1)
  1. The generated SQL is then executed.

You will notice that the generated SQL includes two elements that were not considered in the transformer. Both version and source_resource_id fields are automatically included in every transformer, as they are required for internal logic of updates and deletes.

11.1.2Named Transformers

 

There are many times when you may want to reuse common transformation logic. For example, you may want to process a Patient, and have all of its names be stored in a child table. At the same time, you have a transformer for Practitioners, and you also want their names to be stored in their own child table. Instead of repeating the transformer multiple times, you can instead create a namedTransformer, and make use of it in other transformers. The following example shows a full set of Realtime Export rules which fulfill these exact conditions that were just laid out:

{
   "concatenationDelimiter": "|",
   "overflowStrategy": "TRUNCATE",
   "namedTransformers": {
      "simple-humanname-transformer": {
         "columns": [
            {
               "columnName": "name_use",
               "fhirPath": "use",
               "columnType": "STRING"
            },
            {
               "columnName": "given_names",
               "fhirPath": "given",
               "columnType": "STRING",
               "multiplePrimitiveStrategy": "CONCAT",
            },
            {
               "columnName": "family_name",
               "fhirPath": "family",
               "columnType": "STRING"
            }
         ]
      }
   },
   "transformers": [
      {
         "resourceType": "Patient",
         "tableName": "remote_patient",
         "columns": [
            {
               "columnName": "id",
               "fhirPath": "Patient.id",
               "columnType": "STRING"
            },
         ],
         "childTables": [
            {
               "fhirPath": "Patient.name",
               "tableName": "remote_patient_name",
               "childTransformerName": "simple-humanname-transformer"
            }
         ]
      },
      {
         "resourceType": "Practitioner",
         "tableName": "remote_practitioner",
         "columns": [
            {
               "columnName": "id",
               "fhirPath": "Practitioner.id",
               "columnType": "STRING"
            },
         ],
         "childTables": [
            {
               "fhirPath": "Practitioner.name",
               "tableName": "remote_practitioner_name",
               "childTransformerName": "simple-humanname-transformer"
            }
         ]
      }
   ]
}

For this example to be relevant, we must also include a sample practitioner, seen below.

{
   "resourceType":"Practitioner",
   "id":"321",
   "meta":{
      "versionId":"1",
      "lastUpdated":"2020-10-28T18:38:07.182+00:00"
   },
   "gender":"female",
   "name":[
      {
         "family":"Davar",
         "given":[
            "Shallan"
         ]
      }
   ],
   "address":[
      {
         "use":"home",
         "city":"Vedenar",
         "district":"Jah Keved"
      }
   ]
}

In this example, the Realtime Export module will process Patient and Practitioner names using the same rules, despite the end data being stored in separate tables, like so:

Realtime Export Named Transformers Result

11.1.3Managing high-cardinality elements

 

The previous example shows a very simple case, where all the FHIRPath expressions evaluate down to a single element which can be stored in a database, but the reality of FHIR is more complex. For example, a Patient can have 0..* names. In order to handle this, Realtime Export supports multiple strategies for handling high-cardinality data elements.

Multiple-primitive strategy

One option for high-cardinality data is to simply perform extra processing to reduce the elements down to a single element. This can be done by setting the multiplePrimitiveStrategy in the column json. The following is an example

{
   "columnName": "all_first_names",
   "fhirPath": "Patient.name.given",
   "multiplePrimitiveStrategy": "CONCAT"
}

In this example, the FHIRPath expression Patient.name.given has a good chance of returning multiple entries. the multiplePrimitiveStrategy tells Realtime Export which behaviour to use if it encounters multiple values here. In this case, it is set to CONCAT, which will simply concatenate all the values into a single string. Using the example Patient above, this would end up evaluating to Michael|Gary|Astrid. The delimiter used is set in the top-level JSON, in a field called concatenationDelimiter

An alternative to the CONCAT strategy is the FIRST strategy, which would simply return Michael in this case.

Child tables

It is not always desired to use a simple multiple primitive strategy, and you would instead like to use foreign key relationships to manage this data, as one normally does with relational data.

For example, this transformer will add an entry to the remote_patients table, but also add two entries to the remote_patient_names table for our example Patient.

{
   "resourceType": "Patient",
   "tableName": "remote_patient",
   "columns": [
      {
         "columnName": "id",
         "fhirPath": "Patient.id",
         "columnType": "STRING"
      },
      {
         "columnName": "gender",
         "fhirPath": "Patient.gender",
         "columnType": "STRING"
      }
   ],
   "childTables": [
      {
         "tableName": "remote_patient_name",
         "fhirPath": "Patient.name",
         "childTransformer": {
            "columns": [
               {
                  "columnName": "given_names",
                  "fhirPath": "given",
                  "columnType": "STRING",
                  "multiplePrimitiveStrategy": "CONCAT"
               },
               {
                  "columnName": "family_name",
                  "fhirPath": "family",
                  "columnType": "STRING"
               }
            ]
         }
   ]
}

Note that the definition of the child table contains:

  1. tableName – This is much the same as the top-level transformer tableName field, in that it defines the name of the remote table.
  2. fhirPath – The fhirPath which returns the collection of elements. In this case Patient.name will return a collection of HumanNames.
  3. childTransformer – This field contains the transformation rules for each element being iterated over. Can contain columns and also childTables.

Given a child table, the Realtime Export module will apply the child transformer over every element in the collection defined by the fhirPath.

This will generate the following entries in the remote database:

Realtime Export Basic Child Table

You will notice some entries here that were not in the child transformer. Namely, id and parent_reference. The id is an automatically generated UUID, and the parent_reference is automatically added to the resource to be used as a foreign key on the parent table.

This concept of child tables can be repeatedly nested. For example, you could have a transformer that had a child element for each address, and that address element can have a child entry for each address-line. An example of this is shown below:

{
   "resourceType":"Patient",
   "tableName":"remote_patient",
   "columns":[
      {
         "columnName":"id",
         "fhirPath":"Patient.id",
         "columnType":"STRING"
      },
      {
         "columnName":"gender",
         "fhirPath":"Patient.gender",
         "columnType":"STRING"
      }
   ],
   "childTables":[
      {
         "fhirPath":"Patient.address",
         "tableName":"remote_patient_address",
         "childTransformer":{
            "columns":[
               {
                  "columnName":"address_use",
                  "fhirPath":"use",
                  "columnType":"STRING"
               },
               {
                  "columnName":"address_postal",
                  "fhirPath":"postalCode",
                  "columnType":"STRING"
               },
            ],
            "childTables":[
               {
                  "fhirPath":"line",
                  "tableName":"remote_patient_address_line",
                  "childTransformer":{
                     "columns":[
                        {
                           "columnName":"line",
                           "fhirPath":"$this",
                           "columnType":"STRING"
                        }
                     ]
                  }
               }
            ]
         }
      }
   ]
}

There is one special thing to note in this transformer. The nested child table uses a special FHIRPath expression called $this, which refers to the element itself, whatever it happens to be. This transformer will generate the following database entries.

Realtime Export Advanced Child Table