14.1.1Realtime Export Rules DefinitionTrial
The Realtime Export Rules Definition controls the behaviour of the Realtime Export module. It is composed of some settings, and a collection of transformers, and can be configured either via the Script Text, or the Script File. This is an example Rules Definition:
{
"retainAllHistory": false,
"concatenationDelimiter": " - ",
"overflowStrategy": "TRUNCATE",
"namedTransformers": {
"simple-humanname-transformer": {
"columns": [
{
"columnName": "name_use",
"fhirPath": "use",
"columnType": "STRING"
},
{
"columnName": "given_names",
"fhirPath": "given",
"columnType": "STRING",
"multiplePrimitiveStrategy": "CONCAT",
"maximumSize": 20
},
{
"columnName": "family_name",
"fhirPath": "family",
"columnType": "STRING"
}
]
}
},
"transformers": [
{
"resourceType": "Patient",
"tableName": "rte_patient",
"columns": [
{
"columnName": "id",
"fhirPath": "Patient.id",
"columnType": "STRING"
},
{
"columnName": "first_name",
"fhirPath": "Patient.name.given.first()",
"columnType": "STRING"
},
{
"columnName": "version",
"fhirPath": "Patient.meta.versionId",
"columnType": "INT"
}
],
"childTables": [
{
"fhirPath": "Patient.name",
"tableName": "rte_patient_name",
"childTransformerName": "simple-humanname-transformer"
}
]
}
]
}
retainAllHistory– When disabled, only the current version of each resources is tracked in the remote database. When enabled, historical versions of every table are stored in tables suffixed with_history. For example, if history was enabled, and the remote table to be updated wasrte_patient_table, then the RTE module would also try to update the tablerte_patient_table_history. The history table never receives delete queries, and so will retain the full history of any resources exported.concatenationDelimiter– Any fields withmultiplePrimitiveStrategyset toCONCATwill use the delimiter defined here.overflowStrategy– If themaximumSizeis set on a column, and the value to be stored in it exceeds that length, then the overflow strategy is applied to the data before storage. The options areDROPandTRUNCATE.transformers– A collection of objects which define resource transformation rules.namedTransformers– A common set of transformers that can be used in multiple other transformers.
Transformers are the primary component that you will interact with when configuring the Realtime Export module. Each transformer contains a set of rules and metadata which inform the module as to how to process any given incoming resource type. The remainder of this page discusses transformers in detail.
14.1.2TransformersTrial
{
"resourceType": "Patient",
"tableName": "remote_patient",
"columns": [
{
"columnName": "id",
"fhirPath": "Patient.id",
"columnType": "STRING"
},
{
"columnName": "family_name",
"fhirPath": "Patient.name.family.first()",
"columnType": "STRING",
"maximumSize": 40
},
{
"columnName": "gender",
"fhirPath": "Patient.gender",
"columnType": "STRING"
}
]
}
Transformers are configured as part of the Realtime Export Rules Definition and must each minimally have:
resourceType– This is the type of resource which the transformer is applied to. When a message comes in on a channel matching the resource type, the transformer will be applied. Multiple transformers may each refer to the same resource type.tableName– The name of the table in the remote database where the transformed data will be exported to. Note that each transformer can move data into exactly one remote table.columns– A list of columns in the remote table which need to be populated by this resource.
Here is the anatomy of the column element:
columnName– The name of the column in the remote table.fhirPath– The FHIRPath expression to be applied to the incoming resource, in order to retrieve the needed information.columnType– The database column type in the remote table. This is currently limited to the values defined in ColumnTypeEnum .maximumSize: This should be set to correspond to the maximum length of the column set in the database for STRING fields. This field is optional, but if set, will use the top-leveloverflowStrategydefined in the rules.
Let's use this transformer to show a small example of how Realtime Export will operate if a Patient resource is created in the database.
Take for example this Patient:
{
"resourceType":"Patient",
"id":"123",
"meta":{
"versionId":"1",
"lastUpdated":"2020-09-15T18:38:07.182+00:00"
},
"identifier":[
{
"system": "http://example.org/mrn",
"value": "123"
},
{
"system": "http://example.org/person-id",
"value": "456"
}
],
"gender":"male",
"name":[
{
"family":"Scott",
"given":[
"Michael",
"Gary"
]
},
{
"family":"Levinson",
"given":[
"Astrid"
]
}
],
"address":[
{
"use":"home",
"line":[
"#4 Privet Drive",
"Under the staircase"
],
"city":"Little Whinging",
"district":"Surrey",
"postalCode":"ABC 123"
}
]
}
When this Patient is received in the Realtime Export module, the following will happen:
- First, the module will check all known transformers against the incoming resource.
- Finding that our above transformer matches this resource type (the resource is a Patient, and the resourceType of the transformer is set to Patient), processing will begin for this transformer.
- For each column in the column list, apply the fhirPath expression against the resource. For example, for the column named
family_name, the fhirPath expressionPatient.name.family.first()will be applied. This will return the valueScott. - After this has occurred for each of the columns, the module will generate a sql statement to insert the found data into the remote database, which will look something like the following:
INSERT INTO remote_patient (id, first_name, gender, source_resource_id, version) VALUES ('Patient/123', 'Scott', 'male', 'Patient/123', 1)
- The generated SQL is then executed.
You will notice that the generated SQL includes two elements that were not considered in the transformer. Both version and source_resource_id fields are automatically included in every transformer, as they are required for internal logic of updates and deletes.
14.1.3Named TransformersTrial
There are many times when you may want to reuse common transformation logic. For example, you may want to process a Patient, and have all of its names be stored in a child table. At the same time, you have a transformer for Practitioners, and you also want their names to be stored in their own child table.
Instead of repeating the transformer multiple times, you can instead create a namedTransformer, and make use of it in other transformers. The following example shows a full set of Realtime Export rules which fulfill these exact conditions that were just laid out:
{
"concatenationDelimiter": "|",
"overflowStrategy": "TRUNCATE",
"namedTransformers": {
"simple-humanname-transformer": {
"columns": [
{
"columnName": "name_use",
"fhirPath": "use",
"columnType": "STRING"
},
{
"columnName": "given_names",
"fhirPath": "given",
"columnType": "STRING",
"multiplePrimitiveStrategy": "CONCAT",
},
{
"columnName": "family_name",
"fhirPath": "family",
"columnType": "STRING"
}
]
}
},
"transformers": [
{
"resourceType": "Patient",
"tableName": "remote_patient",
"columns": [
{
"columnName": "id",
"fhirPath": "Patient.id",
"columnType": "STRING"
},
],
"childTables": [
{
"fhirPath": "Patient.name",
"tableName": "remote_patient_name",
"childTransformerName": "simple-humanname-transformer"
}
]
},
{
"resourceType": "Practitioner",
"tableName": "remote_practitioner",
"columns": [
{
"columnName": "id",
"fhirPath": "Practitioner.id",
"columnType": "STRING"
},
],
"childTables": [
{
"fhirPath": "Practitioner.name",
"tableName": "remote_practitioner_name",
"childTransformerName": "simple-humanname-transformer"
}
]
}
]
}
For this example to be relevant, we must also include a sample practitioner, seen below.
{
"resourceType":"Practitioner",
"id":"321",
"meta":{
"versionId":"1",
"lastUpdated":"2020-10-28T18:38:07.182+00:00"
},
"gender":"female",
"name":[
{
"family":"Davar",
"given":[
"Shallan"
]
}
],
"address":[
{
"use":"home",
"city":"Vedenar",
"district":"Jah Keved"
}
]
}
In this example, the Realtime Export module will process Patient and Practitioner names using the same rules, despite the end data being stored in separate tables, like so:
14.1.4Managing high-cardinality elementsTrial
The previous example shows a very simple case, where all the FHIRPath expressions evaluate down to a single element which can be stored in a database, but the reality of FHIR is more complex. For example, a Patient can have 0..* names. In order to handle this, Realtime Export supports multiple strategies for handling high-cardinality data elements.
14.1.4.0.1Multiple-primitive strategy
One option for high-cardinality data is to simply perform extra processing to reduce the elements down to a single element. This can be done by setting the multiplePrimitiveStrategy in the column json. The following is an example
{
"columnName": "all_first_names",
"fhirPath": "Patient.name.given",
"multiplePrimitiveStrategy": "CONCAT"
}
In this example, the FHIRPath expression Patient.name.given has a good chance of returning multiple entries. the multiplePrimitiveStrategy tells Realtime Export which behaviour to use if it encounters multiple values here. In this case, it is set to CONCAT, which will simply concatenate all the values into a single string. Using the example Patient above, this would end up evaluating to
Michael|Gary|Astrid. The delimiter used is set in the top-level JSON, in a field called concatenationDelimiter
An alternative to the CONCAT strategy is the FIRST strategy, which would simply return Michael in this case.
14.1.4.1Child tables
It is not always desired to use a simple multiple primitive strategy, and you would instead like to use foreign key relationships to manage this data, as one normally does with relational data.
For example, this transformer will add an entry to the remote_patients table, but also add two entries to the remote_patient_names table for our example Patient.
{
"resourceType": "Patient",
"tableName": "remote_patient",
"columns": [
{
"columnName": "id",
"fhirPath": "Patient.id",
"columnType": "STRING"
},
{
"columnName": "gender",
"fhirPath": "Patient.gender",
"columnType": "STRING"
}
],
"childTables": [
{
"tableName": "remote_patient_name",
"fhirPath": "Patient.name",
"childTransformer": {
"columns": [
{
"columnName": "given_names",
"fhirPath": "given",
"columnType": "STRING",
"multiplePrimitiveStrategy": "CONCAT"
},
{
"columnName": "family_name",
"fhirPath": "family",
"columnType": "STRING"
}
]
}
]
}
Note that the definition of the child table contains:
tableName– This is much the same as the top-level transformertableNamefield, in that it defines the name of the remote table.fhirPath– The fhirPath which returns the collection of elements. In this casePatient.namewill return a collection of HumanNames.childTransformer– This field contains the transformation rules for each element being iterated over. Can containcolumnsand alsochildTables.
Given a child table, the Realtime Export module will apply the child transformer over every element in the collection defined by the fhirPath.
This will generate the following entries in the remote database:
You will notice some entries here that were not in the child transformer. Namely, id and parent_reference. The id is an automatically generated UUID, and the parent_reference is automatically added to the resource to be used as a foreign key on the parent table.
This concept of child tables can be repeatedly nested. For example, you could have a transformer that had a child element for each address, and that address element can have a child entry for each address-line. An example of this is shown below:
{
"resourceType":"Patient",
"tableName":"remote_patient",
"columns":[
{
"columnName":"id",
"fhirPath":"Patient.id",
"columnType":"STRING"
},
{
"columnName":"gender",
"fhirPath":"Patient.gender",
"columnType":"STRING"
}
],
"childTables":[
{
"fhirPath":"Patient.address",
"tableName":"remote_patient_address",
"childTransformer":{
"columns":[
{
"columnName":"address_use",
"fhirPath":"use",
"columnType":"STRING"
},
{
"columnName":"address_postal",
"fhirPath":"postalCode",
"columnType":"STRING"
},
],
"childTables":[
{
"fhirPath":"line",
"tableName":"remote_patient_address_line",
"childTransformer":{
"columns":[
{
"columnName":"line",
"fhirPath":"$this",
"columnType":"STRING"
}
]
}
}
]
}
}
]
}
There is one special thing to note in this transformer. The nested child table uses a special FHIRPath expression called $this, which refers to the element itself, whatever it happens to be.
This transformer will generate the following database entries.
