This section contains information about methods for creating data in the CDR.
When a resource is written to the repository (create/update/etc), local resource references that are indexed by at least one SearchParameter can be checked to ensure that the target of the reference is valid.
References that are not indexed by at least one SearchParameter are never checked for target existence. Non-local references (i.e. references where the base URL of the reference target refers to a different FHIR server) are never checked for target existence.
Reference Validation
If the config Enforce Referential Integrity on Write is enabled, reference targets will be checked. For example, if a patient contains a reference to managing organization Organization/FOO but FOO is not a valid ID for an organization on the server then the operation will be blocked unless this property has been disabled. This property can cause confusing results for clients of the server since searches, includes, and other FHIR features may not behave as expected when referential integrity is not preserved. In particular, resource references to target resources that do not exist at the time that the source resource is created will not be indexed, even if the target resource is created later. Disable with caution.
Referential Integrity
If the config Enforce Referential Integrity on Delete is enabled on a FHIR Storage (RDBMS) module, resources can only be deleted if there are no other resources with indexed references to the candidate resource for deletion.
This setting is not available on the MongoDB Storage module.
If you POST to [baseUrl]/Bundle you are submitting the bundle for storage as-is. In other words, the bundle is stored as a Bundle, and the contents inside aren’t looked at by the server (aside from any validation that is enabled). This mode is generally used to store Bundle resources with Bundle.type values such as document, and collection. In its default configuration, Smile CDR will prohibit storing a Bundle with a type value of transaction or batch as this is generally a sign that the client is attempting to perform the operations described below but with an incorrect request URL.
Naming: This is not a FHIR Transaction, but instead is simply a simple resource create where the resource happens to be a Bundle resource.
If you POST to [baseUrl] and your Bundle has a Bundle.type value of transaction you are performing a FHIR “transaction operation”, meaning that all of the individual resources inside the bundle will be processed. It is also possible to include other REST operations such as searches in this kind of bundle. The processing works as an atomic unit, meaning that if anything fails (e.g. invalid data in an individual element) the entire thing will be rolled back.
Naming: This operation is referred to as a FHIR Transaction operation.
If you POST to [baseUrl] and your Bundle has a Bundle.type value of batch, the same processing as the transaction applies, except that individual operations are executed in individual database transactions, so an individual failure doesn’t cause the entire operation to be rolled back. In this case, the response Bundle returned by the server will include status entries indicating the outome for the individual operations within. Note that the batch operation does require the entire Bundle to be valid FHIR at a minimum. This means that it can’t have non-existent resource types in it, malformed datatypes, etc.
Naming: This operation is referred to as a FHIR Batch operation.
Often when batch processing data from multiple sources, you will have data from one source that has references to data from other sources.
For example, a collection of Observation resources could be imported from a lab system data source at the same time that a collection of Patient resources is created from a patient administration data source. The Observation resources would have references to the Patient resources. Under ideal conditions, the Patient resource would process first and be present for the Observation to link to. In the real world however, often it is hard to control the order that transactions occur, and so it might be possible for an Observation to be processed before its Patient. By default this would cause an error since the Observation would have an invalid reference, and nothing would be stored.
There are several helpful strategies for solving this issue:
If the Auto-Create Placeholder Reference Targets setting is enabled in the FHIR Storage module configuration, it is possible to have the server automatically create an empty "placeholder" resource with a pre-assigned ID.
This technique is somewhat less complex than the example above, since it does not require a transaction bundle to be created. With this technique, the ID (not the identifier) of the target resource must be known. For example, the following payload could be POSTed to [baseUrl]/Observation, and would result in the creation of an empty resource with the ID Patient/ABC if one does not already exist. Note that if you want to be able to use this technique with purely numeric resource IDs you will also need to adjust the Client ID Mode.
{
"resourceType": "Observation",
"status": "final",
"code": {
"coding": [ {
"system": "http://loinc.org",
"code": "789-8"
} ]
},
"subject": {
"reference": "Patient/ABC"
},
"valueQuantity": {
"value": 4.12,
"system": "http://unitsofmeasure.org",
"code": "10*12/L"
}
}
If the Auto-Create Placeholder Reference Targets setting is enabled in the FHIR Storage module configuration (as shown above), and the Allow Inline Match URL References Enabled setting is also enabled, you can refine the behavior shown above further.
In this case, it is possible to use an inline match URL instead of a hardcoded resource ID, and you can then achieve similar behavior to the Transaction Bundle use case.
Consider the following Observation being POSTed to /Observation.
{
"resourceType": "Observation",
"status": "final",
"code": {
"coding": [ {
"system": "http://loinc.org",
"code": "789-8"
} ]
},
"subject": {
"reference": "Patient?identifier=http://foo|1234",
"identifier": {
"system": "http://foo",
"value": "1234"
}
},
"valueQuantity": {
"value": 4.12,
"system": "http://unitsofmeasure.org",
"code": "10*12/L"
}
}
In this case, the reference will be treated as a local search (in this case for a Patient with the identifier included in the inline match URL), and executed as such. If the search finds zero results, a new Patient resource will be created. If the inline match URL uses an identifier as it does here, Patient.identifier will be populated with the inline match URL's identifer system and value. The reference will then be automatically replaced with a reference to this new Patient. If the search finds one result, the reference will then be automatically replaced with a reference to the found Patient and no placeholder reference target will be created. Note that we use Patient as an example; this applies to any reference targets that include an Identifier element.
Optionally, Reference.identifier can also provide an identifier. This is useful where the inline match URL uses a SearchParameter other than identifier. When Reference.identifier is provided, Patient.identifier may be populated with the system and value provided (see below). If Reference.identifier is provided, it will be persisted within the referencing resource. This is an important consideration because it exposes information from the reference target in the referencing resource, and it may not reflect later changes to the reference target.
Smile CDR uses the following logic to determine which identifiers get added to the newly created placeholder reference target:
Patient?identifier=http://foo|123) is provided, this identifier will be added to the auto-created placeholder reference target.Reference.identifier is provided, its value will be added to the auto-created placeholder reference target.Reference.identifier are provided, and they include the same system and value, only one copy of that identifier will be added to the auto-created placeholder reference target. If they differ, both will be added.Reference.identifier are provided, no identifier will be added to the auto-created placeholder reference target.If neither an inline match URL with identifier nor Reference.identifier are provided such that the auto-created placeholder reference target does not include an identifier, the reference target cannot be subsequently updated using a conditional update. For this reason, it is recommended to always provide at least one identifier for the reference target where the resource ID is unknown.
A placeholder reference target is just another instance of a given resource type (e.g. Patient). FHIR itself does not have a notion of a "placeholder". Such a resource is minimally populated depending on the mechanism by which it was created. It may have a client-assigned ID or it may have a server-assigned ID and one or more identifiers. To help identify such resources, every auto-created placeholder reference target is populated with the following extension:
"extension": [ {
"url": "http://hapifhir.io/fhir/StructureDefinition/resource-placeholder",
"valueBoolean": true
} ],
When the placeholder is subsequently updated, the extension only appears in the first version of the resource.
In all cases, when the Auto-Create Placeholder Reference Targets setting is enabled, Smile CDR will automatically load custom SearchParameters into the repository that allow searching for any resources that were auto-created as placeholders. The following is an example of such a query:
GET http://localhost:8000/Patient?resource-placeholder=true`
In all cases, the code of the SearchParameter is resource-placeholder.
Refer to Enforcing Uniqueness.
A common part of a new rollout of Smile CDR involves an initial backload of data (or a regular periodic batch load).
In order to maximize performance during this kind of activity, a setting called Mass Ingestion Mode is available. When enabled, this setting tunes the system for maximum write performance by tweaking a number of internal functions:
Internal cache sizes are increased and set to have longer timeouts, with cache space being prioritized for functions that improve write performance (as opposed to the default which is a balance of read/write performance)
When updating resources, the system avoids fetching the current contents of the resource by only performing a comparison of a SHA-256 hash of the contents. This avoids an expensive lookup for every update operation.
This setting is controlled using the Mass Ingestion Mode property of the FHIR Storage (RDBMS) module.