On this page:

22.4smileutil: Synchronize FHIR Servers

 

The synchronize-fhir-servers command may be used to keep one server in sync with another by copying all resources from a source to a target, and deleting any resources on the target which no longer exist on the source.

22.4.1Usage

 
bin/smileutil synchronize-fhir-servers -s "http://source.example.com:8000" -t "http://target.example.com:8000"

22.4.2Options

 
  • -v [version] (or --fhir-version [version]) – (optional) The version of FHIR supported by the server.
  • -s [base URL or pattern] (or --source [base URL or pattern]) – This argument should point to the base URL of the source FHIR server endpoint.
  • --source-basic-auth [username:password](optional) This argument may provide a set of credentials to be sent to the source server via HTTP Basic Auth, in the form "username:password".
  • --source-bearer-token [token](optional) This argument provides a bearer token that will be supplied to the source server in an Authorization header.
  • -t [base URL] (or --target [base URL]) – This argument should point to the base URL of the target FHIR server endpoint.
  • --target-basic-auth [username:password](optional) This argument may provide a set of credentials to be sent to the target server via HTTP Basic Auth, in the form "username:password".
  • --target-bearer-token [token](optional) This argument provides a bearer token that will be supplied to the target server in an Authorization header.
  • -r [resource types] (or --resource-types) – (optional) This argument accepts a comma-separated list of resource types to synchronize. See selecting resources to synchronize below for more information.
  • --resource-ids(optional) This argument accepts a comma-separated list of resource IDs to synchronize. See selecting resources to synchronize below for more information.
  • --thread-count [count](optional) This argument may provide a number of threads to use for processing (default is 10).
  • --prefix-numeric-ids [prefix](optional) This argument may provide a prefix to prepend to any purely numeric resource IDs found on the source server when transmitting to the target server. See Prefixing Numeric IDs for more information.
  • -m [mode] (or --mode [mode]) – (optional) This argument specifies the synchronization mode to use (see Synchronization Modes to learn more about available modes).

22.4.3Selecting Resources to Synchronize

 

By default, the synchronize-fhir-servers command will synchronize all non-infrastructure resources on the server. This means that all resources aside from the following set will be synchronized:

  • CapabilityStatement
  • CodeSystem
  • EventDefinition
  • OperationDefinition
  • SearchParameter
  • Subscription
  • StructureDefinition
  • ConceptMap
  • ValueSet

The list of resources to synchronize may be adjusted using the -r or --resource-types parameter. An example is shown below.

--resource-types Patient,Encounter,Observation

Alternately, a list of resources may be specified explicitly using the --resource-ids parameter. An example is shown below.

---resource-ids Patient/123,Patient/456,Encounter/777

Note that resource-types and resource-ids can not be combined.

22.4.4Using a Search as the Source

 

It is possible to use a FHIR search as the source. This takes the following form:

-s "http://example.org/fhir/Patient?name=smith"

When using this style, all resources are fetched from the search and then uploaded to the target server if necessary (when using COMPARE mode as described below, not all resources will necessarily be uploaded).

22.4.5Using a Directory as the Source

 

It is possible to use a local directory as the source. This takes the following form:

Linux/OSX:

-s "file:///home/test/resources/"

Windows:

-s "file:/C:/Path/to/files/"

When using this style, all files in the given directory are loaded and parsed, and then uploaded to the target server.

When using COMPARE mode as described below, some resources may not be uploaded if they already exist on the target server. When using COPY mode, all resources will be uploaded.

22.4.6Prefixing Numeric IDs

 

When synchronizing between two Smile CDR (or HAPI FHIR) servers, one challenge is that these servers do not normally allow clients to specify IDs for client-created resources where the ID is purely numeric. For example, on such a server, a client is allowed to create a resource and specify the ID Patient/ABC but the ID Patient/123 is reserved for assignment by the server and may not be client assigned.

In this situation, the --prefix-numeric-ids parameter may be used to specify an alphanumeric prefix. For example, if the command is executed as shown below, a resource on the source server with the ID Patient/123 will be saved on the target server as Patient/PFX123, and any other resources linking to this patient will have their references adjusted during the sync as well.

bin/smileutil synchronize-fhir-servers --prefix-numeric-ids PFX -s "http://source.example.com:8000" -t "http://target.example.com:8000"

22.4.7Synchronization Modes

 

There are different mechanisms for synchronizing one server to another, each with its own characteristics.

Mode: COPY

The copy synchronization mode replicates resources using brute force by copying resources one-by-one.

This mode is fast for small servers with a smaller number of resources but is generally inefficient for servers with a large numbers of resources as it requires many individual HTTP transactions.

This mode works by following the two steps below.

  1. If the source is a base URL, a series of FHIR searches are performed on the source server (one for each resource type being synchronized), downloading all resources that should be copied. If the results include multiple pages, all pages are loaded. If the source is a FHIR search or a directory, all matching resources are downloaded.
  2. After each download is complete, resources are uploaded one-by-one.

Note the following considerations:

  • The copy synchronization does not delete any resources on the target server that do not exist on the source server (no attempts are made to read or search for data on the target server).
  • Both downloads from the source server and uploads to the target server are multithreaded for better performance. Adjusting the --thread-count parameter may improve performance.

Mode: COMPARE

The compare synchronization mode replicates resources by downloading all resources from both the source and target systems, and then comparing them locally before replicating.

This mode is often faster for larger numbers of resources, although it requires enough free disk space to hold a copy of all resources from both the source and target servers.

This mode works via the following steps:

  1. If the source is a base URL, a series of queries is made to the source system to fetch all resources for all resource types being synchronized. If the source is a FHIR search or a directory, all matching resources are downloaded.
  2. A series of queries is made to the target system to fetch all resources for all resource types being synchronized.
  3. All resources from the previous two steps are stored locally in a pair of temporary directories.
  4. All resources are hashed.
  5. Hashes are compared to come up with a delta between the two systems.
  6. Any resources that exist on the source server and do not exist on the target server or are not up-to-date on the target server are copied via a FHIR update operation.
  7. Any resources that exist on the target server but do not exist on the source server are deleted.

Note the following considerations:

  • The compare synchronization will delete any resources on the target server that do not exist on the source server if the resource no longer exists on the source server.
  • Resource tags, security labels, and profile declarations are also compared. If necessary they will be added or removed from the target server copy of the resource.

Mode: SYNTHEA

See Synthea Mode below for information on this mode.

22.4.8Uploading Synthea (Synthetic Generated) Resources

 

The SYNTHEA mode is used to upload a standard set of synthetic data generated by Synthea to a FHIR server. This command assumes that the generated files are FHIR Transaction Bundles, and uploads them using a standard transaction operation.

The command uploads files in the correct order, uploading Bundles with supporting data (Practitioners, Organizations, etc.) before uploading individual Patient data.

To use this command, first you must execute Synthea to generate a set of Patients. The following example generates 100 patients.

./run_synthea -p 100

Then, execute the Synchronize FHIR Servers command as follows:

bin/smileutil synchronize-fhir-servers -v dstu3 -m SYNTHEA --source /path/to/synthea/output/fhir --target http://localhost:8000

22.4.9Setting up a Resource Gold Master

 

One common pattern for test systems involves creating a standard set of test resources that a repository can be regularly reset to. Under this scenario, there will be two repositories:

  • A staging repository, which is secured so that only administrative users may read/write to it. This repository stores a complete Gold Master set of resources. This collection is curated by an admin.
  • A public repository, which is publically accessible. This repository is regularly reset to the contents of the staging repository, meaning that public users may make modifications to the test data, but these modifications will be short lived since the repository is regularly reset.

The folowing command may be executed regularly (e.g. using a cron job) in order to replace the contents of the public repository with the contents of the staging repository.

bin/smileutil synchronize-fhir-servers \
  -s http://localhost:8000 \
  -t http://localhost:18000 \
  -v DSTU3 \
  --source-basic-auth "user:password"

You may also want to regularly use the expunge operation to clean up deleted resources on the public server:

curl -X POST "http://localhost:18000/\$expunge" -H "Content-Type: application/json" --data '{"resourceType":"Parameters","parameter":[{"name":"limit","valueInteger":50000},{"name":"expungePreviousVersions","valueBoolean":true},{"name":"expungeDeletedResources","valueBoolean":true}]}'