• Realtime Export
  • 14.0 Realtime Export Overview
  • 14.1 Realtime Export Rules Definition
  • 14.2 Using FHIRPath
  • 14.3 Debezium
    • 14.1    Realtime Export Rules Definition   
       Table of Contents

    Realtime Export Overview

    14.0.1Realtime Export Overview
    Trial

     

    The purpose of the Realtime Export module is to permit realtime extraction of FHIR data from Smile CDR into a remote database. The following diagram shows a quick overview of the architecture of the realtime export process.

    Realtime Export Components

    Before discussing configuration of the module, it is important to note that there are two main modes in which Realtime Export can operate. These two modes are POINTCUT and CHANNEL.

    14.0.1.1Pointcut Mode

    In POINTCUT mode, the Smile CDR Storage module has an interceptor registered on it which forwards all Creates/Updates/Deletes to an internal channel, which Realtime Export consumes. The advantage of this mode is that it requires no external software, and works out of the box. To accomplish this, Realtime Export relies on a Storage module as a dependency.

    Realtime Export Pointcut

    14.0.1.2Channel Mode

    Note that as of 2024.08.R01, CHANNEL support is deprecated. As of 2024.11.R01, POINTCUT mode will be the only available mode.

    In CHANNEL mode, Realtime Export has no knowledge of the Storage module, and instead relies on a third-party piece of software (Debezium) to populate the incoming channel. More information about setting up Debezium can be found here. The advantage of running Realtime Export in this mode is that it is extremely low latency since Debezium relies on directly listening to database changes as they happen. In CHANNEL mode, Realtime Export does not rely on a Storage module. In this mode, you must set channel.name to indicate which channel Debezium is placing messages upon.

    Realtime Export Debezium

    14.0.1.3FHIR Storage Module (Pointcut)

    The FHIR Storage is the source of the data used by Realtime Export. If the source type is set to POINTCUT, then this dependency must be filled. If you are instead using CHANNEL, then Realtime Export module does not require a Storage module dependency. Note that if you do depend on the Storage module, you must enable the Pointcut-based Realtime Export mode so that the module correctly registers all necessary interceptors.

    14.0.1.4Cluster Manager Module

    The Cluster Manager module contains the configuration used to connect to the selected message broker.

    See the Message Brokers documentation for information on how to select and configure a message broker. By default an embedded Apache ActiveMQ server will be used, and this is acceptable for testing purposes but an external broker should be used in a production scenario.

    14.0.2Operational Overview
    Trial

     

    At a high level, Realtime Export listens for all changes to FHIR resources. This includes creates, updates, and deletes. Upon detecting such a change, the Realtime Export module will apply any relevant transformations to the resource, in order to convert it into a form that can be easily consumed by an RDBMS. After this, it will generate and execute the SQL commands necessary to update the remote database. This transformation is done via user-configured transformers.

    14.0.3Limitations
    Trial

     

    Currently there are several limitations of Realtime Export.

    Firstly, the remote database schema must already exist. This means that in order to effectively use realtime export, it will be necessary to spend some time creating the schema manually. During schema creation, there are several needs that must be addressed.

    1. Each top-level table must have an integer field called version. This is needed to handle logical updates of remote data.
    2. Each child table must have a string field called id. This will be automatically filled with a UUID.
    3. Each child table must have a string field called parent_reference, which is constrained by the ID of the parent table it refers to.
    4. Each child table must have a string field called source_resource_id. This is needed to handle logical deletes.
    5. While Realtime Export will generate and execute the SQL required to add data to the remote database, creation of foreign key constraints must be handled by the user.

    Next, it is important to note that each remote table is designed to hold exactly one resource type. This means that in its current implementation, remote tables cannot be partially filled by multiple resources. In general, one resource corresponds to one completed row in one or more remote tables.

    Here is an example of an appropriate schema for realtime export. Note that in future examples, some columns may be omitted for brevity.

    Realtime Export Valid Schema

    14.0.4Troubleshooting
    Trial

     

    The Realtime Export Troubleshooting log can be helpful in diagnosing issues relating to Realtime Export. Furthermore, if a message completely fails processing, the cause of the failure is stored along with the failed message.

  • Realtime Export
  • 14.0 Realtime Export Overview
  • 14.1 Realtime Export Rules Definition
  • 14.2 Using FHIRPath
  • 14.3 Debezium
    • 14.1    Realtime Export Rules Definition