The Smile CDR transaction log is responsible for creating a temporary log of incoming and outgoing transactions as they enter and leave the system via FHIR REST or HL7 v2 endpoints. Note that transactions loaded through other endpoints such as ETL Import will not necessarily be logged.
The transaction log can be setup in many different ways:
Configuration for the Cluster Manager and the Transaction Log modules is stored in the Node Configuration Properties File.
This is the original implementation of the transaction log. It can store data in both its local storage, as well as send transaction log data to a broker such as ActiveMQ or Kafka. We no longer recommend using the Cluster Manager transaction log, as it can quickly balloon in size, and reduce portability of the Cluster Manager schema.
This module stores Transaction Log events in a dedicated storage layer separate from the Cluster Manager. It supports all the same configuration options as the Cluster Manager, except the configuration for sending data to a broker. Note that you can setup multiple transaction log modules in order to have different data retention periods for events of different types/subtypes. Transaction Logs stored by this module can be viewed in the Web Admin Console's Runtime Transaction Log page.
This module sends transaction logs off to a topic/queue on a remote broker (ActiveMQ/Kafka). No local persistence of the Transaction Log occurs. Events sent to a module of this type cannot be viewed on the Web Admin Console's Runtime Transaction Log page. When multiple steps of a given Transaction Log are sent separately, (such as is the case with MDM or HL7V2 occasionally), the events are tied together with a matching GUID, allowing you to rebuild a full event downstream, as the sum of its parts.
Note that when Kafka is used as a broker, the Transaction Log's GUID is used as the partitioning key.
The following is an example of a message as it would be received by the terminal broker:
{
"payload": {
"initialTimestamp": "2024-08-20T14:00:30.198-04:00",
"type": "FHIR_REQUEST",
"subType": "FHIR_READ",
"outcome": "SUCCESS",
"endpointNodeId": "Master",
"endpointModuleId": "fhir_endpoint",
"processingTime": 0,
"requestId": "YBNnt0RgBMLNfJmF",
"events": [
{
"endpointLocalHost": "127.0.0.1",
"endpointLocalPort": 8000,
"endpointRemoteHost": "127.0.0.1",
"endpointRemotePort": 37490,
"initialTimestamp": "2024-08-20T14:00:30.198-04:00",
"requestUrl": "http://127.0.0.1:8000/Patient/1403",
"requestVerb": "GET",
"type": "ENDPOINT_RECEIVE"
},
{
"body": "{\n \"resourceType\": \"Patient\",\n \"id\": \"1403\",\n \"meta\": {\n \"versionId\": \"1\",\n \"lastUpdated\": \"2024-08-20T13:59:08.692-04:00\",\n \"source\": \"#NX4jAIEtLyvF8Jpf\"\n }\n}",
"bodyType": "FHIR_JSON",
"endpointLocalHost": "127.0.0.1",
"endpointLocalPort": 8000,
"endpointRemoteHost": "127.0.0.1",
"endpointRemotePort": 37490,
"initialTimestamp": "2024-08-20T14:00:30.223-04:00",
"outcome": "SUCCESS",
"type": "ENDPOINT_REPLY",
"responseStatus": 200
}
],
"userModuleId": "local_security",
"userNodeId": "Master",
"userName": "ADMIN",
"userFamilyName": "GenericUser",
"userGivenName": "Admin",
"transactionGuid": "b5b79664-9bc4-441d-b476-81df6afa6262"
}
}
The transaction log stores the following fields for each transaction it receives:
As relevant, a transaction log entry can contain other information, including facts about the endpoint that facilitated the transaction, as well as the initial payload. On the receiving end, these events are:
In terms of replies generated surrounding a transaction, the following fields are recorded:
For more information see the Transaction Log Events API and the Transaction Log Step API.
The following property may be used to control the transaction log.
The following snippet shows this property being used to disable the transaction log in the configuration properties file.
module.clustermgr.config.transactionlog.enabled=false
persist_bodies_in_clustermgr_db – Controls whether payload bodies will be included in the transaction log. The default value is true. If set to false, items such as raw HTTP requests and responses, HL7 v2.x messages, etc., will not be written to the transaction log. Storing payload bodies is helpful for troubleshooting, but has implications in terms of performance (since this potentially means a lot of extra data is saved in the database) and in terms of privacy (since the transaction log is not aware of the meaning of payloads and therefore can not restrict access by subject or other payload properties). This setting is a master setting, and can be further refined using several other options:
persist_endpoint_receive_bodies – Controls whether the request body for incoming requests such as FHIR create operations will be included in the transaction log entries.
persist_endpoint_reply_bodies – Controls whether the response body for incoming requests such as FHIR search operations will be included in the transaction log entries.
truncate_body.bytes – Limits the size of the transaction log entry by limiting the payload size to a certain number of bytes.
retain_transaction_log_days – Controls how many days to keep transaction log entries before deleting them.
By default, the transaction log will store the request and response bodies for transactions flowing through the system.
By default all events will be written to the transaction log. This is useful for troubleshooting, but can quickly make the transaction log very noisy. For this reason, it is often useful to trim certain events out of the log. Note: these transaction bodies may contain PHI, and access should be controlled accordingly.
Events in the Transaction Log can be customized using the Event Whitelist and Event Blacklist properties. If the Event Whitelist property is set, only events matched by the whitelist will be included in the transaction log. If the Event Blacklist is set, any events that are matched by the blacklist will not be processed. Note that the whitelist is processed first, and any events that pass through the whitelist (or all events, if no whitelist is set) are then filtered by the blacklist.
The format of the whitelist/blacklist is a comma separated list of codes, in the format: [Type Code]|[SubType Code]
. See Event Types and SubTypes below for a list of valid event codes. Both the Type Code and SubType Code may be a code from the list of available codes, or the value *
to match all codes.
The following configuration properties are used to control the lists.
event_whitelist – The whitelist.
event_blacklist – The blacklist.
For example, the following settings will cause the transaction log to include ONLY received HL7 v2.x messages, excluding ADT^A28 and ADT^A31 messages.
module.clustermgr.config.event_whitelist=HL7V2_INBOUND|*
module.clustermgr.config.event_blacklist=HL7V2_INBOUND|HL7V2_ADT_A28, HL7V2_INBOUND|HL7V2_ADT_A31
If you are trying to troubleshoot poor performance on a FHIR operation, enabling Performance Tracing with the transaction log as an enabled target can be a helpful way of diagnosing issues.
With this feature enabled, warnings will be added to your transaction log if the system detects certain types of inefficient queries. In addition, if Capture Raw SQL or Equivalent is also enabled, the raw query sent to the database will be included so you can analyze it, look at query plans, etc.
This section lists the event codes for transaction log entries. Every entry in the transaction log will have exactly one type code and one subtype code.
The Type Code can be thought of as the high level category, with the SubType Code being the specific payload type. Note that these codes can have a many-to-many relationship. For example, the HL7V2_ADT_A01
SubType code means that the specific transaction involves an HL7 v2.x ADT^A01 message. This SubType code will be logged with the Type Code of HL7V2_INBOUND
when this type of message is received by Smile CDR, and will be logged with a Type Code of HL7V2_OUTBOUND
when this message is sent by Smile CDR.
CAMEL_PROCESSOR
|
Camel Processor | Indicates transaction log was generated from a Camel Processor |
CDA_EXCHANGE_PLUS
|
CDA Exchange+ | Indicates that a request has been handled by the CDA Exchange+ module. |
CDS_HOOKS_REQUEST
|
CDS Hooks Request | Indicates that a REST client request has been handled by a CDS Hooks Endpoint. |
ETL_IMPORTER
|
ETL Importer | Indicates that a row has been processed during an ETL import job. |
FHIR_REQUEST
|
FHIR Request | Indicates that a FHIR client request has been handled by a Smile CDR FHIR Server Endpoint. |
HL7V2_INBOUND
|
HL7 v2.x Inbound | Indicates that an HL7 v2.x message has been received by Smile CDR. |
HL7V2_OUTBOUND
|
HL7 v2.x Outbound | Indicates that an HL7 v2.x message has been sent by Smile CDR. |
P2P_BATCH_JOB
|
P2P Batch Job | Indicates that a P2P batch job has been handled by Smile CDR. |
SOAP_REQUEST
|
SOAP Request | Indicates that a SOAP client request has been handled by Smile CDR. |
BATCH_
|
Batch Job Analytics($job-analytics ) |
CAMEL_
|
Camel Process |
CDA_
|
CDA Document failed to import |
CDA_
|
CDA Document Imported |
CDS_
|
Services List |
CDS_
|
Service Request |
CDS_
|
Submit Feedback Request |
DOCREF
|
DOCREF ($docref ) |
DQM_
|
Batch Job Analytics($job-analytics ) |
DQM_
|
Qpp Build($qpp-build ) |
FHIR_
|
Apply Codesystem Delta: Add Codes |
FHIR_
|
Apply Codesystem Delta: Remove Codes |
FHIR_
|
FHIR Binary Access - Read ($binary-access-read ) |
FHIR_
|
FHIR Binary Access - Write ($binary-access-write ) |
FHIR_
|
Code Lookup ($lookup ) |
FHIR_
|
CodeSystem Subsumes ($subsumes ) |
FHIR_
|
CodeSystem Validate Code ($validate-code ) |
FHIR_
|
Generate Document ($document ) |
FHIR_
|
Translate Code ($translate ) |
FHIR_
|
Fetch Conformance (/metadata ) |
FHIR_
|
Create Resource |
FHIR_
|
Delete Resource |
FHIR_
|
Fetch Page |
FHIR_
|
History (Instance) |
FHIR_
|
History (System) |
FHIR_
|
History (Type) |
FHIR_
|
Mark all resources for reindexing |
FHIR_
|
Add to Metadata ($meta-add ) |
FHIR_
|
Delete from Metadata ($meta-delete ) |
FHIR_
|
Fetch Metadata ($meta ) |
FHIR_
|
Add Partition ($fhir-management-add-partition) |
FHIR_
|
Apply ('$apply') |
FHIR_
|
Care Gaps($care-gaps ) |
FHIR_
|
Collect Data($collect-data ) |
FHIR_
|
CQL($cql ) |
FHIR_
|
Data Requirements($data-requirements ) |
FHIR_
|
Delete Expunge Operation ($delete-expunge) |
FHIR_
|
Delete Partition ($fhir-management-delete-partition) |
FHIR_
|
Diff Operation ($diff) |
FHIR_
|
Evaluate($evaluate ) |
FHIR_
|
Evaluate Measure($evaluate-measure ) |
FHIR_
|
Evaluate Measures($evaluate-measures ) |
FHIR_
|
Search Entire Encounter Record ($everything ) |
FHIR_
|
Search Entire Patient Chart ($everything ) |
FHIR_
|
Search Everything Related to Patients ($everything ) |
FHIR_
|
Expunge Operation ($expunge) |
FHIR_
|
Extract ('$extract') |
FHIR_
|
Get Resource Counts Operation ($get-resource-counts) |
FHIR_
|
GraphQL Operation ($graphql ) |
FHIR_
|
FHIR Bulk Import ($import) |
FHIR_
|
Initiate FHIR Bulk Export ($export ) |
FHIR_
|
Search for most recent or last n=number of Observation records ($lastn) |
FHIR_
|
List Partitions ($fhir-management-list-partitions) |
FHIR_
|
LiveBundle ($livebundle ) |
FHIR_
|
LiveBundle Subscription Group Add ($livebundle-group-add ) |
FHIR_
|
LiveBundle Subscription Group Delete ($livebundle-group-delete ) |
FHIR_
|
LiveBundle Watchlist ($livebundle-watchlist ) |
FHIR_
|
LiveBundle Watchlist Add ($livebundle-watchlist-add ) |
FHIR_
|
LiveBundle Watchlist Delete ($livebundle-watchlist-delete ) |
FHIR_
|
LiveBundle Watchlist Subscribers ($livebundle-watchlist-subscribers ) |
FHIR_
|
FHIR Patient Match ($match ) |
FHIR_
|
MDM Match Resource ($mdm-match ) |
FHIR_
|
MDM Clear links($mdm-clear ) |
FHIR_
|
MDM Create Link ($mdm-create-link ) |
FHIR_
|
MDM Duplicate Golden Resources ($mdm-duplicate-golden-resources ) |
FHIR_
|
MDM Link History ($mdm-link-history ) |
FHIR_
|
MDM Merge Golden Resources ($mdm-merge-golden-resources ) |
FHIR_
|
MDM Not Duplicate ($mdm-not-duplicate ) |
FHIR_
|
MDM Query Links ($mdm-query-links ) |
FHIR_
|
MDM Batch Processing($mdm-submit ) |
FHIR_
|
MDM Update Link ($mdm-update-link ) |
FHIR_
|
Member match ($member-match ) |
FHIR_
|
Package ('$package') |
FHIR_
|
Poll For Bulk Export Job Status ($export-poll-status ) |
FHIR_
|
Poll For Bulk Import Job Status ($import-poll-status ) |
FHIR_
|
Populate ('$populate') |
FHIR_
|
Prepopulate ('$prepopulate') |
FHIR_
|
Read Partition ($fhir-management-read-partition) |
FHIR_
|
Reindex Operation ($reindex) |
FHIR_
|
Reindex terminology ($reindex-terminology ) |
FHIR_
|
Snapshot ($snapshot ) |
FHIR_
|
Submit attachment operation ($submit-attachment ) |
FHIR_
|
Submit Data($submit-data ) |
FHIR_
|
Suggest Keywords |
FHIR_
|
Generate IPS Document ($summary ) |
FHIR_
|
Manually Trigger Subscription ($trigger-subscription ) |
FHIR_
|
FHIR Operation (Unclassified) |
FHIR_
|
Update Partition ($fhir-management-update-partition) |
FHIR_
|
Patch Resource |
FHIR_
|
Perform reindexing pass |
FHIR_
|
Process Message ($process-message ) |
FHIR_
|
Read Resource |
FHIR_
|
Search Across All Resource Types |
FHIR_
|
Search for Resources |
FHIR_
|
Transaction |
FHIR_
|
Update Resource |
FHIR_
|
Upload External Terminology Code System |
FHIR_
|
Validate Resource ($validate ) |
FHIR_
|
Expand ValueSet ($expand ) |
FHIR_
|
Invalidate Expansion ($invalidate-expansion) |
FHIR_
|
ValueSet Validate Code ($validate-code ) |
FHIR_
|
VRead Resource |
HFQL_
|
Execute HFQL Request |
HL7V2_
|
ADT^A01 (Admit) |
HL7V2_
|
ADT^A02 (Transfer) |
HL7V2_
|
ADT^A03 (Discharge) |
HL7V2_
|
ADT^A04 (Register) |
HL7V2_
|
ADT^A05 (Pre-Admit) |
HL7V2_
|
ADT^A06 (Convert OP to IP) |
HL7V2_
|
ADT^A07 (Convert IP to OP) |
HL7V2_
|
ADT^A08 (Update Visit) |
HL7V2_
|
ADT_A10 (Patient Arriving Tracking) |
HL7V2_
|
ADT^A11 (Cancel Admit) |
HL7V2_
|
ADT^A12 (Cancel Transfer) |
HL7V2_
|
ADT^A13 (Cancel Discharge) |
HL7V2_
|
ADT^A14 (Pending Admit) |
HL7V2_
|
ADT^A15 (Pending Transfer) |
HL7V2_
|
ADT^A16 (Pending Discharge) |
HL7V2_
|
ADT^A17 (Swap Patients) |
HL7V2_
|
ADT^A21 (Patient begins LOA) |
HL7V2_
|
ADT^A22 (Patient returns from LOA) |
HL7V2_
|
ADT^A24 (Link Patient Information) |
HL7V2_
|
ADT^A25 (Cancel Pending Discharge) |
HL7V2_
|
ADT^A27 (Cancel Pending Admit) |
HL7V2_
|
ADT^A28 (Add Patient) |
HL7V2_
|
ADT^A29 (Delete Person Information) |
HL7V2_
|
ADT^A30 (Merge Person Information) |
HL7V2_
|
ADT^A31 (Update Patient) |
HL7V2_
|
ADT^A34 (Merge Patient Information) |
HL7V2_
|
ADT^A37 (Unlink Patient) |
HL7V2_
|
ADT^A38 (Cancel Pre-Admit) |
HL7V2_
|
ADT^A39 (Merge Person - Patient ID) |
HL7V2_
|
ADT^A40 (Merge Patient) |
HL7V2_
|
ADT^A45 (Move Visit Information - Visit Number) |
HL7V2_
|
ADT^A47 (Change Patient Identifier List) |
HL7V2_
|
ADT^A54 (Change Attending) |
HL7V2_
|
ADT^A60 (Update Allergies) |
HL7V2_
|
ADT^A61 (Change Consulting) |
HL7V2_
|
BAR^P01 (Establish Billing Account) |
HL7V2_
|
BAR^P12 (Update Diagnosis/Procedures) |
HL7V2_
|
DFT^P03 (Post Detailed Financial Transaction) |
HL7V2_
|
OMG^O19 (General Clinical Order Message) |
HL7V2_
|
ORM^O01 (General Clinical Order Message) |
HL7V2_
|
ORU^R01 (Unsolicited Result) |
HL7V2_
|
RAS^O17 (Medication Administration) |
HL7V2_
|
RDE^O01 (Medication Order - Legacy) |
HL7V2_
|
RDE^O11 (Medication Order) |
HL7V2_
|
SIU^S12 (Appointment Scheduling) |
HL7V2_
|
SIU^S13 (Appointment Rescheduling) |
HL7V2_
|
SIU^S14 (Appointment Modification) |
HL7V2_
|
SIU^S15 (Appointment Cancellation) |
HL7V2_
|
SIU^S17 (Appointment Deletion) |
HL7V2_
|
SIU^S26 (Patient Did Not Show Up for Scheduled Appointment) |
HL7V2_
|
Unknown Message Trigger |
HL7V2_
|
VXU^V04 (Unsolicited Vaccination Record Update) |
INVALID_
|
Invalid Request |
OPERATION_
|
CDA Import ($import-cda ) |
P2P_
|
P2P batch job request |
UNSPECIFIED
|
Unspecified |
_
|
ValueSet Subsumes (Not Used) |
The transaction log can be configured to place transaction events on a broker channel for consumption by other systems using:
smilecdr-transaction
.When submitting data to the broker, the steps are submitted individually, with a GUID as an identifier for the parent event.
Transaction logs can be configured to display request body contents when viewed in the admin console.
By default, transaction logs will not display body contents, thereby hiding Personal Health Information (PHI) and Personal Identifiable Information (PII).
This can be turned on for debugging purposes by using:
Transaction logs are stored in Coordinated Universal Time (UTC) in the database. When transaction logs are viewed in the list, the time is displayed as in the server local timezone. And when a transaction log is viewed in a detail view, the time is displayed as in the client local timezone.
Note that if the client timezone is switched to be a different timezone than the server, the time displayed in the transaction log list and in the detail transaction event view might contain some inconsistencies. In this case, clear the cache in your browser will resolve the issue.
You can configure multiple transaction logs to be used by the system. This is useful if you want to have different transaction logs for different purposes or retention periods. For example, you might want to have a transaction log that only logs FHIR transactions, and another transaction log that only logs HL7 v2.x transactions. This could be accomplished by setting up two transaction logs, and setting their allowlist/denylist events to reflect what you are interested in storing.
To setup multiple transaction logs you will need to modify the cdr-config-Master.properties file in Properties mode since due to security reasons this cannot be setup in the Web Admin Console. In the properties file, duplicate the existing Transaction Log parameters and change the module name and the database. Then you can modify whitelist and blacklist properties accordingly. In the following example the transaction log will record all FHIR Request transactions except the binary read & write transactions. See the list of Event Types and SubTypes.
#################################################################
# External Transaction Log DB Config
#################################################################
module.transactionsFHIR.type=TRANSACTION_LOG_PERSISTENCE
module.transactionsFHIR.config.db.driver=H2_EMBEDDED
module.transactionsFHIR.config.db.url=jdbc:h2:file:./database/h2_transactionsFHIR;DB_CLOSE_ON_EXIT=FALSE
<other module settings>
module.transactionsFHIR.config.event_whitelist=FHIR_REQUEST|*
module.transactionsFHIR.config.event_blacklist=FHIR_REQUEST|FHIR_BINARY_ACCESS_READ,FHIR_REQUEST|FHIR_BINARY_ACCESS_WRITE
In this next example, the transaction logs for HL7V2 events will be recorded to the first broker topic, and all other events will be recorded to the second broker topic.
#################################################################
# External Transaction Log DB Config for HL7V2 message transaction logs
#################################################################
module.transaction_broker_hl7.type=TRANSACTION_LOG_BROKER
module.transaction_broker_hl7.config.channel.name=hl7v2-events
module.transaction_broker_hl7.config.persist_bodies_in_clustermgr_db=true
module.transaction_broker_hl7.config.event_whitelist=HL7V2|*
#################################################################
# External Transaction Log DB Config for non-HL7V2 message transaction logs
#################################################################
module.transaction_broker.type=TRANSACTION_LOG_BROKER
module.transaction_broker.config.channel.name=other-events
module.transaction_broker.config.persist_bodies_in_clustermgr_db=false
module.transaction_broker.config.event_blacklist=HL7V2|*