Troubleshooting Logs
There are several specialized logs that can be enabled in order to troubleshoot specific issues within a Smile CDR installation. In the default configuration, only INFO (or higher) level troubleshooting messages will be logged. See below for instructions on enabling DEBUG or TRACE level logging of a troubleshooting logger.
All troubleshooting log messages are directed to the smile-troubleshooting.log log file which is present in the Smile /log directory.
Troubleshooting messages logged at INFO or higher are also included in the operational smile.log logfile.
You can direct any log to the Docker console, and view them via a call to docker logs.
To include all troubleshooting logs, uncomment the following snippet in your customer-lib/logback-smile-custom.xml file:
<logger name="ca.cdr.log" level="DEBUG">
<appender-ref ref="STDOUT_SYNC"/>
</logger>
<logger name="ca.uhn.fhir.log" level="DEBUG">
<appender-ref ref="STDOUT_SYNC"/>
</logger>
The HL7V2 Troubleshooting Log contains details about processing for HL7 v2.x messages. Enable this log to diagnose parsing and processing issues for HL7 v2.x interfaces.
The following snippet can be uncommented in your customer-lib/logback-smile-custom.xml file in order to start capturing details about the HL7V2 server processing pipeline in the smile-troubleshooting.log file.
<!-- <logger name="ca.cdr.log.hl7v2_troubleshooting" level="DEBUG"/> -->
The HTTP Troubleshooting Log contains details about any received HTTP requests including request details, HTTP client details, processing time, etc.
The HTTP Troubleshooting Log is an alternative to using HTTP Access Logs that have been defined on individual modules. Note that HTTP Access Logs are more powerful since they can be defined with much more customizable formats and details, but the HTTP Troubleshooting Log can be enabled and disabled on a running server without needing to perform a restart, so each log has advantages.
The following snippet can be uncommented in your customer-lib/logback-smile-custom.xml file in order to start capturing details about the HTTP server processing pipeline in the smile-troubleshooting.log file.
<!-- <logger name="ca.cdr.log.http_troubleshooting" level="DEBUG"/> -->
If you are trying to troubleshoot networking issues, you can also enable the HTTP Troubleshooting Log in Trace Mode. This will log additional details about request headers, stream reading, etc. To enable trace mode, change the level from DEBUG to TRACE in both the filter level and the logger level above.
The following snippet can be uncommented in your customer-lib/logback-smile-custom.xml file in order to start capturing details about the security layer in the smile-troubleshooting.log file. This can be handy when trying to determine why a particular operation is or is not being permitted.
<!-- <logger name="ca.cdr.log.security_troubleshooting" level="DEBUG"/> -->
The Subscription Troubleshooting log contains details about the subscription processing and any resources that pass through it. It shows the various steps along the pipeline including queueing/dequeuing, deliveries, and failures.
The following snippet can be uncommented in your customer-lib/logback-smile-custom.xml file in order to start capturing details about the subscription processing pipeline in the smile-troubleshooting.log file.
<!-- <logger name="ca.cdr.log.subscription_troubleshooting" level="DEBUG"/> -->
The following snippet can be uncommented in your customer-lib/logback-smile-custom.xml file in order to start capturing LiveBundle details in the smile-troubleshooting.log file. These details include the number of LiveBundle records saved and purged when Subscribers are added to or deleted from Watchlists. Aggregation events have a DEBUG level, and even more detail is available at the TRACE loglevel.
<!-- <logger name="ca.cdr.log.livebundle_troubleshooting" level="DEBUG"/> -->
The following snippet can be uncommented in your customer-lib/logback-smile-custom.xml file in order to start capturing details on MDM rule evaluation on incoming resources in the smile-troubleshooting.log file. MDM outcome events have a DEBUG level, and even more detail is available at the TRACE loglevel.
<!-- <logger name="ca.uhn.fhir.log.mdm_troubleshooting" level="DEBUG"/> -->
The following snippet can be uncommented in your customer-lib/logback-smile-custom.xml file in order to start capturing details on ingestion of resources via the Channel Import module in the smile-troubleshooting.log file.
<!-- <logger name="ca.cdr.log.channel_import_troubleshooting" level="DEBUG"/> -->
The following snippet can be uncommented in your customer-lib/logback-smile-custom.xml file in order to start capturing details on Realtime Export processing in the smile-troubleshooting.log file.
<!-- <logger name="ca.cdr.log.realtime_export_troubleshooting" level="DEBUG"/> -->
Note that this example is set to DEBUG level. Realtime export also supports TRACE level, if you are interested in seeing the actual values populated into the SQL calls for debugging purpose. Note that this should never be enabled in a production environment, and should only be used for testing.
If you are having trouble diagnosing routing logic within a FHIR Gateway module, the FHIR Gateway Troubleshooting Log can provide helpful insight.
The following snippet can be uncommented in your customer-lib/logback-smile-custom.xml file in order to start capturing details on FHIR Gateway activity in the smile-troubleshooting.log file.
<!-- <logger name="ca.cdr.log.fhirgateway_troubleshooting" level="DEBUG"/> -->
The following snippet can be uncommented in your customer-lib/logback-smile-custom.xml file in order to start capturing details on Database Connection Pool activity in the smile-troubleshooting.log file.
DEBUG log level displays stack traces of abandoned connections (if db.connectionpool.log_abandoned and remove_abandoned_on_borrow are both set to true).
<!-- <logger name="ca.cdr.log.connection_pool_troubleshooting" level="DEBUG"/> -->
The following snippet can be uncommented in your customer-lib/logback-smile-custom.xml file in order to start capturing details on AWS HealthLake Export activity in the smile-troubleshooting.log file.
<!-- <logger name="ca.cdr.log.aws_healthlake_export_troubleshooting" level="DEBUG"/> -->
The following snippet can be uncommented in your customer-lib/logback-smile-custom.xml file in order to start capturing details on the operations of the Batch Framework in the smile-troubleshooting.log file.
<!-- <logger name="ca.uhn.fhir.log.batch_troubleshooting" level="DEBUG"/> -->
The following snippet can be uncommented in your customer-lib/logback-smile-custom.xml file in order to start capturing details on Narrative Generation activity in the smile-troubleshooting.log file.
<!-- <logger name="ca.uhn.fhir.log.narrative_generation_troubleshooting" level="DEBUG"/> -->
The following snippet can be uncommented in your customer-lib/logback-smile-custom.xml file in order to start capturing details on Terminology mapping activity in the smile-troubleshooting.log file.
<!-- <logger name="ca.uhn.fhir.log.terminology_troubleshooting" level="DEBUG"/> -->
Custom Logging will allow you to choose what pieces of data to track. Information regarding this can be found at custom troubleshooting log.