There are several specialized logs that can be enabled in order to troubleshoot specific issues within a Smile CDR installation. In the default configuration, only INFO (or higher) level troubleshooting messages will be logged. See below for instructions on enabling DEBUG or TRACE level logging of a troubleshooting logger.
All troubleshooting log messages are directed to the smile-troubleshooting.log log file which is present in the Smile /logs directory.
Troubleshooting messages are also visible on the console.
Troubleshooting messages logged at INFO or higher are also included in the operational smile.log logfile.
You can direct logs to the Docker console, and view them via a call to docker logs.
To include all troubleshooting logs, add the following snippet to your customer-lib/logback-smile-custom.xml file:
<logger name="[logger.name]" level="DEBUG">
<appender-ref ref="STDOUT_SYNC"/>
</logger>
Note that you will have to replace [logger.name] with the name of the logger you would like to output to the console.
For example, if you would like to output the AWS HealthLake Export Troubleshooting log to the console, you would replace [logger.name] with ca.cdr.log.aws_healthlake_export_troubleshooting.
The HL7V2 Troubleshooting Log contains details about processing for HL7 v2.x messages. Enable this log to diagnose parsing and processing issues for HL7 v2.x interfaces.
The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/hl7v2-troubleshooting.log. This file will contain details about the HL7V2 server processing pipeline.
<appender name="HL7V2_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
<file>${smile.basedir}/log/hl7v2-troubleshooting.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${smile.basedir}/log/hl7v2-troubleshooting.log.%i.gz</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>9</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>5MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] R:%X{requestId} %msg%n${log.stackfilter.pattern}</pattern>
</encoder>
</appender>
<!-- Set the level below to TRACE to enable HL7V2 trace mode -->
<logger name="ca.cdr.log.hl7v2_troubleshooting" level="DEBUG">
<appender-ref ref="HL7V2_TROUBLESHOOTING"/>
</logger>
The HTTP Troubleshooting Log contains details about any received HTTP requests including request details, HTTP client details, processing time, etc.
The HTTP Troubleshooting Log is an alternative to using HTTP Access Logs that have been defined on individual modules. Note that HTTP Access Logs are more powerful since they can be defined with much more customizable formats and details, but the HTTP Troubleshooting Log can be enabled and disabled on a running server without needing to perform a restart, so each log has advantages.
The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/http-troubleshooting.log. This file will contain details about the HTTP server processing pipeline.
<appender name="HTTP_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
<file>${smile.basedir}/log/http-troubleshooting.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${smile.basedir}/log/http-troubleshooting.log.%i.gz</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>9</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>5MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] R:%X{requestId} %msg%n${log.stackfilter.pattern}</pattern>
</encoder>
</appender>
<!-- Set the level below to TRACE to enable HTTP trace mode -->
<logger name="ca.cdr.log.http_troubleshooting" level="DEBUG">
<appender-ref ref="HTTP_TROUBLESHOOTING"/>
</logger>
If you are trying to troubleshoot networking issues, you can also enable the HTTP Troubleshooting Log in Trace Mode. This will log additional details about request headers, stream reading, etc. To enable trace mode, change the level from DEBUG to TRACE in both the filter level and the logger level above.
The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/security-troubleshooting.log. This file will contain troubleshooting details for the security layer. This can be handy when trying to determine why a particular operation is or is not being permitted.
<appender name="SECURITY_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
<file>${smile.basedir}/log/security-troubleshooting.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${smile.basedir}/log/security-troubleshooting.log.%i.gz</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>9</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>5MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level R:%X{requestId} %logger{36} - %msg%n${log.stackfilter.pattern}</pattern>
</encoder>
</appender>
<logger name="ca.cdr.log.security_troubleshooting" level="DEBUG">
<appender-ref ref="SECURITY_TROUBLESHOOTING"/>
</logger>
The Subscription Troubleshooting log contains details about the subscription processing and any resources that pass through it. It shows the various steps along the pipeline including queueing/dequeuing, deliveries, and failures.
The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/subscription-troubleshooting.log. This file will contain details about the subscription processing pipeline.
<appender name="SUBSCRIPTION_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
<file>${smile.basedir}/log/subscription-troubleshooting.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${smile.basedir}/log/subscription-troubleshooting.log.%i.gz</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>9</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>5MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level R:%X{requestId} %logger{36} - %msg%n${log.stackfilter.pattern}</pattern>
</encoder>
</appender>
<logger name="ca.cdr.log.subscription_troubleshooting" level="DEBUG">
<appender-ref ref="SUBSCRIPTION_TROUBLESHOOTING"/>
</logger>
The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/livebundle-troubleshooting.log. This log file records the number of LiveBundle records saved and purged when Subscribers are added to or deleted from Watchlists. Aggregation events have a DEBUG level, and even more detail is available at the TRACE loglevel.
<appender name="LIVEBUNDLE_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
<file>${smile.basedir}/log/livebundle-troubleshooting.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${smile.basedir}/log/livebundle-troubleshooting.log.%i.gz</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>9</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>5MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level R:%X{requestId} %logger{36} - %msg%n${log.stackfilter.pattern}</pattern>
</encoder>
</appender>
<logger name="ca.cdr.log.livebundle_troubleshooting" level="DEBUG">
<appender-ref ref="LIVEBUNDLE_TROUBLESHOOTING"/>
</logger>
The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/mdm-troubleshooting.log. This log file provides details
on MDM rule evaluation on incoming resources. MDM outcome events have a DEBUG level, and even more detail is available at the TRACE loglevel.
<appender name="MDM_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
<file>${smile.basedir}/log/mdm-troubleshooting.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${smile.basedir}/log/mdm-troubleshooting.log.%i.gz</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>9</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>5MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level R:%X{requestId} %logger{36} - %msg%n${log.stackfilter.pattern}</pattern>
</encoder>
</appender>
<logger name="ca.uhn.fhir.log.mdm_troubleshooting" level="DEBUG">
<appender-ref ref="MDM_TROUBLESHOOTING"/>
</logger>
The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/channel-import-troubleshooting.log. This log file provides details on ingestion of resources via the Channel Import module.
<appender name="CHANNEL_IMPORT_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
<file>${smile.basedir}/log/channel-import-troubleshooting.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${smile.basedir}/log/channel-import-troubleshooting.log.%i.gz</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>9</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>5MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level R:%X{requestId} %logger{36} - %msg%n${log.stackfilter.pattern}</pattern>
</encoder>
</appender>
<logger name="ca.cdr.log.channel_import_troubleshooting" level="DEBUG">
<appender-ref ref="CHANNEL_IMPORT_TROUBLESHOOTING"/>
</logger>
The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/realtime-export-troubleshooting.log. This log file provides details on Realtime Export processing.
<appender name="REALTIME_EXPORT_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
<file>${smile.basedir}/log/realtime-export-troubleshooting.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${smile.basedir}/log/realtime-export-troubleshooting.log.%i.gz</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>9</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>5MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level R:%X{requestId} %logger{36} - %msg%n${log.stackfilter.pattern}</pattern>
</encoder>
</appender>
<logger name="ca.cdr.log.realtime_export_troubleshooting" level="DEBUG">
<appender-ref ref="REALTIME_EXPORT_TROUBLESHOOTING"/>
</logger>
Note that this example is set to DEBUG level. Realtime export also supports TRACE level, if you are interested in seeing the actual values populated into the SQL calls for debugging purpose. Note that this should never be enabled in a production environment, and should only be used for testing.
If you are having trouble diagnosing routing logic within a FHIR Gateway module, the FHIR Gateway Troubleshooting Log can provide helpful insight.
The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/fhir-gateway-troubleshooting.log. This log file provides details on FHIR Gateway activity.
<appender name="FHIR_GATEWAY_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
<file>${smile.basedir}/log/fhir-gateway-troubleshooting.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${smile.basedir}/log/fhir-gateway-troubleshooting.log.%i.gz</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>9</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>5MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level R:%X{requestId} %logger{36} - %msg%n${log.stackfilter.pattern}</pattern>
</encoder>
</appender>
<logger name="ca.cdr.log.fhirgateway_troubleshooting" level="DEBUG">
<appender-ref ref="FHIR_GATEWAY_TROUBLESHOOTING"/>
</logger>
The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/connection-pool-troubleshooting.log. This log file provides details on Database Connection Pool activity.
DEBUG log level displays stack traces of abandoned connections (if db.connectionpool.log_abandoned and remove_abandoned_on_borrow are both set to true).
<appender name="CONNECTION_POOL_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
<file>${smile.basedir}/log/connection-pool-troubleshooting.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${smile.basedir}/log/connection-pool-troubleshooting.log.%i.gz</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>9</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>5MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level R:%X{requestId} %logger{36} - %msg%n${log.stackfilter.pattern}</pattern>
</encoder>
</appender>
<logger name="ca.cdr.log.connection_pool_troubleshooting" level="DEBUG">
<appender-ref ref="CONNECTION_POOL_TROUBLESHOOTING"/>
</logger>
The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/aws-healthlake-export-troubleshooting.log. This log file provides details on AWS HealthLake Export activity.
<appender name="AWS_HEALTHLAKE_EXPORT_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
<file>${smile.basedir}/log/aws-healthlake-export-troubleshooting.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${smile.basedir}/log/aws-healthlake-export-troubleshooting.log.%i.gz</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>9</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>5MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level R:%X{requestId} %logger{36} - %msg%n${log.stackfilter.pattern}</pattern>
</encoder>
</appender>
<logger name="ca.cdr.log.aws_healthlake_export_troubleshooting" level="DEBUG">
<appender-ref ref="AWS_HEALTHLAKE_EXPORT_TROUBLESHOOTING"/>
</logger>
The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/batch-troubleshooting.log. This log file provides details on the operations of the Batch Framework.
<appender name="BATCH_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
<file>${smile.basedir}/log/batch-troubleshooting.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${smile.basedir}/log/batch-troubleshooting.log.%i.gz</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>9</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>5MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n${log.stackfilter.pattern}</pattern>
</encoder>
</appender>
<logger name="ca.uhn.fhir.log.batch_troubleshooting" level="TRACE">
<appender-ref ref="BATCH_TROUBLESHOOTING"/>
</logger>