Smile CDR v2023.08.PRE
On this page:

26.4Troubleshooting Logs

 

There are several specialized logs that can be enabled in order to troubleshoot specific issues within a Smile CDR installation. In the default configuration, only INFO (or higher) level troubleshooting messages will be logged. See below for instructions on enabling DEBUG or TRACE level logging of a troubleshooting logger.

All troubleshooting log messages are directed to the smile-troubleshooting.log log file which is present in the Smile /logs directory. Troubleshooting messages are also visible on the console. Troubleshooting messages logged at INFO or higher are also included in the operational smile.log logfile.

Note that these logs can cause internal bottlenecks and decrease overall performance. They are intended to be enabled only while troubleshooting issues, and should be disabled otherwise.

26.4.1Docker Considerations

 

You can direct logs to the Docker console, and view them via a call to docker logs. To include all troubleshooting logs, add the following snippet to your customer-lib/logback-smile-custom.xml file:

<logger name="[logger.name]" level="DEBUG">
    <appender-ref ref="STDOUT_SYNC"/>
</logger>

Note that you will have to replace [logger.name] with the name of the logger you would like to output to the console. For example, if you would like to output the AWS HealthLake Export Troubleshooting log to the console, you would replace [logger.name] with ca.cdr.log.aws_healthlake_export_troubleshooting.

26.4.2HL7V2 Troubleshooting Log

 

The HL7V2 Troubleshooting Log contains details about processing for HL7 v2.x messages. Enable this log to diagnose parsing and processing issues for HL7 v2.x interfaces.

The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/hl7v2-troubleshooting.log. This file will contain details about the HL7V2 server processing pipeline.

<appender name="HL7V2_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
   <filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
   <file>${smile.basedir}/log/hl7v2-troubleshooting.log</file>
   <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
      <fileNamePattern>${smile.basedir}/log/hl7v2-troubleshooting.log.%i.gz</fileNamePattern>
      <minIndex>1</minIndex>
      <maxIndex>9</maxIndex>
   </rollingPolicy>
   <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
      <maxFileSize>5MB</maxFileSize>
   </triggeringPolicy>
   <encoder>
      <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] R:%X{requestId} %msg%n${log.stackfilter.pattern}</pattern>
   </encoder>
</appender>
<!-- Set the level below to TRACE to enable HL7V2 trace mode -->
<logger name="ca.cdr.log.hl7v2_troubleshooting" level="DEBUG">
   <appender-ref ref="HL7V2_TROUBLESHOOTING"/>
</logger>

26.4.3HTTP Troubleshooting Log

 

The HTTP Troubleshooting Log contains details about any received HTTP requests including request details, HTTP client details, processing time, etc.

The HTTP Troubleshooting Log is an alternative to using HTTP Access Logs that have been defined on individual modules. Note that HTTP Access Logs are more powerful since they can be defined with much more customizable formats and details, but the HTTP Troubleshooting Log can be enabled and disabled on a running server without needing to perform a restart, so each log has advantages.

The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/http-troubleshooting.log. This file will contain details about the HTTP server processing pipeline.

<appender name="HTTP_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
   <filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
   <file>${smile.basedir}/log/http-troubleshooting.log</file>
   <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
      <fileNamePattern>${smile.basedir}/log/http-troubleshooting.log.%i.gz</fileNamePattern>
      <minIndex>1</minIndex>
      <maxIndex>9</maxIndex>
   </rollingPolicy>
   <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
      <maxFileSize>5MB</maxFileSize>
   </triggeringPolicy>
   <encoder>
      <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] R:%X{requestId} %msg%n${log.stackfilter.pattern}</pattern>
   </encoder>
</appender>
<!-- Set the level below to TRACE to enable HTTP trace mode -->
<logger name="ca.cdr.log.http_troubleshooting" level="DEBUG">
   <appender-ref ref="HTTP_TROUBLESHOOTING"/>
</logger>

Including Extra Detail

If you are trying to troubleshoot networking issues, you can also enable the HTTP Troubleshooting Log in Trace Mode. This will log additional details about request headers, stream reading, etc. To enable trace mode, change the level from DEBUG to TRACE in both the filter level and the logger level above.

26.4.4Security Troubleshooting Log

 

The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/security-troubleshooting.log. This file will contain troubleshooting details for the security layer. This can be handy when trying to determine why a particular operation is or is not being permitted.

<appender name="SECURITY_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
   <filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
   <file>${smile.basedir}/log/security-troubleshooting.log</file>
   <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
      <fileNamePattern>${smile.basedir}/log/security-troubleshooting.log.%i.gz</fileNamePattern>
      <minIndex>1</minIndex>
      <maxIndex>9</maxIndex>
   </rollingPolicy>
   <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
      <maxFileSize>5MB</maxFileSize>
   </triggeringPolicy>
   <encoder>
      <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level R:%X{requestId} %logger{36} - %msg%n${log.stackfilter.pattern}</pattern>
   </encoder>
</appender>
<logger name="ca.cdr.log.security_troubleshooting" level="DEBUG">
   <appender-ref ref="SECURITY_TROUBLESHOOTING"/>
</logger>

26.4.5Subscription Troubleshooting Log

 

The Subscription Troubleshooting log contains details about the subscription processing and any resources that pass through it. It shows the various steps along the pipeline including queueing/dequeuing, deliveries, and failures.

The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/subscription-troubleshooting.log. This file will contain details about the subscription processing pipeline.

<appender name="SUBSCRIPTION_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
   <filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
   <file>${smile.basedir}/log/subscription-troubleshooting.log</file>
   <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
      <fileNamePattern>${smile.basedir}/log/subscription-troubleshooting.log.%i.gz</fileNamePattern>
      <minIndex>1</minIndex>
      <maxIndex>9</maxIndex>
   </rollingPolicy>
   <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
      <maxFileSize>5MB</maxFileSize>
   </triggeringPolicy>
   <encoder>
      <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level R:%X{requestId} %logger{36} - %msg%n${log.stackfilter.pattern}</pattern>
   </encoder>
</appender>
<logger name="ca.cdr.log.subscription_troubleshooting" level="DEBUG">
   <appender-ref ref="SUBSCRIPTION_TROUBLESHOOTING"/>
</logger>

26.4.6LiveBundle Troubleshooting Log

 

The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/livebundle-troubleshooting.log. This log file records the number of LiveBundle records saved and purged when Subscribers are added to or deleted from Watchlists. Aggregation events have a DEBUG level, and even more detail is available at the TRACE loglevel.

	<appender name="LIVEBUNDLE_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
		<filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
		<file>${smile.basedir}/log/livebundle-troubleshooting.log</file>
		<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
			<fileNamePattern>${smile.basedir}/log/livebundle-troubleshooting.log.%i.gz</fileNamePattern>
			<minIndex>1</minIndex>
			<maxIndex>9</maxIndex>
		</rollingPolicy>
		<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
			<maxFileSize>5MB</maxFileSize>
		</triggeringPolicy>
		<encoder>
			<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level R:%X{requestId} %logger{36} - %msg%n${log.stackfilter.pattern}</pattern>
		</encoder>
	</appender>
	<logger name="ca.cdr.log.livebundle_troubleshooting" level="DEBUG">
		<appender-ref ref="LIVEBUNDLE_TROUBLESHOOTING"/>
	</logger>

26.4.7MDM Troubleshooting Log

 

The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/mdm-troubleshooting.log. This log file provides details on MDM rule evaluation on incoming resources. MDM outcome events have a DEBUG level, and even more detail is available at the TRACE loglevel.

	<appender name="MDM_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
		<filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
		<file>${smile.basedir}/log/mdm-troubleshooting.log</file>
		<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
			<fileNamePattern>${smile.basedir}/log/mdm-troubleshooting.log.%i.gz</fileNamePattern>
			<minIndex>1</minIndex>
			<maxIndex>9</maxIndex>
		</rollingPolicy>
		<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
			<maxFileSize>5MB</maxFileSize>
		</triggeringPolicy>
		<encoder>
			<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level R:%X{requestId} %logger{36} - %msg%n${log.stackfilter.pattern}</pattern>
		</encoder>
	</appender>
	<logger name="ca.uhn.fhir.log.mdm_troubleshooting" level="DEBUG">
		<appender-ref ref="MDM_TROUBLESHOOTING"/>
	</logger>

26.4.8Channel Import Troubleshooting Log

 

The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/channel-import-troubleshooting.log. This log file provides details on ingestion of resources via the Channel Import module.

	<appender name="CHANNEL_IMPORT_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
		<filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
		<file>${smile.basedir}/log/channel-import-troubleshooting.log</file>
		<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
			<fileNamePattern>${smile.basedir}/log/channel-import-troubleshooting.log.%i.gz</fileNamePattern>
			<minIndex>1</minIndex>
			<maxIndex>9</maxIndex>
		</rollingPolicy>
		<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
			<maxFileSize>5MB</maxFileSize>
		</triggeringPolicy>
		<encoder>
			<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level R:%X{requestId} %logger{36} - %msg%n${log.stackfilter.pattern}</pattern>
		</encoder>
	</appender>
	<logger name="ca.cdr.log.channel_import_troubleshooting" level="DEBUG">
		<appender-ref ref="CHANNEL_IMPORT_TROUBLESHOOTING"/>
	</logger>

26.4.9Realtime Export Troubleshooting Log

 

The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/realtime-export-troubleshooting.log. This log file provides details on Realtime Export processing.

	<appender name="REALTIME_EXPORT_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
		<filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
		<file>${smile.basedir}/log/realtime-export-troubleshooting.log</file>
		<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
			<fileNamePattern>${smile.basedir}/log/realtime-export-troubleshooting.log.%i.gz</fileNamePattern>
			<minIndex>1</minIndex>
			<maxIndex>9</maxIndex>
		</rollingPolicy>
		<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
			<maxFileSize>5MB</maxFileSize>
		</triggeringPolicy>
		<encoder>
			<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level R:%X{requestId} %logger{36} - %msg%n${log.stackfilter.pattern}</pattern>
		</encoder>
	</appender>
	<logger name="ca.cdr.log.realtime_export_troubleshooting" level="DEBUG">
		<appender-ref ref="REALTIME_EXPORT_TROUBLESHOOTING"/>
	</logger>

Note that this example is set to DEBUG level. Realtime export also supports TRACE level, if you are interested in seeing the actual values populated into the SQL calls for debugging purpose. Note that this should never be enabled in a production environment, and should only be used for testing.

26.4.10FHIR Gateway Troubleshooting Log

 

If you are having trouble diagnosing routing logic within a FHIR Gateway module, the FHIR Gateway Troubleshooting Log can provide helpful insight.

The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/fhir-gateway-troubleshooting.log. This log file provides details on FHIR Gateway activity.

<appender name="FHIR_GATEWAY_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
    <file>${smile.basedir}/log/fhir-gateway-troubleshooting.log</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
        <fileNamePattern>${smile.basedir}/log/fhir-gateway-troubleshooting.log.%i.gz</fileNamePattern>
        <minIndex>1</minIndex>
        <maxIndex>9</maxIndex>
    </rollingPolicy>
    <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
        <maxFileSize>5MB</maxFileSize>
    </triggeringPolicy>
    <encoder>
        <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level R:%X{requestId} %logger{36} - %msg%n${log.stackfilter.pattern}</pattern>
    </encoder>
</appender>
<logger name="ca.cdr.log.fhirgateway_troubleshooting" level="DEBUG">
    <appender-ref ref="FHIR_GATEWAY_TROUBLESHOOTING"/>
</logger>

26.4.11Connection Pool Troubleshooting Log

 

The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/connection-pool-troubleshooting.log. This log file provides details on Database Connection Pool activity.

DEBUG log level displays stack traces of abandoned connections (if db.connectionpool.log_abandoned and remove_abandoned_on_borrow are both set to true).

<appender name="CONNECTION_POOL_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
    <file>${smile.basedir}/log/connection-pool-troubleshooting.log</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
        <fileNamePattern>${smile.basedir}/log/connection-pool-troubleshooting.log.%i.gz</fileNamePattern>
        <minIndex>1</minIndex>
        <maxIndex>9</maxIndex>
    </rollingPolicy>
    <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
        <maxFileSize>5MB</maxFileSize>
    </triggeringPolicy>
    <encoder>
        <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level R:%X{requestId} %logger{36} - %msg%n${log.stackfilter.pattern}</pattern>
    </encoder>
</appender>
<logger name="ca.cdr.log.connection_pool_troubleshooting" level="DEBUG">
    <appender-ref ref="CONNECTION_POOL_TROUBLESHOOTING"/>
</logger>

26.4.12AWS HealthLake Export Troubleshooting Log

 

The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/aws-healthlake-export-troubleshooting.log. This log file provides details on AWS HealthLake Export activity.

<appender name="AWS_HEALTHLAKE_EXPORT_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
    <file>${smile.basedir}/log/aws-healthlake-export-troubleshooting.log</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
        <fileNamePattern>${smile.basedir}/log/aws-healthlake-export-troubleshooting.log.%i.gz</fileNamePattern>
        <minIndex>1</minIndex>
        <maxIndex>9</maxIndex>
    </rollingPolicy>
    <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
        <maxFileSize>5MB</maxFileSize>
    </triggeringPolicy>
    <encoder>
        <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level R:%X{requestId} %logger{36} - %msg%n${log.stackfilter.pattern}</pattern>
    </encoder>
</appender>
<logger name="ca.cdr.log.aws_healthlake_export_troubleshooting" level="DEBUG">
    <appender-ref ref="AWS_HEALTHLAKE_EXPORT_TROUBLESHOOTING"/>
</logger>

26.4.13Batch Framework Troubleshooting Log

 

The following snippet can be placed in your customer-lib/logback-smile-custom.xml file in order to enable the creation of a special log file called log/batch-troubleshooting.log. This log file provides details on the operations of the Batch Framework.

	<appender name="BATCH_TROUBLESHOOTING" class="ch.qos.logback.core.rolling.RollingFileAppender">
		<filter class="ch.qos.logback.classic.filter.ThresholdFilter"><level>DEBUG</level></filter>
		<file>${smile.basedir}/log/batch-troubleshooting.log</file>
		<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
			<fileNamePattern>${smile.basedir}/log/batch-troubleshooting.log.%i.gz</fileNamePattern>
			<minIndex>1</minIndex>
			<maxIndex>9</maxIndex>
		</rollingPolicy>
		<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
			<maxFileSize>5MB</maxFileSize>
		</triggeringPolicy>
		<encoder>
			<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n${log.stackfilter.pattern}</pattern>
		</encoder>
	</appender>
	<logger name="ca.uhn.fhir.log.batch_troubleshooting" level="TRACE">
		<appender-ref ref="BATCH_TROUBLESHOOTING"/>
	</logger>