On this page:

26.1Production Checklist

 

This page provides considerations for reviewing when preparing a production deployment. Because no two deployments are the same, this page is more of a starting point than a final list, but it can be useful in terms of helping to remember important considerations.

26.1.1Host Server Setup

 

When deploying to a server (e.g. a virtual machine running RHEL or Ubuntu with Smile CDR installed) the following considerations apply:

Ensure that ulimit is set appropriately.

The ulimit setting is used in Linux to limit the number of processes, open files, and other resources that an individual user can have open. On some Linux variants (RHEL in particular) this value can be set to a low default in order to prevent individual processes from consuming too many resources.

The normal operation of a high-throughput Smile CDR installation can open large numbers of files, so it is recommended to ensure that the Maximum Number of Open File Descriptors value (known as nofile in limit.conf) is set appropriately. A value of 5000 is recommended.

See this page for information on configuring limit.conf.

Ensure that Smile CDR does not run as root

Running user applications as the root user is generally considered an anti-pattern in Unix system administration. Smile CDR should always be run using a dedicated user account.

Ensure that NTP is enabled

When issues arise, it is vitally important that you are able to trust timestamps in log files. For this reason, it is critical to have a network time service running on your server.

Disable Unneeded Services

Smile CDR had very minimal platform requirements. Consider disabling mail servers, FTP servers, NFS/Samba demons, etc. if they are not needed.

26.1.2CDR Process Settings

 

Ensure that Smile CDR system logging settings are appropriate

The Smile CDR System Logging can be useful to understanding what is happening with Smile CDR at runtime. Consider adjusting the following:

  • Retention: By default the primary log files will keep a maximum of 30 days worth of logs, but this can be adjusted using the Logback settings. Ensure that the chosen settings are appropriate for your organizational policies.

  • Location: You may wish to dedicate separate disk partitions for logging, or centralize your log files in /var/log. The location of log files defaults to [base dir]/log but this can be changed in your Logback settings.

26.1.3FHIR Server Performance

 

Don't use Embedded Databases

The Smile CDR Cluster Manager and FHIR Storage databases use the embedded H2 database by default, as this is a nice option for testing purposes. However, this solution is not recommended for production use, and should be replaced with a supported database.

Tune the Transaction and Audit Log

The Smile CDR Transaction Log is a useful tool for troubleshooting, but keeping a detailed log of every transaction has an impact on system throughput.

Consider reducing retention time, disabling request/response body capture, or even disabling the transaction log entirely on production systems.

The Smile CDR Audit Log keeps a record of accesses and modifications to data and system settings.

Consider disabling the audit log if it is providing functionality that is duplicated in other parts of your overall solution.

Disable Pretty-Print by Default

FHIR Endpoint modules will pretty-print (i.e. indent) responses by default. This is helpful during development, but increases the size of response payloads and therefore decreases performance. Consider disabling this on production systems. Note that clients may disable pretty printing on a request-by-request basis using the _pretty parameter, but this is often forgotten when left to the client.

26.1.4Monitoring

 

Use a Monitoring Tool

The use of a third-part monitoring tool or framework is recommended. Smile CDR provides many useful hooks that can interact with these tools. One commonly used option is Nagios but there are many others as well.

Monitor Endpoint Availability

Configuring your monitoring tool to regularly check the Endpoint Health endpoint ensures that you will be notified if an endpoint is down.

Monitor System Health

Configuring your monitoring tool to regularly check the Runtime Status Health Checks endpoint for module health is a good way to ensure that the system is functioning smoothly. See Runtime Health Checks for information on this feature.

Monitor Disk Space

If it is not already configured, it is important to ensure that alerts will be raised if a disk becomes critically full.

26.1.5Security

 

Review HTTP Server Setup

The HTTP Server Setup page outlines a number of configuration options on HTTP Endpoint modules that should be set appropriately for your specific environment. Please review this page and ensure that you are using sensible settings. Some settings to consider include:

  • [ ] Ensure that Respect Forward Headers is enabled if your environment includes reverse proxies, load balancers, etc. Verify that client request IP addresses that appear in transaction/audit logs are correct and do not simply correspond to the IP address of the network device.

  • [ ] If you are using TLS/SSL terminated by Smile CDR, consider enabling a protocol/cipher whitelist.

  • [ ] Consider setting Suppress Platform Information and Suppress Error Details on your end-user facing endpoint modules (FHIR Endpoint, SMART Outbound Security, etc.), in order to avoid exposing internal detail about your specific Smile CDR instance.

Review FHIR Security Checklist

The FHIR specification contains a Security Page with a number of best practices and suggestions. These should be considered in detail.

Review User Permissions

Ensure that users have been granted only the permissions necessary in order to allow them to do the task that they need to do. For example:

  • [ ] If possible, avoid granting superuser permissions to users.

  • [ ] If possible, grant FHIR permissions that are as specific as possible. For example, if a user does not need to be able to write data, they should be granted only READ-level permissions.

  • [ ] If you are using the Group resource and compartment-based security, note that the ame Group resource may be in multiple Patient compartments, meaning that if a user has access to a single Patient compartment, they may be able to modify the resource by adding/removing other Patients.

26.1.6Message Broker

 

Note that this section applies only if you are using a message broker (i.e. for Subscription delivery). You may skip it if you are not using a message broker in your configuration.

Pick an Appropriate Broker

The default installation of Smile CDR uses an embedded instance of Apache ActiveMQ. This is good for testing and low volume setting but should be replaced with something more robust for high volume setups.

Either a standalone ActiveMQ instance or a standalone Kafka instance should be created and configured. See Message Brokers for more information.

ActiveMQ: Set Resource Limits

If you are using standalone Apache ActiveMQ as your message broker, it is important to configure the Broker Resource Limits appropriately.

26.1.7Cluster Design

 

Put Administrative Modules on Separate Nodes

The Web Admin Console module should be placed on a separate node if it is being used in a large cluster, so that it can be scaled separately from the end user services (e.g. FHIR endpoint modules).

Disable Unneeded Modules

The default configuration of Smile CDR includes a number of modules. Any modules that are not needed in order to support the production configuration should be archived. For example, you may not need:

  • [ ] The SMART Outbound Security module - If you are not using Smile CDR as an OIDC Identity Provider.

  • [ ] The SMART App Demo Host - If you are not using the demonstration SMART applications.