Message Broker: ActiveMQ

As an alternative to Apache Kafka, Apache ActiveMQ can also be used as a message broker.

ActiveMQ is significantly easier to install and administer than Kafka, but has several disadvantages as well:

• It will perform worse than Kafka for larger data volumes
• It does not contain the same partitioning features as Kafka. Smile CDR uses ActiveMQ message groups to process messages with the same message key in order, but by default ActiveMQ tracks only a limited number of groups per queue.

By default, an embedded instance of Apache ActiveMQ is used. This instance is configured with sensible settings, and is suitable for use in a low volume testing installation only. If you are building a server that will handle production workloads (or even important non-production workloads) you should not rely on the embedded ActiveMQ instance and should consider a dedicated Kafka or ActiveMQ instance instead.

Configuration options for the Smile CDR message broker can be found on the Cluster Manager Message Broker page. A sample set of options to configure a remote broker can be seen below.

module.clustermgr.config.messagebroker.type               =REMOTE_ACTIVEMQ
module.clustermgr.config.messagebroker.address            =tcp://localhost:61616
module.clustermgr.config.messagebroker.username           =SOMEUSER
module.clustermgr.config.messagebroker.password           =SOMEPASS

Message Ordering with Message Groups

ActiveMQ delivers all messages that share a message group to the same consumer, so they are processed in the order they were sent, even when a channel has several concurrent consumers.

Smile CDR sets the JMSXGroupID property of each outgoing subscription message to its message key, which by default is the resource ID without its version (for example, Patient/123). Messages for the same resource are therefore processed in order. Messages without a message key are not grouped.

Audit and transaction log events that Smile CDR writes to the broker (when audit_log.broker.enabled or transactionlog.broker.enabled is set) are not grouped either. This affects only systems that consume those queues.

Message Group Selection

• Per resource: The default. No configuration is needed.
• Custom: Use an interceptor to set the message key to any value computed from the message, such as a patient identifier. See Keeping related messages in order.

Larger groups keep more messages in order, reducing parallelism as a consequence.

Group Limits

By default, ActiveMQ remembers the consumer of only the 1024 most recently used groups on each queue. When more groups than that are active, ActiveMQ forgets the least recently used group, and its next message can go to a different consumer while an earlier message in the group is still being processed, so ordering is no longer guaranteed. If many different resources are updated in quick succession, configure the remote broker to remember every group, at the cost of broker memory that grows with the number of groups. Add a policyEntry for queues to the existing <policyEntries> in activemq.xml:

<policyEntry queue=">">
   <messageGroupMapFactory>
      <simpleMessageGroupMapFactory/>
   </messageGroupMapFactory>
</policyEntry>

See Message Groups in the ActiveMQ documentation for the other group map options.

Resource Limits

You can adjust the memory and storage available to ActiveMQ in the activemq.xml configuration file. This is important if you have a subscription endpoint that is sometimes unreachable and a large volume of data backs up waiting to be delivered to that endpoint. These are the settings in the activemq.xml file you can adjust to increase the amount of data that can be held in the queues.

Note that these settings should be carefully considered, and the usage monitored. If ActiveMQ reaches a memory or disk threshold, throughput in your CDR can be stopped entirely as calls to the message broker will be blocked indefinitely.

<systemUsage>
   <systemUsage>
      <memoryUsage>
         <memoryUsage percentOfJvmHeap="70" />
      </memoryUsage>
      <storeUsage>
         <storeUsage limit="400 gb"/>
      </storeUsage>
      <tempUsage>
         <tempUsage limit="50 gb"/>
      </tempUsage>
   </systemUsage>
</systemUsage>

The <storeUsage> setting controls how much space your queues can occupy when they are not being drained.