Message Broker: Kafka
See the Message Broker page for a description of how message queues work by default in Smile CDR.
Rather than using a JMS message broker, you can configure Smile CDR to use Kafka instead.
Set module.clustermgr.config.messagebroker.type to KAFKA in the Cluster Manager to have Smile CDR use Kafka instead of a JMS message broker for subscription processing.
Smile CDR uses the following configuration properties for Kafka configuration:
module.clustermgr.config.messagebroker.type=KAFKA
module.clustermgr.config.kafka.bootstrap_address=localhost:9092
module.clustermgr.config.kafka.group_id=smilecdr
module.clustermgr.config.kafka.auto_commit=false
module.clustermgr.config.kafka.ack_mode=MANUAL
bootstrap_address is a comma-separated list of host and port pairs that are the addresses of the Kafka brokers in a "bootstrap" Kafka cluster that a Kafka client connects to initially to bootstrap itself.
group_id is the name of the kafka consumer group id used by all Smile CDR Kafka subscription consumers. It's fine to leave this as the default.
auto_commit If this property is set to true, Kafka auto-commits the offsets according to its configuration. If it is false, the Kafka Consumer Ack Mode is used to determine commit behavior. See Committing Offsets
ack_mode specifies the Kafka consumer listener container offset commit behavior. This value is only used if Kafka Auto Commit is false. By default, cdr kafka consumers manually acknowledge every successful message processing back to the Kafka broker. This ensures that no message is ever processed twice. However it adds processing overhead. If downstream consumers can process the same message twice, then it is recommended to set kafka.auto_commit to true (in which case the kafka.ack_mode property will be ignored). See Committing Offsets for a list of options.
Like the JMS Message Broker, Kafka uses the Cluster Manager module consumers_per_matching_queue and consumers_per_delivery_queue properties to determine the number of consumers for each topic. Note that for kafka, the number of consumers should be less than or equal to the number of partitions per topic set on the broker num.partitions configuration property. See Spring Kafka Multi-Threaded Message Consumption.
All Subscription Topic names end with .masterNodeId.masterModuleId where masterNodeId and masterModuleId are the node ID and module ID of the Master Node.
The name of the subscription matching topic is subscription.matching.masterNodeId.masterModuleId.
The name of each rest-hook delivery topic is subscription.delivery.rest.hook.NAME.masterNodeId.masterModuleId where NAME depends on the value of module.subscription.config.delivery_queue_naming.
Similarly, the name of each email delivery topic is subscription.delivery.email.NAME.masterNodeId.masterModuleId.