Camel Module Overview

LMA

Apache Camel Introduction

Apache Camel is an Open Source framework that provides robust support for integrating systems.

In Apache Camel, data travelling between systems is represented by a Message, which consists of a body, headers, messageId, and timestamp. Messages participate in a Message Exchange, which is used to represent the request-response pattern.

As a Message travels from system to system, the format of the Message body can vary (i.e. File, JSON, XML, etc.) and Apache Camel provides Type Converters to convert from one data format to another. All data formats available in the Camel Module are listed in the Apache Camel Data Formats section and the Smile Data Formats section.

While a Message is used to represent the data itself, a Route is used to define a series of processing steps a Message should take as it travels from a source to a destination. In Smile, users can define routes using YAML DSL or XML DSL

Processors are another important concept in Apache Camel. Processors can be used to perform logic at any point along a Route. They could be used to translate a message, add headers, perform validation logic, or call a service. Processors are extremely versatile and can be used to perform virtually any logic that can be represented by Java code.

Sometimes it is useful to package related Processors and services together. In Apache Camel, this is called a Component. Components provide access to systems and services that are categorized under a URI scheme. These systems and services can be leveraged as a Message travels along a Route.

For example, the Kafka Component can be used to define the beginning of a Route by consuming Messages from a Kafka topic. It can also be used to produce a Message to a Kafka topic at any point along a Route. All components available in the Camel Module are listed in the Apache Camel Components section and the Smile Component section.

Apache Camel Data Formats

Smile supports all Apache Camel Data Formats.

Smile Data Formats

Smile supports the following custom Data Formats:

• IBaseResource

Apache Camel Components

Core Components

Smile supports all Core Components.

Non-Core Components

Smile supports the following Non-Core Components:

• ActiveMQ
• FHIR
• HTTP
• JDBC
• JMS
• JSONPath
• Kafka
• Netty
• Pulsar
• Netty HTTP
• Spring
• Spring XML
• SQL
• XML Io Dsl
• YAML DSL

Note: Jetty component is currently not provided due to version conflicts with the application jetty library version. Netty-http is provided as a replacement, however, make sure to familiarize with netty stream Note

Adding Other Camel Components

Additional Non-Core Components can be added as needed to Smile CDR to address specific implementation requirements.

To add a Camel Component obtain a copy of the appropriate component jar file and place it in the smilecdr/customerlib folder of your Smile CDR deployment or Docker image. Apache Camel is an open source project and as such jar files can be obtained using Maven or by downloading from a public Maven repository such as MVN Repository. Check the Maven dependency information provided in the Camel component documentation to determine the appropriate Group and Artifact ID information. Also ensure that the version of the jar is compatible with the Camel version in the deployed version of Smile CDR.

Smile CDR also supports Camel Kamelets. To add a Kamelet, first download the Kamelet Source File mentioned in the Kamelet documentation. Normally this will be a yaml file. Create a new sub-folder in the Smile CDR deployment, smilecdr/kamelets and copy the yaml Kamelet Source File to this location. Next obtain copies of appropriate jar files for any dependencies mentioned in the documentation for the Kamelet that are not already included in Smile CDR and copy these to the smilecdr/customerlib folder of your Smile CDR deployment or Docker image. As with Camel components, Camel Kamelet jar files can be downloaded for free from a public Maven repository such as MVN Repository.

Smile Component

The Smile Component is a custom Component with the URI scheme below:

smile:[moduleId]/[processorName]

Which can be broken down into:

• smile: - URI scheme
• [moduleId] - ID of a Smile module
• [processorName] - Name of a Smile module processor followed by optional parameters

See Camel Processors for a list of Smile Processors.

See Camel Converters for a list of Converters provided by Smile for automatic conversion between processors.

Ingesting Broker Messages with Camel Routes

A Camel route that consumes a broker topic can persist messages through two families of processors. They are alternatives, each expecting a different message format, so choose the one that matches the messages on the topic.

FHIR storage processors consume raw FHIR content. These processors require a FHIR Storage module, and the first segment of the smile: URI is that module's ID.

The following example processes incoming Bundle resources and persists them with the Transaction/Batch Bundle Processor (the FHIR Storage module's ID is persistence):

<route id="ingest-fhir">
    <from uri="kafka:ingest-topic?brokers={{env:BROKERS}}"/>
    <to uri="smile:persistence/bundleProcessor"/>
</route>

The Channel Import Processor consumes ResourceOperationMessage envelope messages, the same format consumed by the Channel Import module. Despite its name, this processor runs in the Camel module itself and does not use a Channel Import module.

To persist envelope messages whose payload is a FHIR resource (mediaType of application/fhir+json), the Camel module must declare a dependency on a FHIR Storage module.

The following example processes incoming envelope messages (the Camel module's ID is camel):

<route id="ingest-envelope">
    <from uri="kafka:ingest-topic?brokers={{env:BROKERS}}"/>
    <to uri="smile:camel/channelImportProcessor"/>
</route>

Unlike the Channel Import module, the channelImportProcessor has no built-in retry or failure handling. See the Channel Import Processor documentation for a complete route with retry and failure handling.

Messages can also be ingested without a Camel route by creating a Channel Import module, which subscribes directly to the broker topic named in its Channel Name (channel.name) configuration property.

A topic consumed by a Channel Import module should not also be consumed by a Camel route: the messages will be ingested twice.

Environment Variables in Routes

Deployment-specific values in a route (e.g. a Kafka broker list, host, port, credentials, etc.) can be configured using OS environment variables. Camel reads them in two forms: the {{env:NAME}} property placeholder, and the Simple expression ${env.NAME}.

The {{env:NAME}} property placeholder

Apache Camel's property placeholder substitutes the value of an environment variable wherever it appears in a route, including endpoint URIs:

<route id="kafka-ingest">
	<from uri="kafka:{{env:INBOUND_TOPIC}}?brokers={{env:KAFKA_BROKERS}}"/>
	<convertBodyTo type="java.lang.String"/>
	<to uri="smile:persistence/bundleProcessor"/>
</route>

A colon can be used to supply a default value when the variable is not set:

{{env:KAFKA_BROKERS:localhost:9092}}

When no default is supplied, an unset variable causes route startup to fail with an error naming the missing key, rather than resolving to a blank value.

Environment variables in Simple expressions

The Simple expression ${env.NAME} also reads environment variables, and can be used wherever a Message is available to evaluate against: log, setBody, setHeader, predicates, and toD URIs. It is not available in a from or to URI, which is resolved once at route startup before any Message exists; the {{env:NAME}} placeholder should be used in those positions instead.

Camel Health Checks

Health checks can be defined for a Camel module, by configuring a bean named healthChecks that returns a map of names to health checks. Any health-checks added here will be automatically run with the frequency determined by module.clustermgr.config.stats.heartbeat_persist_frequency_ms (default once every 15 seconds). Server health-checks are available on the Admin Json endpoint, e.g. http://localhost:9000/runtime-status/node-statuses/health-checks. Smile CDR uses the Dropwizard Metrics library to manage health checks. The values of the healthChecks map need to extend com.codahale.metrics.health.HealthCheck.

Note: Don't confuse the HealthCheck class with apache's homonymous class.
You may need to add a dependency as the following to your project to make com.codahale.metrics.health.HealthCheck class visible:

<dependency>
   <groupId>io.dropwizard.metrics</groupId>
   <artifactId>metrics-healthchecks</artifactId>
</dependency>

The context depicted next shows a Spring Context Config class that creates a custom processor and a health check.

The Spring Context Config Class

This is a Spring Framework AnnotationConfigApplicationContext class. It is characterized by having the @Configuration annotation on the class itself, as well as having one or more non-static factory methods annotated with @Bean that create instances of your custom Camel classes.

Smile will automatically bind any @Beans with the following types to the SpringCamelContext registry:

• org.apache.camel.Processor
• org.apache.camel.AggregationStrategy
• org.apache.camel.support.jsse.SSLContextParameters

Implementers can also manually bind custom objects to the SpringCamelContext registry that are not in the list of types above by adding an interceptor to the @Configuration class which implements the CAMEL_SPRING_CAMEL_CONTEXT_PRE_START pointcut. The SpringCamelContextPreStartInterceptor below shows how to bind custom objects to the SpringCamelContext registry.

These @Beans and custom objects will be available to use in your YAML DSL or XML DSL routes.

The following example shows a Spring Context Config class that registers a custom Processor:

_/*-
 * #%L
 * Smile CDR - CDR
 * %%
 * Copyright (C) 2016 - 2026 Smile CDR, Inc.
 * %%
 * All rights reserved.
 * #L%
 */
package com.example.camel;

import ca.cdr.api.fhir.interceptor.CdrHook;
import ca.cdr.api.fhir.interceptor.CdrPointcut;
import ca.uhn.fhir.interceptor.api.Interceptor;
import com.codahale.metrics.health.HealthCheck;
import org.apache.camel.Exchange;
import org.apache.camel.Message;
import org.apache.camel.Predicate;
import org.apache.camel.Processor;
import org.apache.camel.spring.SpringCamelContext;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import java.util.HashMap;
import java.util.Map;

/**
 * A sample @Configuration class that defines custom Camel beans
 */
@Configuration
public class TestCustomAppCtx {

   /**
    * A bean that creates an instance of <code>TestCustomProcessor</code>.
    * Smile will automatically bind this Processor to the <code>SpringCamelContext</code> registry.
    */
   @Bean
   public TestCustomProcessor testCustomProcessor() {
      return new TestCustomProcessor();
   }

   /**
    * This bean is a named list of HealthChecks that will be registered with the server.
    */
   @Bean(name = "healthChecks")
   public Map<String, HealthCheck> healthChecks() {
      Map<String, HealthCheck> retVal = new HashMap<>();
      retVal.put("Remote System", new RemoteSystemHealthCheck());
      return retVal;
   }

   /**
    * A bean that creates an instance of an interceptor having the
    * <code>CAMEL_SPRING_CAMEL_CONTEXT_PRE_START</code> pointcut. This
    * interceptor can be used to bind the custom objects to the
    * <code>SpringCamelContext</code> registry, or make other necessary
    * customizations.
    */
   @Bean
   public SpringCamelContextPreStartInterceptor springCamelContextPreStartInterceptor() {
      return new SpringCamelContextPreStartInterceptor();
   }

   /**
    * A custom health check that extends the HealthCheck class
    */
   public static class RemoteSystemHealthCheck extends HealthCheck {
      @Override
      protected Result check() throws Exception {
         // perform some logic and determine if the system is healthy or not
         return Result.healthy();
      }
   }

   /**
    * A simple custom Camel processor
    */
   public static class TestCustomProcessor implements Processor {
      private static final Logger log = LoggerFactory.getLogger(TestCustomProcessor.class);

      @Override
      public void process(Exchange exchange) throws Exception {

         // get incoming Message
         Message message = exchange.getIn();

         // log incoming Message body
         log.info("Message body: {}", message.getBody(String.class));

         // log incoming Message headers
         log.info("Message headers: {}", message.getHeaders());

         // Modify Message body
         message.setBody("Hello World!");

         // Add Message header
         message.setHeader("some-header-name", "some-header-value");

         // log new Message body
         log.info("Message body: {}", message.getBody(String.class));

         // log new Message headers
         log.info("Message headers: {}", message.getHeaders());
      }
   }

   /**
    * An Interceptor containing the <code>CAMEL_SPRING_CAMEL_CONTEXT_PRE_START</code> pointcut.
    * <br/><br/>
    * This interceptor can be used to bind objects to the <code>SpringCamelContext</code> registry
    * or make other necessary customizations.
    * <br/><br/>
    * <b>Note:</b> Manual binding is not necessary for <code>@Beans</code> of the following types,
    * as Smile binds them automatically:
    * <ul>
    *     <li><code>org.apache.camel.Processor</code></li>
    *     <li><code>org.apache.camel.AggregationStrategy</code></li>
    *     <li><code>org.apache.camel.support.jsse.SSLContextParameters</code></li>
    * </ul>
    */
   @Interceptor
   public static class SpringCamelContextPreStartInterceptor {

      @CdrHook(CdrPointcut.CAMEL_SPRING_CAMEL_CONTEXT_PRE_START)
      public void springCamelContextPreStart(SpringCamelContext theSpringCamelContext) {

         // bind custom Predicate to the SpringCamelContext registry
         theSpringCamelContext.getRegistry().bind("urgentPredicate", new UrgentPredicate());
      }

      /**
       * A Predicate that can be used in a Camel route to filter
       * String message bodies that contain the word "urgent".
       */
      static class UrgentPredicate implements Predicate {
         @Override
         public boolean matches(Exchange theExchange) {
            String body = theExchange.getIn().getBody(String.class);
            return body != null && body.contains("urgent");
         }
      }
   }
}

Packaging Your Camel Module Custom Classes

The Spring Context Config class and all the classes used to create custom Processors, custom health checks and Beans must be packaged into a Java JAR file. If you are using Apache Maven as your build system, this just means you should use a normal project with a packaging of jar.

Deploying Your Camel Module Custom Classes

Once you have created a JAR with all your custom classes, it should be placed in the customerlib/ directory of the Smile CDR installation, and Smile CDR should be restarted. You can then follow the steps below to create a new Camel module:

1. Log into the Web Admin Console.
2. Navigate to the Module Config section (Configuration > Module Config).
3. Create a new module of type Camel.
4. Give the module a sensible ID.
5. Specify your YAML DSL or XML DSL Routes either by adding text to the Camel Routes (Text) field or by adding a file path to a .yaml or .xml file to the Camel Routes (File) field.
6. Under the Spring Context Config Class(es) field, enter the fully qualified class name of your Spring Context Config Class. If you have multiple Configuration classes, list them separated by a space or a comma. For example, in the Example Project below, the fully qualified class name would be com.example.camel.TestCustomAppCtx.

Example Project

An example project for camel can be found on our Demo Projects page.