8.2.1Smile CDR Observability with Grafana, OpenTelemetry, Loki & Tempo

This guide walks you through setting up Smile CDR with a complete observability stack using OpenTelemetry, Grafana, Loki, and Tempo.

8.2.1.1Overview

This guide covers:

• Local development setup.
• Logs, metrics, and traces configuration.

8.2.1.2Requirements

• Docker & Docker Compose
• Smile CDR distribution with OpenTelemetry support (version 2024.02 or newer)

8.2.2OpenTelemetry Integration

OpenTelemetry is a framework and toolkit designed to create and manage telemetry data such as traces, metrics, and logs. Starting with the 2024.02 release, Smile CDR includes built-in support for OpenTelemetry Java agent, making it possible to generate telemetry data out-of-the-box.

Currently, Smile CDR generates traces and metrics using the OpenTelemetry Java agent with some Smile-specific customizations.

Full docs: OpenTelemetry Integration Guide.

8.2.3Step 1: Enable OpenTelemetry in Smile CDR

To enable OpenTelemetry instrumentation in Smile CDR, set the environment variable CDR_OTEL when running Smile CDR.

CDR_OTEL=y OTEL_LOGS_EXPORTER=otlp bin/smilecdr start

Or export the variables separately:

export CDR_OTEL=y
export OTEL_LOGS_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4317  # Recommended when running in containers or remote setup
bin/smilecdr start

8.2.4Java Agent Setup (Optional)

To override the default agent configuration:

export JAVA_TOOL_OPTIONS="-javaagent:/path/to/opentelemetry-javaagent.jar"
export OTEL_JAVAAGENT_CONFIGURATION_FILE=/path/to/javaagent.properties

Starting from 2025.05.R01, this override is optional unless using AWS/Azure-specific OpenTelemetry agents.

Sample javaagent.properties file:

otel.service.name=smilecdr
otel.logs.exporter=otlp
otel.javaagent.extensions=./otel/bin/cdr-otel-agent-extension.jar
otel.instrumentation.logback-appender.experimental.capture-mdc-attributes=*
otel.instrumentation.http.server.capture-response-headers=X-Request-ID
otel.jmx.config=./otel/etc/jmx_rules.yaml

To disable log export:

otel.logs.exporter=none

8.2.5Notes on Metrics

Metrics are sent from Smile CDR via OpenTelemetry to Prometheus. Common metrics include:

• smilecdr.session.count
• smilecdr.storage.duration
• smilecdr.authentication.success_count

8.2.6Step 2: Launch Smile CDR with Observability Stack

8.2.6.1Clone the repository

git clone https://gitlab.com/smilecdr-public/smilcdr-grafana.git
cd smilecdr-grafana

8.2.6.2Start the observability stack

docker compose up -d

This starts:

• OpenTelemetry Collector
• Grafana
• Loki
• Tempo
• Prometheus

8.2.6.3Access Grafana

• URL: http://localhost:3000
• Login: admin
• Password: password

8.2.6.4Explore Prebuilt Dashboards

In Grafana, go to Dashboards → Smile CDR where you will find:

• Metrics from Prometheus
• Logs from Loki
• Traces from Tempo
• Health and ingestion panels

8.2.7Troubleshooting

If metrics are missing, check:

• CDR_OTEL=y is set
• OTEL_LOGS_EXPORTER=otlp is set
• otel.logs.exporter=otlp is set in javaagent.properties
• http://localhost:9464/metrics is reachable
• Prometheus targets are up

Example query in Grafana Explore:

smilecdr_session_count

8.2.8Useful Docker Commands

docker logs otel-collector
docker logs loki
docker logs grafana

8.2.9Summary

With this setup, you have full observability of Smile CDR including logs, metrics, and traces using Grafana, Loki, Tempo, and Prometheus.