34.1    Elastic APM Basics
  • Monitoring
  • 34.0 Monitoring Basics
  • 34.1 Elastic APM Basics
  • 34.2 Externalized Metrics
  • 34.3 OpenTelemetry Integration
    • 34.3    OpenTelemetry Integration   
       Table of Contents

    Externalized Metrics

    34.2.1Externalized Metrics

     

    Prior to the 2023.08 release, metrics appeared in Smile CDR. Metrics have now been externalized using Prometheus (to pull metrics from Smile CDR) and Grafana (to visualize the metrics). The metrics within Smile CDR will be deprecated in a future release, and it is highly recommended to externalize your Smile CDR metrics as soon as possible. In order to externalize metrics in Smile CDR, the following 4 steps must be completed in order.

    34.2.2Step 1 - Install Prometheus

     

    First, install Prometheus by referring to: https://prometheus.io/docs/introduction/first_steps/

    Prometheus is configured using a YAML file. The Prometheus download comes with a sample configuration in a file called prometheus.yml. Modify this file to allow Prometheseus to scrape the metrics from SmileCDR.

    Setup Prometheus with a configuration file:

    • In the global config:
      • Ensure external_labels > monitor is set to smilecdr-monitoring
    • In the scrape_configs:
      • Add a job with job_name: smilecdr to the scrape_configs
      • Ensure metrics path for the smilecdr job is:  /metrics/local/metrics
      • If using basic auth, ensure that username and password are set correctly
      • Ensure targets URL points to your SmileCDR admin_json module URL and port number (default is port 9000)
      • Ensure params.format is ['PROMETHEUS_V2']

    Additional options for the configuration file are available here: https://prometheus.io/docs/prometheus/latest/configuration/configuration/

    A sample prometheus.yml configuration file is attached below that uses basic auth to authenticate the smile server.  If testing locally using Docker, ensure the target for the smilecdr job is host.docker.internal:9000 instead.

    global:
  scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s  # Evaluate rules every 15 seconds. The default is every 1 minute.
  external_labels:
    monitor: 'smilecdr-monitoring'

rule_files:
  # - "first.rules"
  # - "second.rules"

scrape_configs:
  - job_name: prometheus
    static_configs:
      - targets: ['localhost:9090']
    
  - job_name: smilecdr
    metrics_path: '/metrics/local/metrics'
    static_configs:
      - targets: ['localhost:9000']
    basic_auth:
      username: admin
      password: password
    params:
      format: ['PROMETHEUS_V2']

    Deploy Prometheus to Kubernetes:

    A sample YAML file for deployment to Kubernetes. Ensure that the 'hostname:port' value provided in the targets array points to your smilecdr instance.

    ---
apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus-config
data:
  prometheus.yml: |
    global:
      scrape_interval: 15s
      evaluation_interval: 15s
      external_labels:
        monitor: 'smilecdr-monitoring'
    scrape_configs:
      - job_name: 'prometheus'
        static_configs:
          - targets: ['localhost:9090']

      - job_name: smilecdr
        metrics_path: '/json-admin/metrics/local/metrics'
        scheme: https
        static_configs:
          - targets: ['smilecdr.example.com']
        basic_auth:
          username: admin
          password: password
        params:
          format: ['PROMETHEUS_V2']

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: prometheus-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: prometheus
  template:
    metadata:
      labels:
        app: prometheus
    spec:
      containers:
      - name: prometheus
        image: "quay.io/prometheus/prometheus:v2.49.1"
        args:
          - "--config.file=/etc/prometheus/prometheus.yml"
          - "--storage.tsdb.path=/prometheus/"
        ports:
        - containerPort: 9090
        volumeMounts:
        - name: config-volume
          mountPath: /etc/prometheus
        - name: prometheus-storage
          mountPath: /prometheus
      volumes:
      - name: config-volume
        configMap:
          name: prometheus-config
      - name: prometheus-storage
        persistentVolumeClaim:
          claimName: prometheus-server-pvc

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  labels:
    app.kubernetes.io/component: server
    app.kubernetes.io/name: prometheus
    app.kubernetes.io/part-of: prometheus
  name: prometheus-server-pvc
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: "gp2"
  resources:
    requests:
      storage: "8Gi"

    Note that: Above persistentVolumeCliam is used for Amazon Web Services' Elastic Block Store (EBS). The corresponding storageClassName should be defined based on your cloud environment.

    34.2.3Step 2 - Install Grafana

     

    Install Grafana using the instructions found here: https://grafana.com/docs/grafana/latest/setup-grafana/installation/.

    34.2.4Step 3 - Creating a Prometheus data source on Grafana

     

    These instructions were adopted from: https://prometheus.io/docs/visualization/grafana/

    To create a Prometheus data source in Grafana:

    1. Click on the "cogwheel" in the sidebar to open the Configuration menu.
    2. Click on "Data Sources".
    3. Click on "Add data source".
    4. Select "Prometheus" as the type.
    5. Set the appropriate Prometheus server URL (for example, http://localhost:9090/).  If running Prometheus in docker locally for testing purposes, use: http://host.docker.internal:9090.
    6. Adjust other data source settings as desired (for example, choosing the right Access method).
    7. Click "Save & Test" to save the new data source.

    34.2.5Step 4 - Configure Smile CDR

     

    In Grafana, each datasource has a unique ID.  For the Prometheus datasource that you just created, obtain the ID by editing the datasource, and copying everything after the final "/":

    For the following URL: http://localhost:3000/datasources/edit/z0tPZ2-Vz The datasource ID is: z0tPZ2-Vz

    Copy this value, and in SmileCDR in the admin_web console, populate the Prometheus Datasource ID with that value.

    Smile CDR Admin Web

    Navigate to the metrics page in Smile CDR, go to each module you want a dashboard for, and click "Download Grafana JSON".  Unzip the .zip file, you will be uploading the JSON file contained within the zip file to Grafana to create the dashboard in Step 5.  Do this for each module.

    Smile CDR Metrics

    34.2.6Step 5 - Creating a Grafana Dashboard  

     
    1. In the left sidebar, go to the explore tab, and click on "Dashboards"

    Grafana Dashboard

    1. When in dashboards, click "New" button, then "Import"

    Grafana New Dashboard

    1. Click the "Upload JSON file" button and upload the JSON file downloaded from Smile CDR.

    Grafana New Dashboard

    1. Dashboard for the module should now be created and will appear in dashboards.
       34.1    Elastic APM Basics
  • Monitoring
  • 34.0 Monitoring Basics
  • 34.1 Elastic APM Basics
  • 34.2 Externalized Metrics
  • 34.3 OpenTelemetry Integration
    • 34.3    OpenTelemetry Integration