7.1.1Example Online Mode Upgrade

This page shows a start to finish online mode upgrade of a multi-process Smile CDR cluster, running Postgresql as the backing database.

7.1.1.1Steps

Follow these steps to upgrade your server cluster with "zero downtime":

7.1.1.1.1Ensure That You Have Recent Backups

You should ensure that you have recent backups of your Cluster Manager Database as well as all of your FHIR Repository Database(s), before attempting the upgrade. You would typically do this during a maintenance window as part of your normal operational procedures. These will be needed if you run into any problems during the upgrade, you decide not to proceed, and you choose to revert the upgrade by restoring your system from these backups.

7.1.1.1.2Reduce Active Cluster to Servers Running the Older Version of Smile CDR

Using your front-end load balancer or reverse proxy, reduce the number of active servers in your cluster to an acceptable subset of your server cluster, where these remaining active servers are only running the older version of Smile CDR and are still able to serve your incoming requests. If you only have two servers in your cluster, then this will leave just one active server. The active server(s) will remain running the older version of Smile CDR during the Zero Downtime Upgrade process, serving any incoming requests, while a single offline server is upgraded and performs the migration to the newer version of Smile CDR. If you have a high-traffic installation with a large number of servers in your pool, then you may leave several servers on the older version actively serving incoming requests, while you perform this Zero Downtime Upgrade process.

7.1.1.1.3Setting Upgrade Mode Property

Starting with version 2024.02.R01 , you can now set the environment variable CDR_UPGRADE_MODE to true on the non-updated processes in the cluster and restart these processes. When a process is running in CDR_UPGRADE_MODE, consuming from broker channels (e.g. JMS Queues or Kafka topics) is paused, and changes to the configuration are not permitted in the Web Admin Console. If you are running an earlier release of Smile CDR, you may ignore this step.

7.1.1.1.4Migrate the Database

Depending on the database migration mechanism you have selected, perform the database migration.

7.1.1.1.5Upgrade One Smile CDR Process

Using only one Smile CDR process, follow the steps in the updating binaries documentation. This involves shutting down the server and then upgrading the binaries, so you should remove this initial upgrade server from your load balancer and drain it of all active sessions, before shutting it down.

7.1.1.1.6Start The Upgraded Smile CDR Process

Unset the CDR_UPGRADE_MODE environment variable on the upgraded server (unset CDR_UPGRADE_MODE) and then start the server up and monitor the logs while it starts up and it automatically upgrades the DB. Once it completes the startup process and it is fully running, you must verify that all the modules on that newly-upgraded server are working correctly (i.e. no errors reported and no stopped modules).

7.1.1.1.7Reintroduce the Upgraded Process

Switch your front-end load balancer or reverse proxy to stop using the existing "older version" server(s) and start directing incoming traffic to the newly-upgraded server. This may involve a very short window of perceived downtime by your front-end clients, depending on the front-end technology you are using and the method you use to perform this switchover. If you have a high-volume installation, then you can upgrade several non-active servers and have them ready to start accepting traffic, once you are ready to perform the switchover from the older version to the newer version of Smile CDR.

7.1.1.1.8Upgrade The Remaining Servers

Upgrade the rest of the servers in your cluster one-by-one, using the process described above. Ensure that each upgraded server starts up without problems, before proceeding to the next server. Note that for each server, you must monitor the smile.log file while it starts up to ensure that everything works properly and no errors are encountered during the startup phase of this newly-upgraded server. As each server is successfully upgraded to the new version of Smile CDR, it can be added back into your front-end server pool.

7.0 Introduction