How are Churn Scores Calculated?

Introduction

Chargebee's AI-powered Churn Scoring helps you understand and anticipate customer churn by leveraging historical customer data, behavioral patterns, and advanced machine learning models. It predicts the likelihood of a customer canceling their subscription through churn score. This score enables your teams to proactively engage at-risk customers, personalize retention strategies, and make informed decisions that drive long-term value. This document helps you understand how this score is calculated and continuously improved for accuracy and relevance.

Step 1: Setup & Configuration

The first step is to configure how churn scores will be generated and tailored to your unique business needs. Based on your business requirements you can tweak the implementation for churn scores by configuring the following parameters:

Service Level

This defines the level at which churn is tracked within your organization. Typically, this is also the level at which Business Growth and Performance metrics such as ARR, ARR Growth, Activations, and so on are tracked within your organisation. By default scores are generated for each unique customer record within the Chargebee Billing site, however depending on the requirements you can choose to generate these scores at various levels.

• If you have multi business entity enabled and you track churn across different entities you can choose to generate these scores at a combination of Customer ID and Entity ID such that each customer will now have multiple churn scores, one for each entity.

• If you track churn across product families then you can choose to configure the churn scores generation at a combination of Customer ID and Product Family ID such that each customer record will have multiple churn scores one for each product family.

• At last you can choose to configure the churn scores at a combination of Customer ID and Subscription ID such that each Customer will now have multiple churn scores one for each subscription.

Segmentation

Segmentation lets you divide customers into groups to generate more targeted churn models.

• By default segmentation is disabled for your account. This means a single model is utilised to generate the churn scores for all your subscribers.

• If required, you can choose to enable segmentation using common fields such as MRR, Tenure, etc. In such a case we will create different models for each segment and use them to populate the churn scores for each of your customers.

Prediction Time Period

This determines how far into the future churn is predicted for your customers.

• By default we support generating churn scores for the Next 90 days across all your customers.

• Additional time periods such as Next 30 days, Next 60 days can also be supported based on your specific needs.

Once this information is provided to us as part of the implementation call, we will go ahead and setup the churn scores computation for your company.

Step 2: Model Training & Validation

After the setup and configuration step, the next step is to train the model on your specific data. Below are the steps involved in training and validating the model for a particular time period.

1. Data processing: The first step involves cleaning the available data and performing pre-processing to prepare this for model training and validation. In this step:
   • Data is divided into training, validation, and test set.
     • The training data is utilised to train the model on your specific data.
     • The validation data is used to fine tune and optimise model performance to the churn prediction use case.
     • The test data is unseen data that the model has not seen yet and used to evaluate the model performance against actual events.
   • We also perform certain data transformations to prepare the data for model consumption.
     • Features with high sparsity and high correlation are dropped from the data set.
     • Categorical features with only single category are removed. Those that have multiple categories are consolidated into a few categories by clubbing long tail categories into Others.
2. Training and Tuning: This step picks the training and validation data sets from the output of the previous step and performs hyperparameter optimisation on the validation data set. Output is multiple models trained as the result of given possible hyperparameter combinations. It also stores the best performing model in the model repository for future reference.
3. Evaluation: This step picks up the best performing model from the training and tuning step and performs evaluation on the test data set. It produces a json with all evaluation metrics like AUC, F1, precision, recall, accuracy.
4. Performance Check: In this step, each evaluation metric above is checked against a predefined threshold. Any model that does not pass the performance check step is not registered.
5. Model Registration: Once the performance check passes, the model gets registered in the given model group as a new version.

Step 3: Batch Prediction & Scores Refresh

Once a new version of model is available within the model registry, the same is utilised to generate the churn scores and periodically refresh them within your company. Below are the steps involved in batch prediction and scores refresh:

• Data processing: Responsible for picking up the latest available data from the feature store, and selecting the same features as that of the training data set. Same preprocessing steps performed to clean up the training data set are also performed on the data utilised to generate the churn scores.

• Load Latest Model: This step looks up the given model group and gets the latest available version for evaluation.

• Score Generation: This step uses the above model to generate predictions and feature importances.