Build a Retention Prioritizer

Build a churn model and capacity-aware retention queue from telco data.

Introduction

30 Second Summary

A retention team can only contact so many customers each week. Without a clear priority order, high-value customers can leave while attention goes elsewhere.

In this project, you will build a retention prioritizer in Google Colab that turns telco customer records into a ranked outreach queue. The finished workflow connects each customer's churn probability with a limited campaign capacity.

What You'll Build

You will run your notebook and see raw customer records become a ranked queue that tells a retention team who to contact first.

By the end of this project, you'll have:

  • A customer segment chart that shows which groups have the highest observed churn rates.
  • An evaluated probability model that exposes missed churners in a confusion matrix. You can compare average precision, Brier score, and probability calibration before trusting its scores.
  • A capacity-aware retention queue that ranks the holdout cohort by a clearly labeled 12-month value-at-risk proxy. You can export contact ranks and segment-specific actions to retention_priority_list.csv.
  • Secret Mission: Re-rank the campaign by expected net value under explicit offer-cost and save-rate assumptions.

Are there any prerequisites?

This project assumes intermediate or advanced Python data analysis and machine learning experience. You need a Google account to use Google Colab.

Before We Start

Before any analysis begins, this checkpoint locks in the limited-capacity contact decision your retention tool must support. Its 12-month revenue proxy estimates potential revenue at risk while true customer lifetime value also depends on future behavior, retention duration, costs, margins, and other lifetime-value assumptions.

Set Up a Reproducible Colab Runtime

The retention decision tool you committed to building needs the same software behavior every time you rerun it. A pinned Google Colab runtime removes local Windows installation drift.

Colab keeps this browser-based workflow at $0. You will configure the managed runtime with exact Python library versions.

In this step, get ready to:
  • Open a blank Colab notebook with the project runtime.
  • Install the pinned analysis libraries.
  • Verify the runtime with an environment report.
Configure the Colab runtime

Google Colab runs each notebook inside a managed runtime. The Runtime Version value 2026.07 supplies Python 3.12.13 for this project.

Your account sign-in stays separate from the notebook. This project never asks you to paste Google credentials into a code cell.

  • Go to Google Colab in your browser.
  • Sign in with your Google account.
  • Create a blank notebook from the Colab landing page.
  • Select Runtime in the notebook menu bar.
  • Choose Change runtime type.
  • Select 2026.07 from the Runtime Version field.
  • Set Hardware Accelerator to None.

The Runtime Version field now shows 2026.07. The Hardware Accelerator field now shows None.

  • Apply the settings using the dialog's confirmation button.

Why Colab instead of local Python?

A managed Colab runtime avoids differences between local Windows installations. Everyone starts with the same Python release.

Pinning a past runtime also makes future reruns easier to reproduce. The notebook keeps its execution environment explicit.

That is the first source of drift removed. Your notebook now uses the project's managed Python runtime.

Install the pinned libraries

Version pins prevent later package releases from changing the analysis unexpectedly. The notebook needs pandas 3.0.6, scikit-learn 1.9.1, and Matplotlib 3.11.2.

  • Install the three pinned libraries in the current kernel by running this command in the first code cell:
%pip install pandas==3.0.6 scikit-learn==1.9.1 matplotlib==3.11.2

What does this command do?

  • The %pip command installs packages into the notebook's current kernel.
  • Each == pin requests the exact package release used by this project.

The cell completes without a package installation error. Colab returns control to the notebook when every pinned package is ready.

Did the installation cell fail?

  • Confirm that the notebook still shows an active runtime connection.
  • Reconnect the runtime if the connection stopped.
  • Run the same installation cell again after the connection returns.

Ask for help with the installation output if the cell still fails: help me diagnose why my pinned Colab package installation failed

Verify the environment

An environment report turns the setup from an assumption into evidence. The show_versions() report confirms the Python environment behind scikit-learn.

  • Add a new code cell below the installation cell.
  • Paste this environment check into the new cell:
import pandas as pd
import matplotlib
from sklearn import show_versions

print(pd.__version__)
print(matplotlib.__version__)
show_versions()

What does this check prove?

  • The pd.__version__ expression prints the installed pandas release.
  • The matplotlib.__version__ expression prints the installed Matplotlib release.
  • The show_versions() call reports the Python and scikit-learn environment details.

Before you run this cell, pause to predict whether its first two lines will match the package pins.

  • Run the environment-check cell using its play button.

The first two printed lines show 3.0.6 and 3.11.2. The environment report identifies scikit-learn 1.9.1 plus Python 3.12.13 without an import error.

That is your reproducibility checkpoint passed. The notebook now has the exact environment used by the remaining analysis.

Seeing a different version?

  • Rerun the installation cell if any library version differs.
  • Reconnect the notebook runtime if the environment-check cell cannot import a library.
  • Run the installation cell again after reconnecting.

Use the environment report to get targeted help: help me fix my Colab environment version mismatch

Your reproducible notebook is ready. Next, you will turn raw customer records into visible business segments.

Create Customer Segments

Your verified Google Colab runtime is ready to turn raw customer records into a retention story. Individual rows hide the patterns that a retention team needs to see.

This step creates the first visible version of your decision tool. You will group customers into four segments before displaying their observed churn rates.

In this step, get ready to:
  • Load the IBM customer records into a cleaned dataset.
  • Assign each customer to a business segment.
  • Display the segment summary in a churn-rate chart.
Load and clean the customer records

A pandas DataFrame gives each customer attribute a named column. Cleaning those columns now keeps invalid charge values out of every segment calculation.

  • Create a new code cell below your environment check.
  • Paste the following code into the new cell.
  • Run the cell using its play button.
%%writefile retention_prioritizer.py
# STEP 2: Load, clean, segment, and visualize the customer base.
import pandas as pd
import matplotlib.pyplot as plt

DATA_URL = "https://raw.githubusercontent.com/IBM/telco-customer-churn-on-icp4d/master/data/Telco-Customer-Churn.csv"

df = pd.read_csv(DATA_URL)
df["TotalCharges"] = pd.to_numeric(df["TotalCharges"], errors="coerce")
df = df.dropna(subset=["TotalCharges"]).copy()
df["ChurnFlag"] = df["Churn"].eq("Yes").astype(int)

service_columns = [
    "OnlineSecurity",
    "OnlineBackup",
    "DeviceProtection",
    "TechSupport",
    "StreamingTV",
    "StreamingMovies",
]
df["ServiceCount"] = df[service_columns].eq("Yes").sum(axis=1)

What does this code do?

  • The `DATA_URL` constant keeps the IBM dataset location in one place.
  • The numeric conversion turns invalid `TotalCharges` values into missing values.
  • The cleanup removes rows that cannot support charge-based analysis.
  • The `ChurnFlag` column turns the churn outcome into a numeric target.
  • The `ServiceCount` column counts six adopted add-on services for each customer.
  • Confirm that retention_prioritizer.py appears beside the notebook when you select the folder icon in Colab's left sidebar.

Your script now has a clean customer table plus two decision-ready features. That foundation keeps the segment rules readable.

File missing from the notebook session?

Make sure the first line starts with %%writefile retention_prioritizer.py. Run the cell again after correcting the filename.

A dataset-loading failure can also come from a temporary network problem. Confirm that the Colab runtime still has an internet connection before retrying.

help me diagnose why my Colab cell did not create retention_prioritizer.py or load the IBM CSV

Create the business segments

Business segmentation turns customer traits into groups that a retention team can discuss. These ordered rules use tenure first before refining longer-tenure customers through commitment or service adoption.

  • Create a new code cell below the file-creation cell.
  • Paste the following append cell into the notebook.
  • Run the cell using its play button.
%%writefile -a retention_prioritizer.py

df["CustomerSegment"] = "Flexible Regular"
df.loc[df["tenure"] <= 12, "CustomerSegment"] = "New Explorer"
df.loc[
    (df["tenure"] > 12) & (df["Contract"] != "Month-to-month"),
    "CustomerSegment",
] = "Committed Core"
df.loc[
    (df["tenure"] > 12) & (df["ServiceCount"] >= 4),
    "CustomerSegment",
] = "Multi-Service Champion"

segment_summary = (
    df.groupby("CustomerSegment", as_index=False)
    .agg(
        customers=("customerID", "count"),
        observed_churn_rate=("ChurnFlag", "mean"),
        monthly_revenue=("MonthlyCharges", "sum"),
    )
    .sort_values("observed_churn_rate", ascending=False)
)
segment_summary["observed_churn_rate_pct"] = (
    segment_summary["observed_churn_rate"] * 100
).round(1)

How do the segment rules work?

Every customer starts as a Flexible Regular. Later rules replace that label when the customer meets a more specific condition.

  • Customers with at most 12 months of tenure become `New Explorer` customers.
  • Longer-tenure customers with a committed contract become `Committed Core` customers.
  • Longer-tenure customers with at least four adopted services become `Multi-Service Champion` customers.
  • The `segment_summary` table calculates customer count for each final segment.
  • The same table calculates observed churn rate plus monthly revenue.
  • Execute the current script by running this command in a new code cell:
%run retention_prioritizer.py

What does this command do?

The %run command executes the current script inside the notebook. This confirms that the cleaning plus segmentation rules work together.

  • Display the current segment summary by running this exact code in another code cell:
print("\nCustomer segment summary")
print(
    segment_summary[
        ["CustomerSegment", "customers", "observed_churn_rate_pct", "monthly_revenue"]
    ].to_string(index=False)
)

What does this check show?

This check prints the four segment labels with their customer counts. It also shows each segment's observed churn percentage plus monthly revenue.

You should see rows for New Explorer, Flexible Regular, Committed Core, plus Multi-Service Champion.

Missing one of the four segments?

Check that the append cell starts with %%writefile -a retention_prioritizer.py. Without the append option, the new cell can replace the earlier cleaning code.

Compare the tenure conditions plus the service-count condition with the code above. A changed comparison can move customers into the wrong segment.

help me find why my customer segment summary is missing a segment

Print and chart the segment story

A table preserves the exact segment metrics. A Matplotlib bar chart makes the differences in observed churn easier to compare.

  • Create one more code cell below the segment cells.
  • Paste the following append cell into the notebook.
  • Run the cell using its play button.
%%writefile -a retention_prioritizer.py

print("\nCustomer segment summary")
print(
    segment_summary[
        ["CustomerSegment", "customers", "observed_churn_rate_pct", "monthly_revenue"]
    ].to_string(index=False)
)

fig, ax = plt.subplots(figsize=(9, 4.5), layout="constrained")
ax.bar(
    segment_summary["CustomerSegment"],
    segment_summary["observed_churn_rate_pct"],
    color="#4C78A8",
)
ax.set_title("Observed churn rate by customer segment")
ax.set_xlabel("Customer segment")
ax.set_ylabel("Observed churn rate (%)")
ax.tick_params(axis="x", rotation=18)
plt.show()

How does the chart reveal the story?

  • The bar height represents each segment's observed churn percentage.
  • The descending summary order places the highest observed churn rate first.
  • The rotated labels keep all four segment names readable.
  • The chart title states that these rates describe observed outcomes.

Before you run the completed script, which customer segment do you expect to have the tallest churn-rate bar?

  • Run the completed script in a new code cell with this command:
%run retention_prioritizer.py

What does the final run prove?

This run executes the entire file from data loading through visualization. The table plus chart prove that the raw rows now support a visible segment comparison.

You should see a segment summary containing customer counts plus observed churn percentages. You should also see a bar chart titled Observed churn rate by customer segment.

You have created the first visible retention view in the project. Raw customer records now tell a segment-level churn story.

Chart not displaying?

Make sure the final line of retention_prioritizer.py is plt.show(). Run the completed script again after fixing that line.

If the table prints but the chart fails, compare every chart column name with segment_summary. The capitalization must match the file exactly.

help me troubleshoot why my segment summary prints but my Matplotlib chart does not display

✔️ Awesome, I've got everything!

Your script now loads the IBM data before cleaning it. It also creates four segments plus a visible churn comparison.

  • Keep the completed retention_prioritizer.py file in the current Colab runtime.

ⓧ I'd like to double check the full code

# STEP 2: Load, clean, segment, and visualize the customer base.
import pandas as pd
import matplotlib.pyplot as plt

DATA_URL = "https://raw.githubusercontent.com/IBM/telco-customer-churn-on-icp4d/master/data/Telco-Customer-Churn.csv"

df = pd.read_csv(DATA_URL)
df["TotalCharges"] = pd.to_numeric(df["TotalCharges"], errors="coerce")
df = df.dropna(subset=["TotalCharges"]).copy()
df["ChurnFlag"] = df["Churn"].eq("Yes").astype(int)

service_columns = [
    "OnlineSecurity",
    "OnlineBackup",
    "DeviceProtection",
    "TechSupport",
    "StreamingTV",
    "StreamingMovies",
]
df["ServiceCount"] = df[service_columns].eq("Yes").sum(axis=1)

df["CustomerSegment"] = "Flexible Regular"
df.loc[df["tenure"] <= 12, "CustomerSegment"] = "New Explorer"
df.loc[
    (df["tenure"] > 12) & (df["Contract"] != "Month-to-month"),
    "CustomerSegment",
] = "Committed Core"
df.loc[
    (df["tenure"] > 12) & (df["ServiceCount"] >= 4),
    "CustomerSegment",
] = "Multi-Service Champion"

segment_summary = (
    df.groupby("CustomerSegment", as_index=False)
    .agg(
        customers=("customerID", "count"),
        observed_churn_rate=("ChurnFlag", "mean"),
        monthly_revenue=("MonthlyCharges", "sum"),
    )
    .sort_values("observed_churn_rate", ascending=False)
)
segment_summary["observed_churn_rate_pct"] = (
    segment_summary["observed_churn_rate"] * 100
).round(1)

print("\nCustomer segment summary")
print(
    segment_summary[
        ["CustomerSegment", "customers", "observed_churn_rate_pct", "monthly_revenue"]
    ].to_string(index=False)
)

fig, ax = plt.subplots(figsize=(9, 4.5), layout="constrained")
ax.bar(
    segment_summary["CustomerSegment"],
    segment_summary["observed_churn_rate_pct"],
    color="#4C78A8",
)
ax.set_title("Observed churn rate by customer segment")
ax.set_xlabel("Customer segment")
ax.set_ylabel("Observed churn rate (%)")
ax.tick_params(axis="x", rotation=18)
plt.show()

Your segment view shows where observed churn differs across the customer base. Next up, you will build a probability model before exposing what its default accuracy hides.

Evaluate the Churn Model

Your Google Colab notebook already turns raw customer records into visible segments. The retention decision now needs customer-level probabilities from a scikit-learn model.

A useful accuracy score can still hide genuine churners. This step tests the default 0.50 cutoff on an untouched holdout cohort so those missed cases become visible.

In this step, get ready to:
  • Prepare reproducible training and holdout cohorts.
  • Train a leakage-safe mixed-type probability model.
  • Evaluate missed churners, probability quality, model drivers, and calibration.
Build the leakage-safe probability model

A Pipeline fits preprocessing rules on the training cohort. The untouched holdout cohort provides an honest evaluation surface.

Why this approach?

The ColumnTransformer keeps numeric scaling separate from categorical encoding. The pipeline learns both transformations from training data only.

Logistic regression produces probabilities with coefficients you can inspect. That combination supports evaluation plus business explanation.

  • Add a new code cell beneath the cell that last ran retention_prioritizer.py.
  • Append the feature preparation code by running this cell:
%%writefile -a retention_prioritizer.py

# STEP 3: Build and evaluate a leakage-safe probability model.
from sklearn.compose import ColumnTransformer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, average_precision_score, brier_score_loss, confusion_matrix
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

numeric_features = ["SeniorCitizen", "tenure", "MonthlyCharges", "TotalCharges", "ServiceCount"]
categorical_features = ["gender", "Partner", "Dependents", "PhoneService", "MultipleLines", "InternetService", "OnlineSecurity", "OnlineBackup", "DeviceProtection", "TechSupport", "StreamingTV", "StreamingMovies", "Contract", "PaperlessBilling", "PaymentMethod"]
feature_columns = numeric_features + categorical_features

X = df[feature_columns]
y = df["ChurnFlag"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=42, stratify=y)

What Does This Code Do?

  • The feature lists separate numeric values from categorical labels.
  • The train_test_split call reserves 20 percent of the cleaned data for evaluation.
  • The stratify=y setting preserves the churn-class balance in both cohorts.
  • The fixed random_state=42 value makes the split reproducible.
  • Confirm that Colab reports the file was updated.
  • Add another code cell beneath the feature preparation cell.
  • Append the preprocessing pipeline by running this cell:
%%writefile -a retention_prioritizer.py

preprocessor = ColumnTransformer(transformers=[("numeric", StandardScaler(), numeric_features), ("categorical", OneHotEncoder(handle_unknown="ignore"), categorical_features)])
model = Pipeline(steps=[("preprocessor", preprocessor), ("classifier", LogisticRegression(l1_ratio=0.0, max_iter=1000))])
model.fit(X_train, y_train)

How Does the Pipeline Work?

  • The StandardScaler learns numeric scaling values from X_train.
  • The OneHotEncoder converts categorical labels into model-ready columns.
  • The handle_unknown="ignore" setting allows unseen holdout categories to pass through the encoder.
  • The LogisticRegression estimator learns churn relationships from the prepared training data.
  • Confirm that Colab reports another update to retention_prioritizer.py.
  • Fit the model inside the complete script by running this cell:
%run retention_prioritizer.py

What Does This Run Prove?

The command reloads the data before rebuilding the segments. It then fits the model on the reproducible training cohort.

You will see the segment summary plus its chart again. The cell completes without an import or training error.

Model Training Stops?

Confirm that both append cells ran before the script cell. A skipped cell can leave the feature lists or pipeline undefined.

Check the capitalization of MonthlyCharges plus TotalCharges against the existing dataset columns.

Use the failing line to get targeted help: help me debug the model training pipeline

Expose the accuracy trap

The default 0.50 cutoff converts probabilities into binary predictions. A confusion matrix separates correct predictions from each mistake type.

What Do the Probability Metrics Measure?

Average precision summarizes how effectively higher scores rank churners near the top. It evaluates the ordering created by the probability model.

Brier score loss measures the squared gap between predicted probabilities and observed outcomes. A smaller loss indicates better probability quality.

  • Add a new code cell beneath the model-training cell.
  • Append the default-threshold evaluation by running this cell:
%%writefile -a retention_prioritizer.py

test_probability = model.predict_proba(X_test)[:, 1]
default_prediction = (test_probability >= 0.50).astype(int)
baseline_accuracy = max(y_test.mean(), 1 - y_test.mean())
model_accuracy = accuracy_score(y_test, default_prediction)
confusion = confusion_matrix(y_test, default_prediction, labels=[0, 1])
tn, fp, fn, tp = confusion.ravel().tolist()
average_precision = average_precision_score(y_test, test_probability)
brier_loss = brier_score_loss(y_test, test_probability)

print("\nDefault-threshold evaluation")
print(f"Majority baseline accuracy: {baseline_accuracy:.3f}")
print(f"Model accuracy at 0.50: {model_accuracy:.3f}")
print("Confusion matrix [[TN, FP], [FN, TP]]:")
print(confusion)
print(f"False negatives at 0.50: {fn}")
print(f"Average precision: {average_precision:.3f}")
print(f"Brier score loss: {brier_loss:.3f}")

How Does the Evaluation Work?

  • The test_probability array stores one churn probability for each holdout customer.
  • The default_prediction array applies the default cutoff.
  • The baseline_accuracy value represents an always-predict-the-majority strategy.
  • The fn value counts actual churners predicted as non-churners.

Before you run the evaluation, consider whether useful overall accuracy can coexist with missed churners.

  • Reveal the default cutoff's holdout performance by running this cell:
%run retention_prioritizer.py

What Does This Run Check?

The script retrains the pipeline on the same split. It then evaluates the untouched holdout cohort with the default cutoff.

You will see the majority baseline beside the model accuracy. You will also see a nonzero false-negative count in the confusion-matrix output.

That result exposes the intended accuracy trap. A useful aggregate score can still miss customers who actually churn.

Evaluation Output Missing?

Confirm that the evaluation cell updated retention_prioritizer.py before you ran the script.

Check that test_probability >= 0.50 matches the evaluation code. A changed comparison produces different predictions.

Ask for help with the missing output: help me troubleshoot the holdout evaluation

The fitted model can also show which transformed features push churn predictions upward or downward. Logistic regression exposes those directions through its coefficients.

  • Add a new code cell beneath the evaluation cell.
  • Append the coefficient-driver table by running this cell:
%%writefile -a retention_prioritizer.py

feature_names = model.named_steps["preprocessor"].get_feature_names_out()
coefficients = model.named_steps["classifier"].coef_[0]
drivers = pd.DataFrame({"Feature": feature_names, "Coefficient": coefficients}).sort_values("Coefficient", ascending=False)

print("\nStrongest positive churn drivers")
print(drivers.head(8).to_string(index=False))
print("\nStrongest negative churn drivers")
print(drivers.tail(8).sort_values("Coefficient").to_string(index=False))

What Do the Coefficients Show?

  • The feature_names array recovers names from the fitted preprocessor.
  • The coefficients array contains the fitted logistic-regression weights.
  • Positive coefficients push estimated churn tendency upward.
  • Negative coefficients push estimated churn tendency downward.
  • Print the strongest coefficient drivers by running this cell:
%run retention_prioritizer.py

What Does This Run Add?

The script prints the strongest positive churn drivers. It also prints the strongest negative churn drivers.

These coefficients describe patterns learned from this dataset. They do not establish that a feature causes churn.

You will see two driver tables beneath the holdout metrics. Each row pairs a transformed feature with its learned coefficient.

Driver Table Fails to Print?

Confirm that model.fit(X_train, y_train) executes before the script reads the fitted coefficients.

Check that the pipeline step names remain preprocessor plus classifier.

Get targeted help with the fitted pipeline: help me debug the coefficient-driver table

Inspect probability calibration

Probability calibration compares mean predictions with observed churn rates. The diagonal reference line shows where those two values match.

  • Add a new code cell beneath the driver-table cell.
  • Append the calibration summary by running this cell:
%%writefile -a retention_prioritizer.py

calibration = pd.DataFrame({"predicted_probability": test_probability, "actual_churn": y_test.to_numpy()})
calibration["probability_band"] = pd.cut(calibration["predicted_probability"], bins=[0.0, 0.2, 0.4, 0.6, 0.8, 1.0], include_lowest=True)
calibration_summary = calibration.groupby("probability_band", observed=True).agg(predicted_probability=("predicted_probability", "mean"), observed_churn_rate=("actual_churn", "mean"), customers=("actual_churn", "size")).reset_index()

print("\nCalibration by probability band")
print(calibration_summary.to_string(index=False))

How Are the Probability Bands Built?

  • The calibration table pairs each holdout probability with the actual outcome.
  • The pd.cut call divides probabilities into five bands.
  • The calibration_summary table compares the mean prediction with the observed churn rate inside each populated band.
  • The customers column shows how many holdout records support each comparison.
  • Print the probability-band summary by running this cell:
%run retention_prioritizer.py

What Does This Run Add?

The script prints one row for each populated probability band. Every row places the mean predicted probability beside the observed churn rate.

You will see the calibration summary beneath the coefficient tables. The customer count shows how much holdout evidence supports each band.

  • Add a new code cell beneath the calibration-summary cell.
  • Append the calibration chart by running this cell:
%%writefile -a retention_prioritizer.py

fig, ax = plt.subplots(figsize=(6, 5), layout="constrained")
ax.plot([0, 1], [0, 1], linestyle="--", color="gray", label="Perfect calibration")
ax.scatter(calibration_summary["predicted_probability"], calibration_summary["observed_churn_rate"], s=70, color="#F58518", label="Model bands")
ax.set_title("Predicted probability vs observed churn")
ax.set_xlabel("Mean predicted churn probability")
ax.set_ylabel("Observed churn rate")
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.legend()
plt.show()

What Does the Chart Show?

Each point represents one populated probability band. Its position compares the band's mean prediction with its observed churn rate.

The dashed diagonal represents perfect agreement. Distance from that line makes probability mismatch visible.

Before the final run, consider how closely the model-band points will follow the diagonal reference line.

  • Run the complete evaluation by executing this cell:
%run retention_prioritizer.py

What Should You See?

The output prints baseline accuracy plus model accuracy. It also prints a 2 by 2 confusion matrix with a nonzero false-negative count.

You will see average precision plus Brier score loss. The output continues with coefficient drivers plus the calibration summary.

The final chart is titled Predicted probability vs observed churn.

That is the accuracy trap exposed. Your model now has an honest holdout evaluation that covers classification mistakes plus probability quality.

Calibration Chart Missing?

Confirm that the calibration-summary cell ran before the chart cell. The chart depends on calibration_summary.

Check that both calibration append cells updated retention_prioritizer.py before the final run.

Ask for help with the chart section: help me debug the calibration chart

Use these tabs to confirm that your cumulative script matches the completed evaluation workflow.

✔️ Awesome, I've got everything!

Great work. Confirm that every append cell has run before you continue.

ⓧ I'd like to double check the full code

# STEP 2: Load, clean, segment, and visualize the customer base.
import pandas as pd
import matplotlib.pyplot as plt

DATA_URL = "https://raw.githubusercontent.com/IBM/telco-customer-churn-on-icp4d/master/data/Telco-Customer-Churn.csv"

df = pd.read_csv(DATA_URL)
df["TotalCharges"] = pd.to_numeric(df["TotalCharges"], errors="coerce")
df = df.dropna(subset=["TotalCharges"]).copy()
df["ChurnFlag"] = df["Churn"].eq("Yes").astype(int)

service_columns = [
    "OnlineSecurity",
    "OnlineBackup",
    "DeviceProtection",
    "TechSupport",
    "StreamingTV",
    "StreamingMovies",
]
df["ServiceCount"] = df[service_columns].eq("Yes").sum(axis=1)

df["CustomerSegment"] = "Flexible Regular"
df.loc[df["tenure"] <= 12, "CustomerSegment"] = "New Explorer"
df.loc[(df["tenure"] > 12) & (df["Contract"] != "Month-to-month"), "CustomerSegment"] = "Committed Core"
df.loc[(df["tenure"] > 12) & (df["ServiceCount"] >= 4), "CustomerSegment"] = "Multi-Service Champion"

segment_summary = df.groupby("CustomerSegment", as_index=False).agg(customers=("customerID", "count"), observed_churn_rate=("ChurnFlag", "mean"), monthly_revenue=("MonthlyCharges", "sum")).sort_values("observed_churn_rate", ascending=False)
segment_summary["observed_churn_rate_pct"] = (segment_summary["observed_churn_rate"] * 100).round(1)

print("\nCustomer segment summary")
print(segment_summary[["CustomerSegment", "customers", "observed_churn_rate_pct", "monthly_revenue"]].to_string(index=False))

fig, ax = plt.subplots(figsize=(9, 4.5), layout="constrained")
ax.bar(segment_summary["CustomerSegment"], segment_summary["observed_churn_rate_pct"], color="#4C78A8")
ax.set_title("Observed churn rate by customer segment")
ax.set_xlabel("Customer segment")
ax.set_ylabel("Observed churn rate (%)")
ax.tick_params(axis="x", rotation=18)
plt.show()

# STEP 3: Build and evaluate a leakage-safe probability model.
from sklearn.compose import ColumnTransformer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, average_precision_score, brier_score_loss, confusion_matrix
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

numeric_features = ["SeniorCitizen", "tenure", "MonthlyCharges", "TotalCharges", "ServiceCount"]
categorical_features = ["gender", "Partner", "Dependents", "PhoneService", "MultipleLines", "InternetService", "OnlineSecurity", "OnlineBackup", "DeviceProtection", "TechSupport", "StreamingTV", "StreamingMovies", "Contract", "PaperlessBilling", "PaymentMethod"]
feature_columns = numeric_features + categorical_features

X = df[feature_columns]
y = df["ChurnFlag"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=42, stratify=y)

preprocessor = ColumnTransformer(transformers=[("numeric", StandardScaler(), numeric_features), ("categorical", OneHotEncoder(handle_unknown="ignore"), categorical_features)])
model = Pipeline(steps=[("preprocessor", preprocessor), ("classifier", LogisticRegression(l1_ratio=0.0, max_iter=1000))])
model.fit(X_train, y_train)

test_probability = model.predict_proba(X_test)[:, 1]
default_prediction = (test_probability >= 0.50).astype(int)
baseline_accuracy = max(y_test.mean(), 1 - y_test.mean())
model_accuracy = accuracy_score(y_test, default_prediction)
confusion = confusion_matrix(y_test, default_prediction, labels=[0, 1])
tn, fp, fn, tp = confusion.ravel().tolist()
average_precision = average_precision_score(y_test, test_probability)
brier_loss = brier_score_loss(y_test, test_probability)

print("\nDefault-threshold evaluation")
print(f"Majority baseline accuracy: {baseline_accuracy:.3f}")
print(f"Model accuracy at 0.50: {model_accuracy:.3f}")
print("Confusion matrix [[TN, FP], [FN, TP]]:")
print(confusion)
print(f"False negatives at 0.50: {fn}")
print(f"Average precision: {average_precision:.3f}")
print(f"Brier score loss: {brier_loss:.3f}")

feature_names = model.named_steps["preprocessor"].get_feature_names_out()
coefficients = model.named_steps["classifier"].coef_[0]
drivers = pd.DataFrame({"Feature": feature_names, "Coefficient": coefficients}).sort_values("Coefficient", ascending=False)

print("\nStrongest positive churn drivers")
print(drivers.head(8).to_string(index=False))
print("\nStrongest negative churn drivers")
print(drivers.tail(8).sort_values("Coefficient").to_string(index=False))

calibration = pd.DataFrame({"predicted_probability": test_probability, "actual_churn": y_test.to_numpy()})
calibration["probability_band"] = pd.cut(calibration["predicted_probability"], bins=[0.0, 0.2, 0.4, 0.6, 0.8, 1.0], include_lowest=True)
calibration_summary = calibration.groupby("probability_band", observed=True).agg(predicted_probability=("predicted_probability", "mean"), observed_churn_rate=("actual_churn", "mean"), customers=("actual_churn", "size")).reset_index()

print("\nCalibration by probability band")
print(calibration_summary.to_string(index=False))

fig, ax = plt.subplots(figsize=(6, 5), layout="constrained")
ax.plot([0, 1], [0, 1], linestyle="--", color="gray", label="Perfect calibration")
ax.scatter(calibration_summary["predicted_probability"], calibration_summary["observed_churn_rate"], s=70, color="#F58518", label="Model bands")
ax.set_title("Predicted probability vs observed churn")
ax.set_xlabel("Mean predicted churn probability")
ax.set_ylabel("Observed churn rate")
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.legend()
plt.show()

How to Use This Reference

This block shows the complete cumulative file for comparison. The step comments separate the segmentation workflow from the holdout evaluation.

Your holdout evaluation now exposes missed churners plus probability quality. Next, you will replace the binary cutoff with a capacity-aware value-at-risk queue.

Rank Customers by Value at Risk

Your probability model now identifies churn risk across the untouched holdout cohort. Those probabilities still leave the retention team without a practical order for outreach.

Inside your Google Colab notebook, this step converts each probability into a value-at-risk queue. A 10 percent capacity rule turns the ranking into a concrete contact plan.

In this step, get ready to:
  • Build an honest scoring table from the untouched holdout cohort.
  • Calculate a 12-month revenue proxy for every holdout customer.
  • Create a capacity-aware queue with segment-specific retention actions.
Build the holdout scoring table

The holdout cohort simulates customers the model has not seen during training. Scoring this cohort keeps the decision queue separate from the data used to fit the model.

  • Create a new code cell directly below the cell from the previous step.
  • Append the holdout scoring logic to retention_prioritizer.py by pasting the cell below:
%%writefile -a retention_prioritizer.py

# STEP 4: Convert holdout probabilities into a capacity-aware value-at-risk queue.
score_columns = [
    "customerID",
    "CustomerSegment",
    "Contract",
    "tenure",
    "MonthlyCharges",
    "ServiceCount",
]
scored = df.loc[X_test.index, score_columns].copy()
scored["ActualChurn"] = y_test.to_numpy()
scored["ChurnProbability"] = test_probability
scored["AnnualRevenueProxy"] = scored["MonthlyCharges"] * 12
scored["ValueAtRisk12M"] = scored["ChurnProbability"] * scored["AnnualRevenueProxy"]
scored = scored.sort_values("ValueAtRisk12M", ascending=False).reset_index(drop=True)
scored["ContactPriority"] = scored.index + 1

How does the scoring table work?

  • The X_test.index values select customers from the untouched holdout cohort.
  • The ChurnProbability column stores the probability already calculated for each holdout customer.
  • The AnnualRevenueProxy column projects current monthly charges across 12 months.
  • The ValueAtRisk12M column weights that revenue proxy by predicted churn probability.

Both measures are decision proxies. Customer lifetime value needs validated long-term economics beyond this dataset.

  • Run the cell to append the scoring logic.
  • Confirm that the notebook reports an update to retention_prioritizer.py.

Scoring code not appended?

Check that %%writefile -a retention_prioritizer.py is the first line of the cell. The append option preserves the segmentation and model code already in the file.

Make sure the cell finishes without reporting a syntax problem.

Help me check why my scoring code was not appended to retention_prioritizer.py.

Turn the ranking into an outreach queue

A retention team can contact only a limited number of customers. The scenario selects the highest-ranked 10 percent of the holdout cohort.

Each selected customer also receives an action based on their segment. Everyone outside the capacity limit remains under monitoring.

  • Create another code cell below the scoring cell.
  • Append the capacity rules and retention actions by pasting the cell below:
%%writefile -a retention_prioritizer.py

CAMPAIGN_CAPACITY_RATE = 0.10
campaign_capacity = max(1, int(len(scored) * CAMPAIGN_CAPACITY_RATE))
scored["ContactFlag"] = (scored["ContactPriority"] <= campaign_capacity).astype(int)
scored["RetentionAction"] = "Monitor"

action_map = {
    "New Explorer": "Onboarding check-in",
    "Flexible Regular": "Contract review",
    "Committed Core": "Loyalty benefit review",
    "Multi-Service Champion": "Priority service review",
}
contact_mask = scored["ContactFlag"].eq(1)
scored.loc[contact_mask, "RetentionAction"] = scored.loc[contact_mask, "CustomerSegment"].map(action_map)

print(f"\nCampaign capacity: {campaign_capacity} holdout customers")
print("Top ten customers by 12-month value at risk")
print(scored[["ContactPriority", "customerID", "CustomerSegment", "ChurnProbability", "AnnualRevenueProxy", "ValueAtRisk12M", "RetentionAction"]].head(10).to_string(index=False))

How does capacity shape the queue?

  • The CAMPAIGN_CAPACITY_RATE value limits outreach to 0.10 of the holdout cohort.
  • The campaign_capacity calculation guarantees at least one contact.
  • The ContactFlag column separates selected customers from monitored customers.
  • The action_map value connects each selected segment to a specific retention response.
  • Run the cell to append the capacity-aware queue logic.
  • Confirm that the notebook reports another update to retention_prioritizer.py.

Queue logic not appended?

Confirm that the new cell starts with %%writefile -a retention_prioritizer.py. A missing append option can replace the existing script.

Check that the action_map keys match the four values stored in CustomerSegment.

Help me troubleshoot the capacity and retention action code.

✔️ Awesome, I've got everything!

Your script now contains the scoring logic and capacity-aware retention rules. Make sure both append cells have finished running.

ⓧ I'd like to double check the full code

# STEP 2: Load, clean, segment, and visualize the customer base.
import pandas as pd
import matplotlib.pyplot as plt

DATA_URL = "https://raw.githubusercontent.com/IBM/telco-customer-churn-on-icp4d/master/data/Telco-Customer-Churn.csv"

df = pd.read_csv(DATA_URL)
df["TotalCharges"] = pd.to_numeric(df["TotalCharges"], errors="coerce")
df = df.dropna(subset=["TotalCharges"]).copy()
df["ChurnFlag"] = df["Churn"].eq("Yes").astype(int)

service_columns = [
    "OnlineSecurity",
    "OnlineBackup",
    "DeviceProtection",
    "TechSupport",
    "StreamingTV",
    "StreamingMovies",
]
df["ServiceCount"] = df[service_columns].eq("Yes").sum(axis=1)

df["CustomerSegment"] = "Flexible Regular"
df.loc[df["tenure"] <= 12, "CustomerSegment"] = "New Explorer"
df.loc[(df["tenure"] > 12) & (df["Contract"] != "Month-to-month"), "CustomerSegment"] = "Committed Core"
df.loc[(df["tenure"] > 12) & (df["ServiceCount"] >= 4), "CustomerSegment"] = "Multi-Service Champion"

segment_summary = df.groupby("CustomerSegment", as_index=False).agg(customers=("customerID", "count"), observed_churn_rate=("ChurnFlag", "mean"), monthly_revenue=("MonthlyCharges", "sum")).sort_values("observed_churn_rate", ascending=False)
segment_summary["observed_churn_rate_pct"] = (segment_summary["observed_churn_rate"] * 100).round(1)

print("\nCustomer segment summary")
print(segment_summary[["CustomerSegment", "customers", "observed_churn_rate_pct", "monthly_revenue"]].to_string(index=False))

fig, ax = plt.subplots(figsize=(9, 4.5), layout="constrained")
ax.bar(segment_summary["CustomerSegment"], segment_summary["observed_churn_rate_pct"], color="#4C78A8")
ax.set_title("Observed churn rate by customer segment")
ax.set_xlabel("Customer segment")
ax.set_ylabel("Observed churn rate (%)")
ax.tick_params(axis="x", rotation=18)
plt.show()

# STEP 3: Build and evaluate a leakage-safe probability model.
from sklearn.compose import ColumnTransformer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, average_precision_score, brier_score_loss, confusion_matrix
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

numeric_features = ["SeniorCitizen", "tenure", "MonthlyCharges", "TotalCharges", "ServiceCount"]
categorical_features = ["gender", "Partner", "Dependents", "PhoneService", "MultipleLines", "InternetService", "OnlineSecurity", "OnlineBackup", "DeviceProtection", "TechSupport", "StreamingTV", "StreamingMovies", "Contract", "PaperlessBilling", "PaymentMethod"]
feature_columns = numeric_features + categorical_features

X = df[feature_columns]
y = df["ChurnFlag"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=42, stratify=y)

preprocessor = ColumnTransformer(transformers=[("numeric", StandardScaler(), numeric_features), ("categorical", OneHotEncoder(handle_unknown="ignore"), categorical_features)])
model = Pipeline(steps=[("preprocessor", preprocessor), ("classifier", LogisticRegression(l1_ratio=0.0, max_iter=1000))])
model.fit(X_train, y_train)

test_probability = model.predict_proba(X_test)[:, 1]
default_prediction = (test_probability >= 0.50).astype(int)
baseline_accuracy = max(y_test.mean(), 1 - y_test.mean())
model_accuracy = accuracy_score(y_test, default_prediction)
confusion = confusion_matrix(y_test, default_prediction, labels=[0, 1])
tn, fp, fn, tp = confusion.ravel().tolist()
average_precision = average_precision_score(y_test, test_probability)
brier_loss = brier_score_loss(y_test, test_probability)

print("\nDefault-threshold evaluation")
print(f"Majority baseline accuracy: {baseline_accuracy:.3f}")
print(f"Model accuracy at 0.50: {model_accuracy:.3f}")
print("Confusion matrix [[TN, FP], [FN, TP]]:")
print(confusion)
print(f"False negatives at 0.50: {fn}")
print(f"Average precision: {average_precision:.3f}")
print(f"Brier score loss: {brier_loss:.3f}")

feature_names = model.named_steps["preprocessor"].get_feature_names_out()
coefficients = model.named_steps["classifier"].coef_[0]
drivers = pd.DataFrame({"Feature": feature_names, "Coefficient": coefficients}).sort_values("Coefficient", ascending=False)

print("\nStrongest positive churn drivers")
print(drivers.head(8).to_string(index=False))
print("\nStrongest negative churn drivers")
print(drivers.tail(8).sort_values("Coefficient").to_string(index=False))

calibration = pd.DataFrame({"predicted_probability": test_probability, "actual_churn": y_test.to_numpy()})
calibration["probability_band"] = pd.cut(calibration["predicted_probability"], bins=[0.0, 0.2, 0.4, 0.6, 0.8, 1.0], include_lowest=True)
calibration_summary = calibration.groupby("probability_band", observed=True).agg(predicted_probability=("predicted_probability", "mean"), observed_churn_rate=("actual_churn", "mean"), customers=("actual_churn", "size")).reset_index()

print("\nCalibration by probability band")
print(calibration_summary.to_string(index=False))

fig, ax = plt.subplots(figsize=(6, 5), layout="constrained")
ax.plot([0, 1], [0, 1], linestyle="--", color="gray", label="Perfect calibration")
ax.scatter(calibration_summary["predicted_probability"], calibration_summary["observed_churn_rate"], s=70, color="#F58518", label="Model bands")
ax.set_title("Predicted probability vs observed churn")
ax.set_xlabel("Mean predicted churn probability")
ax.set_ylabel("Observed churn rate")
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.legend()
plt.show()

# STEP 4: Convert holdout probabilities into a capacity-aware value-at-risk queue.
score_columns = [
    "customerID",
    "CustomerSegment",
    "Contract",
    "tenure",
    "MonthlyCharges",
    "ServiceCount",
]
scored = df.loc[X_test.index, score_columns].copy()
scored["ActualChurn"] = y_test.to_numpy()
scored["ChurnProbability"] = test_probability
scored["AnnualRevenueProxy"] = scored["MonthlyCharges"] * 12
scored["ValueAtRisk12M"] = scored["ChurnProbability"] * scored["AnnualRevenueProxy"]
scored = scored.sort_values("ValueAtRisk12M", ascending=False).reset_index(drop=True)
scored["ContactPriority"] = scored.index + 1

CAMPAIGN_CAPACITY_RATE = 0.10
campaign_capacity = max(1, int(len(scored) * CAMPAIGN_CAPACITY_RATE))
scored["ContactFlag"] = (scored["ContactPriority"] <= campaign_capacity).astype(int)
scored["RetentionAction"] = "Monitor"

action_map = {
    "New Explorer": "Onboarding check-in",
    "Flexible Regular": "Contract review",
    "Committed Core": "Loyalty benefit review",
    "Multi-Service Champion": "Priority service review",
}
contact_mask = scored["ContactFlag"].eq(1)
scored.loc[contact_mask, "RetentionAction"] = scored.loc[contact_mask, "CustomerSegment"].map(action_map)

print(f"\nCampaign capacity: {campaign_capacity} holdout customers")
print("Top ten customers by 12-month value at risk")
print(scored[["ContactPriority", "customerID", "CustomerSegment", "ChurnProbability", "AnnualRevenueProxy", "ValueAtRisk12M", "RetentionAction"]].head(10).to_string(index=False))
Run the ranked retention queue

The completed script now rebuilds the customer segments before retraining the model. It finishes by ranking the untouched holdout cohort for outreach.

Before you run it, which customers do you expect to rise when churn probability is weighted by the 12-month revenue proxy?

  • Run the complete retention workflow in a new code cell by executing this command:
%run retention_prioritizer.py

What does this command do?

The command executes retention_prioritizer.py from top to bottom. It recreates the segments before fitting the model and ranking the holdout customers.

You will see the campaign capacity followed by a top-ten table ordered by ValueAtRisk12M. Each displayed customer has a segment-specific RetentionAction.

That decision layer is working. Your model now produces a capacity-aware outreach queue from its holdout probabilities.

Queue missing or out of order?

Confirm that both append cells ran before the %run retention_prioritizer.py cell. Each append cell must keep %%writefile -a retention_prioritizer.py on its first line.

Check that scored sorts by ValueAtRisk12M before ContactPriority is assigned.

Help me debug my value-at-risk queue.

Your holdout probabilities now drive a concrete retention queue. Next up, you will visualize the decision and export it for another team to review.

Export the Retention Brief

Your capacity-aware queue now ranks the untouched holdout cohort by 12-month value at risk. Each selected customer also has a retention action tied to their segment.

A ranked table is difficult to review across a team. This step turns the queue into an executive chart plus a CSV file that another team can use.

In this step, get ready to:
  • Summarize the retention decision by customer segment.
  • Plot the holdout queue by contact status.
  • Export every ranked holdout customer to a decision-ready CSV file.
Summarize the decision by segment

The priority_summary table compresses the customer-level queue into one row per segment. It shows where the campaign capacity sits across the holdout cohort.

  • Add a new code cell beneath the cell you used for Step 4.
  • Append the segment summary to retention_prioritizer.py by running this cell:
%%writefile -a retention_prioritizer.py
# STEP 5: Build the executive view and export the decision artifact.
priority_summary = (
    scored.groupby("CustomerSegment", as_index=False)
    .agg(
        holdout_customers=("customerID", "count"),
        contacts=("ContactFlag", "sum"),
        average_churn_probability=("ChurnProbability", "mean"),
        total_value_at_risk=("ValueAtRisk12M", "sum"),
    )
    .sort_values("total_value_at_risk", ascending=False)
)

print("\nRetention decision summary")
print(priority_summary.to_string(index=False))

How Does the Segment Brief Work?

  • The grouped summary creates one executive-level row for each customer segment.
  • The named aggregations show cohort size, selected contacts, average churn probability, and total value at risk.
  • The final sort places the segment with the highest total value at risk first.

The appended code depends on the segmentation, model, and queue created earlier. Running the complete script tests that entire chain.

  • Run the complete script in a new code cell with:
%run retention_prioritizer.py

What Does This Command Do?

The command executes every saved section of retention_prioritizer.py in order. The new segment summary therefore uses the same holdout queue you already evaluated.

You'll see a Retention decision summary table with one row per customer segment. The table includes holdout customer counts, contact counts, average churn probabilities, and total value at risk.

Missing the Segment Summary?

Confirm that the append cell completed before you ran the full script. Check that the priority_summary block appears after the Step 4 code in retention_prioritizer.py.

If the script stops before the summary, inspect the final Step 4 output to find the first failing section.

Help me diagnose why priority_summary is missing when I run my retention prioritizer script.

Visualize the contact decision

A decision chart makes the campaign boundary visible. Matplotlib places churn probability on the horizontal axis and the 12-month revenue proxy on the vertical axis.

  • Add another code cell beneath the segment summary cell.
  • Append the contact-versus-monitor chart to retention_prioritizer.py by running this cell:
%%writefile -a retention_prioritizer.py
contacted = scored[scored["ContactFlag"].eq(1)]
monitored = scored[scored["ContactFlag"].eq(0)]
fig, ax = plt.subplots(figsize=(8, 5), layout="constrained")
ax.scatter(
    monitored["ChurnProbability"],
    monitored["AnnualRevenueProxy"],
    alpha=0.35,
    color="#9D9D9D",
    label="Monitor",
)
ax.scatter(
    contacted["ChurnProbability"],
    contacted["AnnualRevenueProxy"],
    alpha=0.8,
    color="#E45756",
    label="Contact",
)
ax.set_title("Retention queue: probability, value, and capacity")
ax.set_xlabel("Predicted churn probability")
ax.set_ylabel("12-month revenue proxy")
ax.legend()
plt.show()

What Does This Chart Show?

  • The contacted group contains customers inside the campaign capacity.
  • The monitored group contains the remaining holdout customers.
  • The two scatter layers reveal how probability, revenue proxy, and limited capacity shape the contact decision.
  • Run the updated script in a new code cell with:
%run retention_prioritizer.py

Why Run the Whole Script Again?

The chart reads the ranked scored table created earlier in the file. A full run rebuilds that table before drawing the new executive view.

That is a useful visual checkpoint. You can now see contacted customers highlighted against the monitored holdout cohort on the chart titled Retention queue: probability, value, and capacity.

Chart Not Displaying?

Confirm that both scatter calls appear before plt.show(). Also confirm that Step 4 creates ContactFlag before this chart runs.

Help me fix a missing retention queue scatter plot in my Colab notebook.

Export the ranked retention list

The chart supports review. The exported file carries the customer-level decision into an operational workflow.

  • Add one more code cell beneath the chart cell.
  • Append the export logic to retention_prioritizer.py by running this cell:
%%writefile -a retention_prioritizer.py
export_columns = [
    "ContactPriority",
    "customerID",
    "CustomerSegment",
    "Contract",
    "tenure",
    "MonthlyCharges",
    "ServiceCount",
    "ChurnProbability",
    "AnnualRevenueProxy",
    "ValueAtRisk12M",
    "ContactFlag",
    "RetentionAction",
]
OUTPUT_FILE = "retention_priority_list.csv"
scored[export_columns].to_csv(OUTPUT_FILE, index=False)
print(f"\nCreated {OUTPUT_FILE} with {len(scored)} ranked rows.")
print(scored[export_columns].head().to_string(index=False))

What Makes This File Operational?

  • The export keeps one ranked row for every customer in the untouched holdout cohort.
  • The decision columns preserve contact priority, contact status, and the segment-specific retention action.
  • The probability plus revenue-proxy columns make the ranking traceable during review.
  • The index=False setting keeps the DataFrame row index out of the finished CSV.

Before you run the final check, what do you expect the script to create after it displays the decision chart?

  • Run the complete retention prioritizer in a new code cell with:
%run retention_prioritizer.py

What Does the Final Run Prove?

This run rebuilds the segments, evaluates the model, ranks the holdout cohort, and exports the final decision artifact. One successful run proves the workflow is reproducible from the raw dataset.

You'll see Created retention_priority_list.csv with the selected row count. The next lines show the first ranked customers with contact priorities and retention actions.

  • Confirm that the printed rows include ContactPriority, ContactFlag, and RetentionAction.

CSV Not Created?

Confirm that the export cell appended to retention_prioritizer.py successfully. The OUTPUT_FILE assignment must appear before the CSV export call.

If the script stops earlier, fix the first reported issue before checking the transient runtime for the output file.

Help me diagnose why retention_priority_list.csv is not being created by my script.

✔️ Awesome, I've got everything!

Your full script is saved. Keep the successful output visible for the checkpoint below.

ⓧ I'd like to double check the full code

# STEP 2: Load, clean, segment, and visualize the customer base.
import pandas as pd
import matplotlib.pyplot as plt

DATA_URL = "https://raw.githubusercontent.com/IBM/telco-customer-churn-on-icp4d/master/data/Telco-Customer-Churn.csv"

df = pd.read_csv(DATA_URL)
df["TotalCharges"] = pd.to_numeric(df["TotalCharges"], errors="coerce")
df = df.dropna(subset=["TotalCharges"]).copy()
df["ChurnFlag"] = df["Churn"].eq("Yes").astype(int)

service_columns = [
    "OnlineSecurity",
    "OnlineBackup",
    "DeviceProtection",
    "TechSupport",
    "StreamingTV",
    "StreamingMovies",
]
df["ServiceCount"] = df[service_columns].eq("Yes").sum(axis=1)

df["CustomerSegment"] = "Flexible Regular"
df.loc[df["tenure"] <= 12, "CustomerSegment"] = "New Explorer"
df.loc[
    (df["tenure"] > 12) & (df["Contract"] != "Month-to-month"),
    "CustomerSegment",
] = "Committed Core"
df.loc[
    (df["tenure"] > 12) & (df["ServiceCount"] >= 4),
    "CustomerSegment",
] = "Multi-Service Champion"

segment_summary = (
    df.groupby("CustomerSegment", as_index=False)
    .agg(
        customers=("customerID", "count"),
        observed_churn_rate=("ChurnFlag", "mean"),
        monthly_revenue=("MonthlyCharges", "sum"),
    )
    .sort_values("observed_churn_rate", ascending=False)
)
segment_summary["observed_churn_rate_pct"] = (
    segment_summary["observed_churn_rate"] * 100
).round(1)

print("\nCustomer segment summary")
print(
    segment_summary[
        ["CustomerSegment", "customers", "observed_churn_rate_pct", "monthly_revenue"]
    ].to_string(index=False)
)

fig, ax = plt.subplots(figsize=(9, 4.5), layout="constrained")
ax.bar(
    segment_summary["CustomerSegment"],
    segment_summary["observed_churn_rate_pct"],
    color="#4C78A8",
)
ax.set_title("Observed churn rate by customer segment")
ax.set_xlabel("Customer segment")
ax.set_ylabel("Observed churn rate (%)")
ax.tick_params(axis="x", rotation=18)
plt.show()

# STEP 3: Build and evaluate a leakage-safe probability model.
from sklearn.compose import ColumnTransformer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (
    accuracy_score,
    average_precision_score,
    brier_score_loss,
    confusion_matrix,
)
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

numeric_features = [
    "SeniorCitizen",
    "tenure",
    "MonthlyCharges",
    "TotalCharges",
    "ServiceCount",
]
categorical_features = [
    "gender",
    "Partner",
    "Dependents",
    "PhoneService",
    "MultipleLines",
    "InternetService",
    "OnlineSecurity",
    "OnlineBackup",
    "DeviceProtection",
    "TechSupport",
    "StreamingTV",
    "StreamingMovies",
    "Contract",
    "PaperlessBilling",
    "PaymentMethod",
]
feature_columns = numeric_features + categorical_features

X = df[feature_columns]
y = df["ChurnFlag"]
X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.20,
    random_state=42,
    stratify=y,
)

preprocessor = ColumnTransformer(
    transformers=[
        ("numeric", StandardScaler(), numeric_features),
        (
            "categorical",
            OneHotEncoder(handle_unknown="ignore"),
            categorical_features,
        ),
    ]
)
model = Pipeline(
    steps=[
        ("preprocessor", preprocessor),
        ("classifier", LogisticRegression(l1_ratio=0.0, max_iter=1000)),
    ]
)
model.fit(X_train, y_train)

test_probability = model.predict_proba(X_test)[:, 1]
default_prediction = (test_probability >= 0.50).astype(int)
baseline_accuracy = max(y_test.mean(), 1 - y_test.mean())
model_accuracy = accuracy_score(y_test, default_prediction)
confusion = confusion_matrix(y_test, default_prediction, labels=[0, 1])
tn, fp, fn, tp = confusion.ravel().tolist()
average_precision = average_precision_score(y_test, test_probability)
brier_loss = brier_score_loss(y_test, test_probability)

print("\nDefault-threshold evaluation")
print(f"Majority baseline accuracy: {baseline_accuracy:.3f}")
print(f"Model accuracy at 0.50: {model_accuracy:.3f}")
print("Confusion matrix [[TN, FP], [FN, TP]]:")
print(confusion)
print(f"False negatives at 0.50: {fn}")
print(f"Average precision: {average_precision:.3f}")
print(f"Brier score loss: {brier_loss:.3f}")

feature_names = model.named_steps["preprocessor"].get_feature_names_out()
coefficients = model.named_steps["classifier"].coef_[0]
drivers = pd.DataFrame(
    {"Feature": feature_names, "Coefficient": coefficients}
).sort_values("Coefficient", ascending=False)

print("\nStrongest positive churn drivers")
print(drivers.head(8).to_string(index=False))
print("\nStrongest negative churn drivers")
print(drivers.tail(8).sort_values("Coefficient").to_string(index=False))

calibration = pd.DataFrame(
    {
        "predicted_probability": test_probability,
        "actual_churn": y_test.to_numpy(),
    }
)
calibration["probability_band"] = pd.cut(
    calibration["predicted_probability"],
    bins=[0.0, 0.2, 0.4, 0.6, 0.8, 1.0],
    include_lowest=True,
)
calibration_summary = (
    calibration.groupby("probability_band", observed=True)
    .agg(
        predicted_probability=("predicted_probability", "mean"),
        observed_churn_rate=("actual_churn", "mean"),
        customers=("actual_churn", "size"),
    )
    .reset_index()
)

print("\nCalibration by probability band")
print(calibration_summary.to_string(index=False))

fig, ax = plt.subplots(figsize=(6, 5), layout="constrained")
ax.plot([0, 1], [0, 1], linestyle="--", color="gray", label="Perfect calibration")
ax.scatter(
    calibration_summary["predicted_probability"],
    calibration_summary["observed_churn_rate"],
    s=70,
    color="#F58518",
    label="Model bands",
)
ax.set_title("Predicted probability vs observed churn")
ax.set_xlabel("Mean predicted churn probability")
ax.set_ylabel("Observed churn rate")
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.legend()
plt.show()

# STEP 4: Convert holdout probabilities into a capacity-aware value-at-risk queue.
score_columns = [
    "customerID",
    "CustomerSegment",
    "Contract",
    "tenure",
    "MonthlyCharges",
    "ServiceCount",
]
scored = df.loc[X_test.index, score_columns].copy()
scored["ActualChurn"] = y_test.to_numpy()
scored["ChurnProbability"] = test_probability
scored["AnnualRevenueProxy"] = scored["MonthlyCharges"] * 12
scored["ValueAtRisk12M"] = (
    scored["ChurnProbability"] * scored["AnnualRevenueProxy"]
)
scored = scored.sort_values("ValueAtRisk12M", ascending=False).reset_index(drop=True)
scored["ContactPriority"] = scored.index + 1

CAMPAIGN_CAPACITY_RATE = 0.10
campaign_capacity = max(1, int(len(scored) * CAMPAIGN_CAPACITY_RATE))
scored["ContactFlag"] = (
    scored["ContactPriority"] <= campaign_capacity
).astype(int)
scored["RetentionAction"] = "Monitor"

action_map = {
    "New Explorer": "Onboarding check-in",
    "Flexible Regular": "Contract review",
    "Committed Core": "Loyalty benefit review",
    "Multi-Service Champion": "Priority service review",
}
contact_mask = scored["ContactFlag"].eq(1)
scored.loc[contact_mask, "RetentionAction"] = scored.loc[
    contact_mask, "CustomerSegment"
].map(action_map)

print(f"\nCampaign capacity: {campaign_capacity} holdout customers")
print("Top ten customers by 12-month value at risk")
print(
    scored[
        [
            "ContactPriority",
            "customerID",
            "CustomerSegment",
            "ChurnProbability",
            "AnnualRevenueProxy",
            "ValueAtRisk12M",
            "RetentionAction",
        ]
    ]
    .head(10)
    .to_string(index=False)
)

# STEP 5: Build the executive view and export the decision artifact.
priority_summary = (
    scored.groupby("CustomerSegment", as_index=False)
    .agg(
        holdout_customers=("customerID", "count"),
        contacts=("ContactFlag", "sum"),
        average_churn_probability=("ChurnProbability", "mean"),
        total_value_at_risk=("ValueAtRisk12M", "sum"),
    )
    .sort_values("total_value_at_risk", ascending=False)
)

print("\nRetention decision summary")
print(priority_summary.to_string(index=False))

contacted = scored[scored["ContactFlag"].eq(1)]
monitored = scored[scored["ContactFlag"].eq(0)]
fig, ax = plt.subplots(figsize=(8, 5), layout="constrained")
ax.scatter(
    monitored["ChurnProbability"],
    monitored["AnnualRevenueProxy"],
    alpha=0.35,
    color="#9D9D9D",
    label="Monitor",
)
ax.scatter(
    contacted["ChurnProbability"],
    contacted["AnnualRevenueProxy"],
    alpha=0.8,
    color="#E45756",
    label="Contact",
)
ax.set_title("Retention queue: probability, value, and capacity")
ax.set_xlabel("Predicted churn probability")
ax.set_ylabel("12-month revenue proxy")
ax.legend()
plt.show()

export_columns = [
    "ContactPriority",
    "customerID",
    "CustomerSegment",
    "Contract",
    "tenure",
    "MonthlyCharges",
    "ServiceCount",
    "ChurnProbability",
    "AnnualRevenueProxy",
    "ValueAtRisk12M",
    "ContactFlag",
    "RetentionAction",
]
OUTPUT_FILE = "retention_priority_list.csv"
scored[export_columns].to_csv(OUTPUT_FILE, index=False)
print(f"\nCreated {OUTPUT_FILE} with {len(scored)} ranked rows.")
print(scored[export_columns].head().to_string(index=False))

Your model now produces a reviewable retention brief plus a ranked decision file. The workflow reaches all the way from raw customer records to an artifact another team can act on.

Secret mission

Optimize the Campaign for Expected Net Value

A full contact queue can still spend money on customers whose expected return does not cover the offer. Add an economic decision rule that selects positive-value customers under explicit campaign assumptions.

Clean Up Your Resources

Clean Up Your Resources

Google Colab is free of charge for this project. Choose the cleanup option that fits what you want to do next.

Resources you used:

  • A Google Colab project notebook stored in Google Drive.
  • A managed runtime containing retention_prioritizer.py.
  • The retention_priority_list.csv capacity-aware retention queue.
  • The economics_optimized_retention_list.csv economics-optimized retention queue.

Keep everything running

No cleanup action is needed. Choose this option if you plan to keep developing the prioritizer.

  • Leave the project notebook in Google Drive.
  • Download retention_prioritizer.py if you want a local copy of the complete script.
  • Download retention_priority_list.csv if you want to retain the capacity-aware queue.
  • Download economics_optimized_retention_list.csv if you want to retain the economics-optimized queue.

The notebook remains saved in Google Drive. Files inside the managed runtime remain temporary.

Pause - I'll come back to this later

Pausing keeps the notebook while allowing the temporary runtime to expire after inactivity. Choose this option when you plan to return later.

  • Download retention_prioritizer.py if you want to preserve the generated script.
  • Download retention_priority_list.csv if you want to preserve the capacity-aware queue.
  • Download economics_optimized_retention_list.csv if you want to preserve the economics-optimized queue.
  • Close the Google Colab notebook tab.
  • Return later by opening the same notebook from Google Drive.

Your notebook stays saved. Google Colab may reclaim the managed runtime after inactivity.

Delete - I don't want to use this again

Removing the runtime and notebook is permanent. A backup protects any artifact you value.

  • Download retention_prioritizer.py if you want to keep the complete script.
  • Download retention_priority_list.csv if you want to keep the capacity-aware queue.
  • Download economics_optimized_retention_list.csv if you want to keep the economics-optimized queue.
  • Select Runtime in the Google Colab menu bar.
  • Select Disconnect and delete runtime.
  • Confirm that the notebook no longer has an active runtime connection.

The runtime session ends. Its copies of retention_prioritizer.py and both generated CSV files are removed.

  • Return to Google Drive.
  • Delete the project notebook from Google Drive.
  • Confirm that the notebook no longer appears in your Google Drive files.

Nice Work!

Outstanding work! Your Google Colab project now turns raw customer records into a ranked retention decision artifact. You also tested how campaign economics changes the contact list under explicit assumptions.

You've learned how to:

  • Turn raw customer records into business segments that reveal observed churn differences in a bar chart.
  • Train a leakage-safe mixed-type logistic regression pipeline. Expose missed churners with a confusion matrix. Evaluate ranking quality with average precision. Check probability quality with Brier score loss plus a calibration chart.
  • Rank the untouched holdout cohort with a 12-month value-at-risk proxy. Apply outreach capacity plus segment-specific retention actions. Export the decision queue to retention_priority_list.csv.
  • Secret Mission: Re-rank the campaign by expected net value under editable offer-cost plus save-rate assumptions. Export the economics-optimized queue to economics_optimized_retention_list.csv.

Ready to quiz yourself?