Customer Churn Prediction with sklearn

Build an end-to-end ML pipeline to predict telecom customer churn.

Introduction

30 Second Summary

Every subscription service you use has a team dedicated to one question: which customers are about to cancel? If they can predict who is leaving, they can intervene before it happens.

In this project, you will build a machine learning model that predicts which telecom customers will cancel their service. You will load real customer data, handle class imbalance, train three different models, and evaluate them with industry-standard metrics.

What You'll Build

You will have a Jupyter Notebook where you feed in customer attributes and get back a churn probability, with three models compared side-by-side and clear visualizations showing which features drive customer loss.

By the end of this project, you'll have:

  • A working prediction pipeline where you feed in customer attributes (tenure, contract type, monthly charges) and get a churn probability back.
  • A side-by-side model comparison showing Logistic Regression vs Random Forest vs Gradient Boosting, evaluated with ROC-AUC scores and confusion matrices.
  • Visualizations of which features drive churn the most, helping you explain your model's decisions in interviews.
  • Secret Mission: Extract feature importances and write actionable business recommendations based on your model's findings.

Are there any prerequisites?

You should be comfortable with basic Python syntax (variables, loops, functions) and familiar with pandas DataFrames. No prior machine learning experience is required.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters.

You are building a machine learning model that predicts which telecom customers will cancel their service. This is one of the most common real-world ML problems companies hire data scientists to solve.

Explore the Data and Find the Problem

Your data is loaded, but loading is not understanding. Before training any model, you need to see what patterns live inside your dataset and spot problems that could silently ruin your predictions.

In this step, you will explore the churn dataset visually and uncover a critical problem: class imbalance. Only about 27% of customers actually churned. This means a naive model could get 73% accuracy by always predicting "No Churn" while being completely useless at the actual business task.

In this step, get ready to:
  • Examine the dataset structure and discover the class imbalance problem.
  • Visualize which features correlate with customer churn.
  • Clean the data so it is ready for modeling.
Explore the dataset structure and class distribution

Every dataset has a story hiding in its column types, null counts, and value distributions. The first step of exploratory data analysis (EDA) is checking what kind of data you actually have.

  • In your notebook, create a new cell below Cell 2 and check the data types by running:
# Cell 3: Data Info
df.info()

You should see a table listing each column, its data type, and how many non-null entries it has. Notice that TotalCharges might show up as object instead of a number. You will fix that later in this step.

Now for the most important check. Before you run the next cell, think about this: what percentage of customers do you expect churned? Is it a 50/50 split, or something else?

  • Check the target variable distribution and visualize it by running this in a new cell:
# Cell 4: Class Distribution
print(df['Churn'].value_counts(normalize=True))
sns.countplot(x='Churn', data=df)
plt.title('Churn Distribution')
plt.show()

You should see a bar chart with a tall bar for "No" (roughly 73%) and a much shorter bar for "Yes" (roughly 27%). This is class imbalance.

Why is class imbalance a problem?

A model that always predicts "No Churn" would score 73% accuracy. That sounds good until you realize it catches zero actual churners. For a telecom company, missing churners means losing customers they could have saved.

This is why accuracy alone is a misleading metric for imbalanced datasets. You will see this problem in action when you build a naive model in the next step.

Visualize key relationships

Numbers in a table are hard to interpret at a glance. Visualizations reveal patterns instantly. You want to see which features look different for customers who churned versus those who stayed.

  • Plot three key relationships side by side by running this in a new cell:
# Cell 5: Visualize Key Relationships
fig, axes = plt.subplots(1, 3, figsize=(15, 5))

sns.boxplot(x='Churn', y='MonthlyCharges', data=df, ax=axes[0])
axes[0].set_title('Monthly Charges vs Churn')

sns.boxplot(x='Churn', y='Tenure', data=df, ax=axes[1])
axes[1].set_title('Tenure vs Churn')

sns.countplot(x='Contract', hue='Churn', data=df, ax=axes[2])
axes[2].set_title('Contract Type vs Churn')

plt.tight_layout()
plt.show()

What does this code do?

  • plt.subplots(1, 3, figsize=(15, 5)) creates a single figure with three side-by-side panels.
  • The first boxplot compares MonthlyCharges for churners vs non-churners.
  • The second boxplot compares Tenure (how long they have been a customer).
  • The third countplot breaks down contract types by churn status.

You should see three charts appear. Look for these patterns: churners tend to have higher monthly charges, shorter tenure, and month-to-month contracts. Short tenure plus month-to-month contract plus high charges equals likely churn.

Not seeing any charts?

  • Make sure you ran Cell 1 (imports) first. The matplotlib and seaborn imports must be loaded before you can plot.
  • Check that your column names match. Some dataset versions use lowercase (tenure) instead of title case (Tenure). Run df.columns.tolist() to check your exact column names.

Help me debug my seaborn charts not displaying

Handle data quality issues

Raw datasets almost always have quirks that will break your model. The TotalCharges column contains whitespace characters that prevent it from being treated as a number. You also need to drop columns that are not useful features and encode the target variable as numeric.

  • Clean the dataset by running this in a new cell:
# Cell 6: Data Cleaning
df['TotalCharges'] = pd.to_numeric(df['TotalCharges'], errors='coerce')
df.dropna(inplace=True)
df.drop('CustomerID', axis=1, inplace=True)

df['Churn'] = df['Churn'].map({'Yes': 1, 'No': 0}) if df['Churn'].dtype == 'object' else df['Churn']
print(f"Clean dataset shape: {df.shape}")

What does this code do?

  • pd.to_numeric(..., errors='coerce') converts TotalCharges to a number. Any value it cannot convert becomes NaN instead of crashing.
  • df.dropna(inplace=True) removes the handful of rows with missing values.
  • Dropping CustomerID removes a column that is just an identifier, not a predictive feature.
  • The final line maps "Yes"/"No" strings to 1/0 so the model can work with numeric targets.

You should see output like Clean dataset shape: (9950, 8) (the exact numbers may vary slightly depending on your dataset version). The row count dropped by a few because of the null values you removed.

Seeing a KeyError?

  • If you see an error mentioning a column name, your dataset likely uses different capitalization. Run df.columns.tolist() to see exact column names, then update the code to match.
  • If the Churn column is already numeric (0 and 1), the mapping line skips itself automatically. No action needed.

Help me fix my data cleaning cell

✔️ Awesome, I've got everything!

Double check that you have saved your notebook with all six cells.

ⓧ I'd like to double check the full code

Scroll through your notebook and confirm these cells are present in order:

  • Cell 1: Imports (all libraries imported, prints "All imports successful!").
  • Cell 2: Load Data (reads CSV, prints shape, shows head).
  • Cell 3: Data Info (df.info()).
  • Cell 4: Class Distribution (prints value counts, shows countplot).
  • Cell 5: Visualize Key Relationships (three side-by-side plots).
  • Cell 6: Data Cleaning (converts TotalCharges, drops nulls, drops CustomerID, encodes Churn, prints clean shape).

You now know the shape of your data, the key patterns driving churn, and the critical class imbalance problem. Next up, you will build a naive model that ignores all of this and watch it fail at the actual business task.

Build a Naive Model (and Watch It Fail)

Your EDA revealed a critical fact: only ~27% of customers in this dataset actually churned.

In this step, you will train a quick Logistic Regression without handling that imbalance. The model will appear to work. Then you will look closer and discover it is completely useless for the actual business problem.

In this step, get ready to:
  • Prepare features and split data into training and test sets.
  • Train a naive Logistic Regression and evaluate its performance.
  • Understand why accuracy is misleading for imbalanced datasets.
Prepare features and split the data

Before training any model, you need to separate your data into features (the inputs) and the target (the output you want to predict). You also need a train/test split so you can measure performance on data the model has never seen.

  • Create a new cell in your notebook and enter the following code to prepare features and split the data:
# Encode categorical columns as numbers (quick approach)
X_naive = pd.get_dummies(df.drop('Churn', axis=1), drop_first=True)
y = df['Churn']

# Split into train (80%) and test (20%), preserving class ratio
X_train_naive, X_test_naive, y_train, y_test = train_test_split(
    X_naive, y, test_size=0.2, random_state=42, stratify=y
)

What does this code do?

pd.get_dummies() converts categorical columns (like Contract and PaymentMethod) into numeric columns that a model can read. The drop_first=True avoids redundancy by dropping one category per feature.

stratify=y ensures the 73/27 class ratio is preserved in both the training and test sets. Without this, one set could end up with more churners by random chance.

  • Run the cell.
  • In a new cell below, run X_train_naive.shape to confirm the split worked. You should see a tuple showing roughly 80% of your total rows and the number of feature columns.

Seeing a KeyError or empty DataFrame?

Make sure you ran all previous cells first (Cells 1 through 6). The df variable must already exist and have the Churn column encoded as 0/1.

If you see a column-name error, check your CSV's exact column names with df.columns.tolist(). Column names are case-sensitive.

Help me fix my data preparation code

Train a naive Logistic Regression

Your data is split and ready. Now you will train the simplest possible model and look at how it performs. Remember: ~73% of your data is "No Churn" and only ~27% is "Churn".

Before you run the next cell, ask yourself: if the model learns nothing useful and just predicts "No Churn" for everyone, what accuracy would it get?

  • Create a new cell and enter this code to train and evaluate the naive model:
# Train a basic Logistic Regression (no class balancing)
lr_naive = LogisticRegression(max_iter=1000, random_state=42)
lr_naive.fit(X_train_naive, y_train)
y_pred_naive = lr_naive.predict(X_test_naive)

# Evaluate performance
print("=== NAIVE MODEL (No SMOTE, No Pipeline) ===")
print(classification_report(y_test, y_pred_naive))
print("Notice: Recall for class 1 (Churn) is poor!")
  • Run the cell.

Look at the classification report. The overall accuracy is around 79-81%. That sounds decent. Now look at the recall value for class 1 (the Churn row). It is below 0.55. The model is missing nearly half of all customers who actually churned.

What does this classification report tell me?

  • Precision for class 1: of all the customers the model flagged as churners, how many actually churned.
  • Recall for class 1: of all customers who actually churned, how many did the model catch. This is the critical metric here.
  • Accuracy is misleading because a model that always predicts "No Churn" would still get ~73% accuracy just by guessing the majority class.

Seeing a ConvergenceWarning?

This warning means the model did not fully converge during training. The max_iter=1000 setting should prevent this, but if you see it, the results are still usable for this demonstration. The poor recall is the point, not a bug.

Help me understand my model's output

Understand why this model fails the business problem

This is exactly the result you should see. The model is not broken. It is doing exactly what you told it to do: optimize for overall accuracy.

Why does high accuracy mean a useless model here?

The model is biased toward the majority class (No Churn) because that is what 73% of the training data looks like. Predicting "No Churn" is the safe bet that maximizes accuracy.

For a telecom company, this model is worse than useless. Every churner it misses is a customer who cancels their plan. The whole point of building this model is to catch churners before they leave. A recall below 0.55 means you are losing nearly half of them.

In the next step, you will fix this with SMOTE (to balance the training data) and a proper scikit-learn Pipeline (to prevent data leakage).

✔️ Awesome, I've got everything!

Great. Make sure your notebook is saved. You should have your data preparation cell and your naive model training cell both showing their outputs.

ⓧ I'd like to double check the full code

Here is the complete code you added in this step. Your notebook should now contain this as the cell(s) after your data cleaning code:

# Cell 7: Naive Model (Quick and Dirty)
X_naive = pd.get_dummies(df.drop('Churn', axis=1), drop_first=True)
y = df['Churn']

X_train_naive, X_test_naive, y_train, y_test = train_test_split(
    X_naive, y, test_size=0.2, random_state=42, stratify=y
)

lr_naive = LogisticRegression(max_iter=1000, random_state=42)
lr_naive.fit(X_train_naive, y_train)
y_pred_naive = lr_naive.predict(X_test_naive)

print("=== NAIVE MODEL (No SMOTE, No Pipeline) ===")
print(classification_report(y_test, y_pred_naive))
print("Notice: Recall for class 1 (Churn) is poor!")

You have proven the problem: a naive model looks fine on the surface but fails at the one thing you actually need it to do. Next up, you will fix this with SMOTE and a proper Pipeline that dramatically improves recall on churners.

Build a Proper Pipeline with SMOTE

Your naive model just proved something important. High accuracy means nothing when the model misses half the customers who actually churn.

The fix has two parts. First, you need a ColumnTransformer that properly scales numeric data and encodes categorical data in a single object. Second, you need SMOTE to generate synthetic minority samples so the model learns what churners actually look like.

In this step, get ready to:
  • Define a proper preprocessing pipeline with ColumnTransformer.
  • Apply SMOTE to balance your training data without leaking into the test set.
  • Retrain Logistic Regression and see recall jump significantly.
Define preprocessing with ColumnTransformer

Your dataset has two types of columns that need different treatment. Numeric columns like Tenure and MonthlyCharges need to be scaled so large values do not dominate. Categorical columns like Contract and PaymentMethod need to be converted to numbers the model can read.

  • Create a new cell in your notebook and define the preprocessor by running this code:
# Cell 8: Proper Preprocessing with ColumnTransformer
numeric_features = ['Tenure', 'MonthlyCharges', 'TotalCharges', 'SeniorCitizen']
categorical_features = ['Gender', 'Contract', 'PaymentMethod']

preprocessor = ColumnTransformer(
    transformers=[
        ('num', StandardScaler(), numeric_features),
        ('cat', OneHotEncoder(drop='first', handle_unknown='ignore'), categorical_features)
    ]
)

What does this code do?

A ColumnTransformer applies different transformations to different columns in one clean step.

  • StandardScaler centers numeric columns to mean 0 and scales them to unit variance. This stops high-value columns from dominating the model.
  • OneHotEncoder converts each category into a binary column. The drop='first' parameter avoids redundancy by dropping one category per feature.

Check your column names

Column names are case-sensitive. If your dataset uses lowercase names (like tenure instead of Tenure), run df.columns.tolist() to check and adjust the lists above to match.

  • Run the cell. It should complete without any errors or output.

Seeing a KeyError?

A KeyError means a column name in your list does not match your dataset exactly. Run df.columns.tolist() to see the actual column names and update your lists to match.

help me fix a KeyError in my ColumnTransformer setup

Apply SMOTE to the training data only

Now you need to split your data fresh (using the original DataFrame columns, not the get_dummies version from the naive model), transform it with your preprocessor, and then apply SMOTE to the training set.

  • Create a new cell to prepare your features and apply the preprocessor:
# Cell 9: Prepare Features
X = df.drop('Churn', axis=1)
y = df['Churn']

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

X_train_processed = preprocessor.fit_transform(X_train)
X_test_processed = preprocessor.transform(X_test)

Why fit_transform on train but only transform on test?

fit_transform learns the scaling parameters (mean, standard deviation) from the training data and then applies them. transform on the test set applies those same learned parameters without re-learning.

This prevents data leakage. If you fit on the test set too, information from the test data leaks into your preprocessing and your evaluation scores are artificially inflated.

  • Run the cell. It should complete without errors.
  • Now create a new cell to apply SMOTE and see the class balance change:
# Cell 10: Apply SMOTE to Training Data Only
smote = SMOTE(random_state=42)
X_train_resampled, y_train_resampled = smote.fit_resample(X_train_processed, y_train)

print(f"Before SMOTE: {pd.Series(y_train).value_counts().to_dict()}")
print(f"After SMOTE:  {pd.Series(y_train_resampled).value_counts().to_dict()}")

What is SMOTE doing here?

SMOTE (Synthetic Minority Over-sampling Technique) creates new synthetic examples of the minority class (churners) by interpolating between existing ones. It does not simply duplicate rows.

Crucially, you apply SMOTE to the training data only. If you applied it before splitting, synthetic data based on test samples would leak into your training set.

  • Run the cell. You should see the class counts printed, with both classes now showing equal numbers after SMOTE.

The "Before" line shows your original imbalanced split. The "After" line shows both classes at the same count. Your model now has equal examples to learn from.

Seeing a ValueError about input shape?

Make sure you ran Cell 9 first (the preprocessing step). SMOTE needs the already-transformed X_train_processed array, not the raw DataFrame.

help me debug a SMOTE ValueError

Train Logistic Regression on balanced data

Now for the moment of truth. You will train the same Logistic Regression algorithm on the balanced data and compare its recall to the naive model from the previous step.

  • Before you run the next cell, predict: do you think recall for the Churn class will be higher or lower than the naive model's recall (which was below 0.55)?
  • Create a new cell to train and evaluate:
# Cell 11: Train Logistic Regression on Balanced Data
lr = LogisticRegression(max_iter=1000, random_state=42)
lr.fit(X_train_resampled, y_train_resampled)
y_pred_lr = lr.predict(X_test_processed)

print("=== LOGISTIC REGRESSION (with SMOTE) ===")
print(classification_report(y_test, y_pred_lr))
print("Pipeline fitted successfully")

What does this code do?

  • lr.fit(X_train_resampled, y_train_resampled) trains on the SMOTE-balanced data so the model sees equal examples of churners and non-churners.
  • lr.predict(X_test_processed) evaluates against the untouched test set. The test set was never resampled, so this score reflects real-world performance.
  • classification_report shows precision, recall, and F1 for each class. Focus on recall for class 1 (Churn).
  • Run the cell.

You should see === LOGISTIC REGRESSION (with SMOTE) === followed by the classification report. Look at the recall value for class 1 (Churn). It should now be above 0.70, a massive improvement over the naive model's score below 0.55.

The last line prints Pipeline fitted successfully. That confirms your entire preprocessing and training pipeline ran cleanly.

Why did recall improve so much?

The naive model was biased toward predicting "No Churn" because that was the majority class. By balancing the training data with SMOTE, the model now has equal incentive to learn both classes. It catches more actual churners (higher recall) even if it flags a few non-churners as false positives.

Recall for Churn is still low?

Make sure you are training on X_train_resampled and y_train_resampled (the SMOTE-balanced data), not the original X_train_processed and y_train. Also confirm you are predicting on X_test_processed (the preprocessed test set).

help me debug low recall after SMOTE

✔️ Awesome, I've got everything!

Great. Double check you have saved your notebook with all four new cells (Cell 8 through Cell 11).

ⓧ I'd like to double check the full code

Here are the four cells you added in this step. Scroll up and confirm each one is present in order.

Cell 8 (Preprocessor):

# Cell 8: Proper Preprocessing with ColumnTransformer
numeric_features = ['Tenure', 'MonthlyCharges', 'TotalCharges', 'SeniorCitizen']
categorical_features = ['Gender', 'Contract', 'PaymentMethod']

preprocessor = ColumnTransformer(
    transformers=[
        ('num', StandardScaler(), numeric_features),
        ('cat', OneHotEncoder(drop='first', handle_unknown='ignore'), categorical_features)
    ]
)

Cell 9 (Prepare Features):

# Cell 9: Prepare Features
X = df.drop('Churn', axis=1)
y = df['Churn']

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

X_train_processed = preprocessor.fit_transform(X_train)
X_test_processed = preprocessor.transform(X_test)

Cell 10 (SMOTE):

# Cell 10: Apply SMOTE to Training Data Only
smote = SMOTE(random_state=42)
X_train_resampled, y_train_resampled = smote.fit_resample(X_train_processed, y_train)

print(f"Before SMOTE: {pd.Series(y_train).value_counts().to_dict()}")
print(f"After SMOTE:  {pd.Series(y_train_resampled).value_counts().to_dict()}")

Cell 11 (Train on Balanced Data):

# Cell 11: Train Logistic Regression on Balanced Data
lr = LogisticRegression(max_iter=1000, random_state=42)
lr.fit(X_train_resampled, y_train_resampled)
y_pred_lr = lr.predict(X_test_processed)

print("=== LOGISTIC REGRESSION (with SMOTE) ===")
print(classification_report(y_test, y_pred_lr))
print("Pipeline fitted successfully")

Your model now catches significantly more churners. But one model is never enough for a real project. Next up, you will train Random Forest and Gradient Boosting alongside Logistic Regression and crown a winner using ROC-AUC curves.

Compare Three Models and Pick the Winner

You have a working Logistic Regression that catches churners far better than the naive version. But one model is not enough for a portfolio project.

Interviewers want to see that you compared alternatives and made a data-driven decision. In this step, you will train Random Forest and Gradient Boosting alongside your Logistic Regression and compare all three using ROC-AUC.

In this step, get ready to:
  • Train Random Forest and Gradient Boosting on the balanced data.
  • Evaluate all three models side-by-side with ROC-AUC scores.
  • Plot ROC curves and identify the winner.
Train Random Forest and Gradient Boosting

Random Forest creates hundreds of decision trees and averages their votes. Gradient Boosting builds trees sequentially, with each tree correcting the errors of the previous one.

  • Create a new cell in your notebook (Cell 12) and train both models by running:
# Cell 12: Train Random Forest and Gradient Boosting
rf = RandomForestClassifier(n_estimators=200, random_state=42, class_weight='balanced')
rf.fit(X_train_resampled, y_train_resampled)
y_pred_rf = rf.predict(X_test_processed)

gb = GradientBoostingClassifier(n_estimators=200, learning_rate=0.1, random_state=42)
gb.fit(X_train_resampled, y_train_resampled)
y_pred_gb = gb.predict(X_test_processed)

What does this code do?

  • RandomForestClassifier(n_estimators=200) creates a forest of 200 decision trees. class_weight='balanced' gives the model an extra push to pay attention to the minority class.
  • GradientBoostingClassifier(n_estimators=200, learning_rate=0.1) builds 200 trees sequentially, each learning from the previous tree's mistakes at a controlled pace.
  • Both models train on X_train_resampled (the SMOTE-balanced data), then predict on the untouched test set.
  • Run the cell. Training may take a few seconds since these models are more complex than Logistic Regression.
  • The cell completes without errors. No output is printed, but both rf and gb are now trained and ready for evaluation.

Seeing a MemoryError or very slow training?

If your machine struggles, try reducing n_estimators to 100. This trains fewer trees and uses less memory. The results will still be meaningful for comparison.

Help me debug my model training

Evaluate all three models with ROC-AUC

ROC-AUC measures how well a model separates the two classes across all possible thresholds. A score of 1.0 is perfect separation. A score of 0.5 is no better than random guessing.

  • Create Cell 13 and compute ROC-AUC for all three models by running:
# Cell 13: Compare Models with ROC-AUC
lr_auc = roc_auc_score(y_test, lr.predict_proba(X_test_processed)[:, 1])
rf_auc = roc_auc_score(y_test, rf.predict_proba(X_test_processed)[:, 1])
gb_auc = roc_auc_score(y_test, gb.predict_proba(X_test_processed)[:, 1])

results = pd.DataFrame({
    'Model': ['Logistic Regression', 'Random Forest', 'Gradient Boosting'],
    'ROC-AUC': [lr_auc, rf_auc, gb_auc]
}).sort_values('ROC-AUC', ascending=False)

print("\n=== MODEL COMPARISON ===")
print(results.to_string(index=False))

What does this code do?

  • predict_proba(X_test_processed)[:, 1] gets the predicted probability of churn (class 1) for each customer. ROC-AUC uses these probabilities, not hard yes/no predictions.
  • The results DataFrame sorts all three models by their ROC-AUC score, highest first. This makes it immediately clear which model wins.
  • Run the cell. You should see a printed comparison table with all three models ranked by their ROC-AUC score.
  • Gradient Boosting should appear at the top with the highest score (typically around 0.83-0.85).

Why ROC-AUC instead of accuracy?

You already saw in Step 3 that accuracy is misleading for imbalanced datasets. ROC-AUC evaluates the model's ability to rank churners higher than non-churners across every possible decision threshold. It is the standard metric for binary classification problems in industry.

All three scores look very similar?

Small differences (0.01-0.02) between models are normal. The important thing is that all three beat the naive model significantly. If all scores are below 0.60, check that you trained on X_train_resampled (the SMOTE-balanced data), not X_train_processed.

Help me understand my ROC-AUC scores

Plot ROC curves and identify the winner

A comparison table gives you the numbers. An ROC curve plot gives you the visual proof that one model consistently outperforms the others at every threshold.

  • Create Cell 14 and plot all three ROC curves on one figure by running:
# Cell 14: Plot ROC Curves
fig, ax = plt.subplots(figsize=(8, 6))

for model, name in [(lr, 'Logistic Regression'), (rf, 'Random Forest'), (gb, 'Gradient Boosting')]:
    y_proba = model.predict_proba(X_test_processed)[:, 1]
    fpr, tpr, _ = roc_curve(y_test, y_proba)
    auc_score = roc_auc_score(y_test, y_proba)
    ax.plot(fpr, tpr, label=f'{name} (AUC = {auc_score:.3f})')

ax.plot([0, 1], [0, 1], 'k--', label='Random Classifier')
ax.set_xlabel('False Positive Rate')
ax.set_ylabel('True Positive Rate')
ax.set_title('ROC Curve Comparison - Customer Churn Prediction')
ax.legend()
plt.tight_layout()
plt.show()

What does this plot show?

  • Each curve shows the trade-off between catching churners (True Positive Rate) and false alarms (False Positive Rate) at different thresholds.
  • The dashed diagonal line represents a random classifier. Any model worth using must curve above it.
  • The curve that hugs the top-left corner most tightly is the best model. Its AUC score is the area under that curve.
  • Run the cell. You should see a plot with three colored curves and a dashed diagonal line.
  • The Gradient Boosting curve should sit highest, with its AUC score in the legend confirming it is the winner.

Now print the winning model's full classification report so you have the complete picture.

  • Create Cell 15 and print the best model's detailed metrics by running:
# Cell 15: Best Model - Full Report
best_model = gb
y_pred_best = best_model.predict(X_test_processed)

print("\n=== BEST MODEL: FULL CLASSIFICATION REPORT ===")
print(classification_report(y_test, y_pred_best))
print("\nConfusion Matrix:")
print(confusion_matrix(y_test, y_pred_best))

What does this code do?

  • best_model = gb assigns Gradient Boosting as the winner. If your results show a different winner, swap gb for rf or lr.
  • classification_report shows precision, recall, and F1-score for each class. The confusion matrix shows exactly how many churners were caught versus missed.

Before you run this cell, what do you expect the Churn recall to be for Gradient Boosting compared to the naive model from Step 3?

  • Run the cell.
  • You should see a full classification report with Churn recall significantly above the naive model's ~0.55. The confusion matrix confirms how many churners the model correctly identified.

Recall for Churn is still below 0.60?

Double-check that your best_model variable points to the model that actually scored highest in your comparison table. Also verify that you ran Cell 10 (SMOTE) before training.

Help me understand my model's classification report

✔️ Awesome, I've got everything!

Great. Double check you have saved your notebook with all cells from Cell 12 through Cell 15.

ⓧ I'd like to double check the full code

Scroll through your notebook and confirm the following cells are present in order:

  • Cell 12: Train Random Forest and Gradient Boosting (trains rf and gb, generates predictions for both).
  • Cell 13: Compare Models with ROC-AUC (computes lr_auc, rf_auc, gb_auc, prints sorted comparison table).
  • Cell 14: Plot ROC Curves (plots all three curves plus a diagonal reference line).
  • Cell 15: Best Model - Full Report (prints classification report and confusion matrix for gb).

Secret mission

Feature Importance and Business Recommendations

A model that predicts churn is only half the job. The real value is explaining WHY customers churn and recommending actions the business can take. Can you extract feature importances, visualize the top churn drivers, and translate your technical findings into business language?

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Your notebook and dataset live on your machine in the churn-prediction folder.

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • The churn-prediction folder on your Desktop (contains your notebook and dataset).
  • The imbalanced-learn Python package installed via pip.

Keep everything running

No action needed. Choose this if you are still actively building or plan to push your notebook to GitHub as a portfolio project.

  • Your churn-prediction folder stays on your Desktop.
  • The imbalanced-learn package remains installed in your Python environment.
  • Consider pushing your completed notebook to GitHub so others can see your work.

Pause - I'll come back to this later

Shut down Jupyter Notebook to free up memory. Your files stay on disk for when you return.

  • In the terminal where Jupyter is running, press Ctrl+C to stop the Jupyter server.
  • Your notebook and dataset remain in the churn-prediction folder, ready to reopen anytime.

Delete - I don't want to use this again

Remove all project files and uninstall the package you added during this project.

  • Stop Jupyter Notebook by pressing Ctrl+C in the terminal where it is running.
  • Delete the project folder by running this command on macOS/Linux:
rm -rf ~/Desktop/churn-prediction

On Windows, run:

rmdir /s /q %USERPROFILE%\Desktop\churn-prediction
  • Uninstall imbalanced-learn by running:
pip uninstall imbalanced-learn

Confirm the removal by entering y when prompted.

Nice Work!

Nice Work!

You just built a complete customer churn prediction system from scratch using scikit-learn! Starting with raw telecom data, you explored, cleaned, and transformed it into a machine learning pipeline that identifies at-risk customers before they leave.

You've learned how to:

  • Build an end-to-end ML pipeline from raw data to predictions, using ColumnTransformer and SMOTE to prevent data leakage and handle class imbalance.
  • Compare three models (Logistic Regression, Random Forest, Gradient Boosting) and pick a winner using ROC-AUC scores and ROC curve visualizations.
  • Understand why accuracy alone is misleading for imbalanced datasets by building a naive model that looks good on paper but fails at the actual business problem.
  • Secret Mission: Extracted feature importances from your best model and translated them into actionable business recommendations a stakeholder could act on.

Ready to quiz yourself?