Advanced EDA for Credit Score Churn

Uncover why bank customers churn using statistical EDA on 10,000 records.

Introduction

30 Second Summary

Banks lose millions every year when customers leave without warning. If patterns hiding in the data could predict who is about to walk out the door, catching them early could save entire customer relationships.

In this project, you will perform advanced exploratory data analysis on a real bank customer dataset to uncover the statistical patterns behind customer churn. You'll go beyond basic charts to apply hypothesis testing, outlier detection, and feature engineering that produces actionable business insights.

What You'll Build

A comprehensive Jupyter Notebook that analyzes 10,000 European bank customer records using statistical tests, correlation analysis, and engineered features to identify the strongest predictors of customer churn.

By the end of this project, you'll have:

  • A fully-documented EDA notebook with data quality assessment, missing value analysis, and class imbalance quantification on 10,000 customer records.
  • Statistical evidence from chi-square tests and point-biserial correlation ranking the strongest predictors of customer churn.
  • Engineered features (BalancePerProduct, TenurePerAge, AgeGroup) that amplify churn signal and are ready for downstream modelling.
  • Secret Mission: Validate your EDA findings with SHAP-based feature importance using a Random Forest model to bridge statistical analysis with machine learning interpretability.

Are there any prerequisites?

You should have basic Python knowledge and familiarity with pandas DataFrames. No prior statistics experience is required.

Load and Assess Data Quality

Your environment is ready with all the libraries installed and the dataset waiting in your project folder. Now it is time to open that data and understand what you are working with.

Before building any charts or running statistical tests, you need to know the basics: how big is the dataset, are there missing values, and how imbalanced is the target variable. This step builds that foundation.

In this step, get ready to:
  • Import your data science libraries and configure plot styling.
  • Load and inspect the churn dataset for shape, types, and structure.
  • Assess data quality, quantify class imbalance, and remove irrelevant columns.
Import libraries and configure styling

Every analysis notebook starts with an imports cell that loads the tools you need and sets consistent styling for all plots.

  • In your churn_eda.ipynb notebook, click into the first empty cell and paste the following code:
# Cell 1: Imports and Setup
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
from scipy import stats

sns.set_style('whitegrid')
plt.rcParams['figure.figsize'] = (12, 6)

What does this code do?

  • pandas handles tabular data, numpy provides math operations, seaborn and matplotlib produce charts, and scipy.stats provides statistical tests.
  • sns.set_style('whitegrid') applies a clean grid background to every plot so they are easier to read.
  • plt.rcParams['figure.figsize'] = (12, 6) sets a wide default figure size for all charts in the notebook.
  • Run the cell by pressing Shift+Enter.

No visible output appears. The cell runs silently because it only imports modules and sets configuration.

Seeing a ModuleNotFoundError?

This means one of the libraries is not installed in the Python environment Jupyter is using.

  • Open a new terminal and re-run the install command from Step 1: pip install pandas==2.2.3 numpy seaborn==0.13.2 scipy==1.14.1 matplotlib.
  • Make sure Jupyter is running in the same Python environment where you installed the packages.

Help me fix a ModuleNotFoundError in Jupyter Notebook

Load and inspect the dataset

With imports ready, you can load the CSV file and take a first look at its structure. This tells you how many rows and columns you have, what data types each column holds, and what the raw data looks like.

  • Create a new cell below by pressing Shift+Enter (which runs the current cell and creates a new one), then paste the following code:
# Cell 2: Load Data
df = pd.read_csv('Churn_Modelling.csv')
print(f"Dataset shape: {df.shape}")
print(f"\nColumn types:\n{df.dtypes}")
print(f"\nFirst 5 rows:")
df.head()

What does this code do?

  • pd.read_csv('Churn_Modelling.csv') reads the CSV file into a DataFrame called df.
  • df.shape reports the row and column count (expect 10,000 rows and 14 columns).
  • df.dtypes lists each column and its data type so you can spot numeric vs object (text) features.
  • df.head() displays the first 5 rows as a preview table.
  • Run the cell with Shift+Enter.

You should see output showing 10,000 rows and 14 columns, followed by a column type listing and a table of the first 5 customers.

What is the target variable?

The Exited column is the target variable for churn prediction. A value of 1 means the customer churned (left the bank). A value of 0 means the customer was retained.

Seeing a FileNotFoundError?

This means Python cannot find Churn_Modelling.csv in the folder where Jupyter is running.

  • Confirm that Churn_Modelling.csv is saved in the same folder where you launched jupyter notebook.
  • Check the filename spelling exactly, including the underscore and capital letters.

Help me fix a FileNotFoundError for my CSV file in Jupyter

Check data quality and quantify class imbalance

Now that the data is loaded, you need to answer two critical questions: is anything missing or duplicated, and how imbalanced is the churn target? Imbalanced targets affect every downstream analysis and model.

  • Create a new cell and paste the following code to check for missing values and duplicates:
# Cell 3: Data Quality Check
print("=" * 50)
print("DATA QUALITY ASSESSMENT")
print("=" * 50)
print(f"\nMissing values:\n{df.isnull().sum()}")
print(f"\nDuplicate rows: {df.duplicated().sum()}")
print(f"\nBasic statistics:")
df.describe()

What does this code do?

  • df.isnull().sum() counts missing values in every column. Zero across the board means no imputation is needed.
  • df.duplicated().sum() counts rows that are exact copies of another row.
  • df.describe() shows summary statistics (mean, std, min, max, quartiles) for all numeric columns.
  • Run the cell with Shift+Enter.

You should see 0 missing values across all columns, 0 duplicate rows, and a statistics table showing ranges for each numeric feature.

Seeing unexpected missing values?

If you see non-zero missing value counts, your CSV may have been corrupted during download.

  • Re-download Churn_Modelling.csv from Kaggle and replace the file in your project folder.
  • Re-run Cell 2 and Cell 3 to reload and re-check.

Help me troubleshoot unexpected missing values in my churn dataset

  • Create a new cell and paste the following code to visualize the class imbalance:
# Cell 4: Class Imbalance Analysis
churn_counts = df['Exited'].value_counts()
churn_rate = df['Exited'].mean()
print(f"Churn rate: {churn_rate:.1%}")
print(f"Retained: {churn_counts[0]} ({churn_counts[0]/len(df):.1%})")
print(f"Churned: {churn_counts[1]} ({churn_counts[1]/len(df):.1%})")

fig, ax = plt.subplots(figsize=(8, 5))
sns.countplot(x='Exited', data=df, ax=ax)
ax.set_xticklabels(['Retained (0)', 'Churned (1)'])
ax.set_title('Class Distribution: Customer Churn')
for p in ax.patches:
    ax.annotate(f'{p.get_height()} ({p.get_height()/len(df):.1%})',
                (p.get_x() + p.get_width() / 2., p.get_height()),
                ha='center', va='bottom')
plt.tight_layout()
plt.show()

What does this code do?

  • df['Exited'].mean() calculates the churn rate as a proportion. Expect roughly 20% churned and 80% retained.
  • sns.countplot() creates a bar chart showing how many customers fall into each class.
  • The for p in ax.patches loop annotates each bar with its exact count and percentage so you have precise numbers on the chart.

This ~80/20 split is a moderate class imbalance. It means any model trained naively would achieve 80% accuracy by always predicting "retained." Statistical tests in later steps will account for this.

  • Run the cell with Shift+Enter.

You should see printed churn statistics followed by a bar chart with two bars. The left bar (Retained) is roughly 4x taller than the right bar (Churned), with exact counts annotated above each.

  • Create a new cell and paste the following code to drop columns that would add noise or leak information:
# Cell 5: Drop Irrelevant Columns
df = df.drop(columns=['RowNumber', 'CustomerId', 'Surname'])
print(f"Columns after cleanup: {list(df.columns)}")

Why drop these columns?

RowNumber is just a row index. CustomerId and Surname are unique identifiers with no predictive value.

Keeping them could mislead correlation analysis into finding spurious patterns. Dropping them now keeps all downstream analysis clean.

  • Run the cell with Shift+Enter.

You should see a list of 11 remaining columns, all of which are meaningful features or the target variable.

✔️ Awesome, I've got everything!

Great. Double check you have saved your notebook (Ctrl+S or Cmd+S).

ⓧ I'd like to double check the full code

Here is the complete code across all 5 cells at this point in the project:

# Cell 1: Imports and Setup
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
from scipy import stats

sns.set_style('whitegrid')
plt.rcParams['figure.figsize'] = (12, 6)

# Cell 2: Load Data
df = pd.read_csv('Churn_Modelling.csv')
print(f"Dataset shape: {df.shape}")
print(f"\nColumn types:\n{df.dtypes}")
print(f"\nFirst 5 rows:")
df.head()

# Cell 3: Data Quality Check
print("=" * 50)
print("DATA QUALITY ASSESSMENT")
print("=" * 50)
print(f"\nMissing values:\n{df.isnull().sum()}")
print(f"\nDuplicate rows: {df.duplicated().sum()}")
print(f"\nBasic statistics:")
df.describe()

# Cell 4: Class Imbalance Analysis
churn_counts = df['Exited'].value_counts()
churn_rate = df['Exited'].mean()
print(f"Churn rate: {churn_rate:.1%}")
print(f"Retained: {churn_counts[0]} ({churn_counts[0]/len(df):.1%})")
print(f"Churned: {churn_counts[1]} ({churn_counts[1]/len(df):.1%})")

fig, ax = plt.subplots(figsize=(8, 5))
sns.countplot(x='Exited', data=df, ax=ax)
ax.set_xticklabels(['Retained (0)', 'Churned (1)'])
ax.set_title('Class Distribution: Customer Churn')
for p in ax.patches:
    ax.annotate(f'{p.get_height()} ({p.get_height()/len(df):.1%})',
                (p.get_x() + p.get_width() / 2., p.get_height()),
                ha='center', va='bottom')
plt.tight_layout()
plt.show()

# Cell 5: Drop Irrelevant Columns
df = df.drop(columns=['RowNumber', 'CustomerId', 'Surname'])
print(f"Columns after cleanup: {list(df.columns)}")

You now have a clean, inspected dataset with 11 meaningful columns and a clear picture of the ~20% churn rate. Next up, you will dig into how each feature relates to churn through univariate and bivariate analysis.

Analyze Feature Distributions and Detect Outliers

Your dataset is clean, trimmed to 11 meaningful columns, and you know the target class is imbalanced at roughly 80/20. The next question is: which features actually differ between customers who churn and those who stay?

In this step, you will visualize how numeric and categorical features distribute across churned vs retained groups. You will also detect outliers using the IQR method and check whether those extreme values correlate with churn.

In this step, get ready to:
  • Plot numeric feature distributions split by churn status.
  • Visualize churn rates across categorical features.
  • Detect outliers using the IQR method and analyze their relationship to churn.
Plot numeric feature distributions by churn status

A histogram overlaid with a KDE (kernel density estimate) curve reveals where churned and retained customers cluster differently along each numeric axis. If the two curves separate cleanly, that feature likely carries predictive power.

  • Create a new cell in your notebook (Cell 6) and paste the following code to generate density histograms for all six numeric features:
# Cell 6: Numeric Feature Distributions by Churn Status
numeric_features = ['CreditScore', 'Age', 'Tenure', 'Balance', 'NumOfProducts', 'EstimatedSalary']

fig, axes = plt.subplots(2, 3, figsize=(18, 10))
axes = axes.flatten()

for i, feature in enumerate(numeric_features):
    sns.histplot(data=df, x=feature, hue='Exited', kde=True, ax=axes[i], stat='density')
    axes[i].set_title(f'{feature} Distribution by Churn Status')
    axes[i].legend(labels=['Retained', 'Churned'])

plt.tight_layout()
plt.show()

What does this code do?

The numeric_features list holds the six columns you want to compare across churn groups.

For each feature, sns.histplot() draws overlapping density histograms for retained (0) and churned (1) customers. The kde=True parameter adds a smooth curve that makes it easier to spot distribution shape differences.

  • Run the cell by pressing Shift+Enter.
  • You should see a 2x3 grid of histograms. Look at the Age plot specifically. Churned customers (orange) concentrate heavily in the 40-60 range, while retained customers skew younger.

What patterns should I look for?

Strong separation between the two curves means the feature behaves differently for churned customers. Age shows the most obvious split. Balance shows a bimodal distribution for churned customers (peaks at zero and at high balances).

Features where the curves overlap almost completely (like CreditScore or EstimatedSalary) are weaker predictors of churn on their own.

Seeing an empty plot or error?

If you get a KeyError, make sure you ran Cell 5 (the column drop cell) before this one. The numeric_features list references columns that must still exist in df.

If the legend labels appear as 0/1 instead of Retained/Churned, that is cosmetic and does not affect your analysis.

Help me fix my numeric distribution plots

Visualize churn rates for categorical features

Categorical features need a different visualization approach. Instead of distributions, you want to see the churn rate (percentage who churned) within each category.

  • Create a new cell (Cell 7) and paste this code to generate churn rate bar plots for all four categorical features:
# Cell 7: Categorical Feature Churn Rates
categorical_features = ['Geography', 'Gender', 'HasCrCard', 'IsActiveMember']

fig, axes = plt.subplots(1, 4, figsize=(20, 5))

for i, feature in enumerate(categorical_features):
    churn_by_cat = df.groupby(feature)['Exited'].mean().reset_index()
    sns.barplot(x=feature, y='Exited', data=churn_by_cat, ax=axes[i])
    axes[i].set_title(f'Churn Rate by {feature}')
    axes[i].set_ylabel('Churn Rate')
    for p in axes[i].patches:
        axes[i].annotate(f'{p.get_height():.2%}',
                        (p.get_x() + p.get_width() / 2., p.get_height()),
                        ha='center', va='bottom', fontsize=9)

plt.tight_layout()
plt.show()

What does this code do?

The .groupby(feature)['Exited'].mean() call calculates the average of the binary Exited column within each category. Since Exited is 0 or 1, the mean equals the churn rate for that group.

The annotation loop adds exact percentage labels above each bar so you can compare groups without eyeballing the y-axis.

  • Run the cell with Shift+Enter.
  • You should see four bar charts side by side. Germany shows roughly 32% churn compared to France at roughly 16%. Inactive members churn at approximately double the rate of active members.

Why is HasCrCard interesting here?

You will likely notice that HasCrCard shows almost no difference in churn rate between groups. This is a useful finding. It tells you that simply having a credit card does not predict whether a customer will leave.

Bar plot labels overlapping or cut off?

If annotations overlap the bars or get clipped at the top, ensure plt.tight_layout() is the last line before plt.show(). You can also increase the figure height from 5 to 6 in the figsize tuple.

Help me fix overlapping annotations on my bar plots

Detect outliers using the IQR method

Outliers are data points that fall far outside the typical range for a feature. The IQR (interquartile range) method flags any value below Q1 minus 1.5 times the IQR or above Q3 plus 1.5 times the IQR.

The key question is whether these extreme values are random noise or whether they cluster among churned customers. If outliers churn at a higher rate than the overall population, they represent a specific customer profile worth investigating.

  • Create a new cell (Cell 8) and paste the following code to compute IQR-based outliers and visualize them with churn-aware box plots:
# Cell 8: Outlier Detection with IQR Method
print("=" * 50)
print("OUTLIER DETECTION (IQR Method)")
print("=" * 50)

outlier_summary = []
fig, axes = plt.subplots(2, 3, figsize=(18, 10))
axes = axes.flatten()

for i, feature in enumerate(numeric_features):
    Q1 = df[feature].quantile(0.25)
    Q3 = df[feature].quantile(0.75)
    IQR = Q3 - Q1
    lower_bound = Q1 - 1.5 * IQR
    upper_bound = Q3 + 1.5 * IQR
    outliers = df[(df[feature] < lower_bound) | (df[feature] > upper_bound)]
    outlier_summary.append({
        'Feature': feature,
        'Outlier Count': len(outliers),
        'Outlier %': f"{len(outliers)/len(df):.2%}",
        'Outlier Churn Rate': f"{outliers['Exited'].mean():.2%}" if len(outliers) > 0 else 'N/A'
    })
    sns.boxplot(x='Exited', y=feature, data=df, ax=axes[i])
    axes[i].set_title(f'{feature} - Outliers by Churn Status')
    axes[i].set_xticklabels(['Retained', 'Churned'])

plt.tight_layout()
plt.show()

outlier_df = pd.DataFrame(outlier_summary)
print("\nOutlier Summary:")
print(outlier_df.to_string(index=False))

What does this code do?

For each numeric feature, the code calculates Q1 (25th percentile), Q3 (75th percentile), and the IQR (Q3 minus Q1). Any row where the feature value falls below Q1 - 1.5 * IQR or above Q3 + 1.5 * IQR is flagged as an outlier.

The outlier_summary list collects how many outliers each feature has and what their churn rate is. The box plots let you visually compare the spread and outliers between retained and churned groups.

  • Run the cell with Shift+Enter.
  • You should see a 2x3 grid of box plots and a printed outlier summary table below them. The table shows outlier count, percentage, and churn rate for each feature.

How do I read the outlier summary?

Compare each feature's outlier churn rate to the overall churn rate of roughly 20%. If outliers for a particular feature churn at a much higher rate, those extreme values are a genuine signal, not just noise to discard.

Getting a NameError on numeric_features?

Cell 8 references the numeric_features list defined in Cell 6. If you skipped Cell 6 or restarted your kernel, run Cell 6 first to define that list before running Cell 8.

Help me debug my outlier detection cell

✔️ Awesome, I've got everything!

Great. Double check you have saved your notebook with all three new cells (6, 7, and 8) producing output.

ⓧ I'd like to double check the full code

Here is the complete notebook up to this point. Compare your cells against this to catch any differences.

# Cell 1: Imports and Setup
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
from scipy import stats

sns.set_style('whitegrid')
plt.rcParams['figure.figsize'] = (12, 6)

# Cell 2: Load Data
df = pd.read_csv('Churn_Modelling.csv')
print(f"Dataset shape: {df.shape}")
print(f"\nColumn types:\n{df.dtypes}")
print(f"\nFirst 5 rows:")
df.head()

# Cell 3: Data Quality Check
print("=" * 50)
print("DATA QUALITY ASSESSMENT")
print("=" * 50)
print(f"\nMissing values:\n{df.isnull().sum()}")
print(f"\nDuplicate rows: {df.duplicated().sum()}")
print(f"\nBasic statistics:")
df.describe()

# Cell 4: Class Imbalance Analysis
churn_counts = df['Exited'].value_counts()
churn_rate = df['Exited'].mean()
print(f"Churn rate: {churn_rate:.1%}")
print(f"Retained: {churn_counts[0]} ({churn_counts[0]/len(df):.1%})")
print(f"Churned: {churn_counts[1]} ({churn_counts[1]/len(df):.1%})")

fig, ax = plt.subplots(figsize=(8, 5))
sns.countplot(x='Exited', data=df, ax=ax)
ax.set_xticklabels(['Retained (0)', 'Churned (1)'])
ax.set_title('Class Distribution: Customer Churn')
for p in ax.patches:
    ax.annotate(f'{p.get_height()} ({p.get_height()/len(df):.1%})',
                (p.get_x() + p.get_width() / 2., p.get_height()),
                ha='center', va='bottom')
plt.tight_layout()
plt.show()

# Cell 5: Drop Irrelevant Columns
df = df.drop(columns=['RowNumber', 'CustomerId', 'Surname'])
print(f"Columns after cleanup: {list(df.columns)}")

# Cell 6: Numeric Feature Distributions by Churn Status
numeric_features = ['CreditScore', 'Age', 'Tenure', 'Balance', 'NumOfProducts', 'EstimatedSalary']

fig, axes = plt.subplots(2, 3, figsize=(18, 10))
axes = axes.flatten()

for i, feature in enumerate(numeric_features):
    sns.histplot(data=df, x=feature, hue='Exited', kde=True, ax=axes[i], stat='density')
    axes[i].set_title(f'{feature} Distribution by Churn Status')
    axes[i].legend(labels=['Retained', 'Churned'])

plt.tight_layout()
plt.show()

# Cell 7: Categorical Feature Churn Rates
categorical_features = ['Geography', 'Gender', 'HasCrCard', 'IsActiveMember']

fig, axes = plt.subplots(1, 4, figsize=(20, 5))

for i, feature in enumerate(categorical_features):
    churn_by_cat = df.groupby(feature)['Exited'].mean().reset_index()
    sns.barplot(x=feature, y='Exited', data=churn_by_cat, ax=axes[i])
    axes[i].set_title(f'Churn Rate by {feature}')
    axes[i].set_ylabel('Churn Rate')
    for p in axes[i].patches:
        axes[i].annotate(f'{p.get_height():.2%}',
                        (p.get_x() + p.get_width() / 2., p.get_height()),
                        ha='center', va='bottom', fontsize=9)

plt.tight_layout()
plt.show()

# Cell 8: Outlier Detection with IQR Method
print("=" * 50)
print("OUTLIER DETECTION (IQR Method)")
print("=" * 50)

outlier_summary = []
fig, axes = plt.subplots(2, 3, figsize=(18, 10))
axes = axes.flatten()

for i, feature in enumerate(numeric_features):
    Q1 = df[feature].quantile(0.25)
    Q3 = df[feature].quantile(0.75)
    IQR = Q3 - Q1
    lower_bound = Q1 - 1.5 * IQR
    upper_bound = Q3 + 1.5 * IQR
    outliers = df[(df[feature] < lower_bound) | (df[feature] > upper_bound)]
    outlier_summary.append({
        'Feature': feature,
        'Outlier Count': len(outliers),
        'Outlier %': f"{len(outliers)/len(df):.2%}",
        'Outlier Churn Rate': f"{outliers['Exited'].mean():.2%}" if len(outliers) > 0 else 'N/A'
    })
    sns.boxplot(x='Exited', y=feature, data=df, ax=axes[i])
    axes[i].set_title(f'{feature} - Outliers by Churn Status')
    axes[i].set_xticklabels(['Retained', 'Churned'])

plt.tight_layout()
plt.show()

outlier_df = pd.DataFrame(outlier_summary)
print("\nOutlier Summary:")
print(outlier_df.to_string(index=False))

You now have a visual and quantitative understanding of how each feature behaves across churn groups. Next, you will put statistical rigour behind these observations using chi-square tests and point-biserial correlation to confirm which features are truly significant churn predictors.

Run Statistical Hypothesis Tests

In the previous step, you uncovered visual patterns in how features distribute across churned and retained customers. You spotted that older customers and German customers churn more frequently.

Visual patterns can be misleading. A bar chart might suggest a difference, but random noise could explain it just as well. In this step, you will apply statistical hypothesis tests to quantify which features have a real, statistically significant relationship with churn.

In this step, get ready to:
  • Run chi-square tests on categorical features to check for independence from churn.
  • Calculate point-biserial correlations to rank numeric features by churn association.
  • Visualize a full correlation heatmap to see relationships across all numeric columns.
Run chi-square tests for categorical features

A chi-square test of independence answers the question: is there a statistically significant relationship between a categorical feature and churn, or could the observed differences be due to chance? You will test each of the four categorical features against the Exited target.

  • In your Jupyter Notebook, create a new cell below Cell 8 and paste the following code:
# Cell 9: Chi-Square Tests for Categorical Features
print("=" * 50)
print("CHI-SQUARE TESTS OF INDEPENDENCE")
print("=" * 50)

chi2_results = []
for feature in categorical_features:
    contingency = pd.crosstab(df[feature], df['Exited'])
    chi2, p_value, dof, expected = stats.chi2_contingency(contingency)
    chi2_results.append({
        'Feature': feature,
        'Chi-Square': round(chi2, 4),
        'p-value': f"{p_value:.6f}",
        'Significant (p<0.05)': 'Yes' if p_value < 0.05 else 'No'
    })

chi2_df = pd.DataFrame(chi2_results)
print("\nChi-Square Test Results:")
print(chi2_df.to_string(index=False))

What does this code do?

For each categorical feature, the code builds a contingency table using pd.crosstab(). This table counts how many customers fall into each combination of category and churn status.

Then stats.chi2_contingency() runs the chi-square test and returns the test statistic plus a p-value. If the p-value is below 0.05, you can reject the null hypothesis that the feature and churn are independent.

  • Run the cell by pressing Shift+Enter.
  • You should see a table with four rows. Geography, Gender, and IsActiveMember show "Yes" in the significance column. HasCrCard shows "No".

Why is HasCrCard not significant?

The chi-square test confirms that simply having a credit card does not predict whether a customer leaves. The churn rate is nearly identical for card holders and non-holders. This matches what you saw in the bar chart from the previous step.

Seeing a NameError for categorical_features?

Make sure you ran all previous cells in order. The categorical_features list was defined in Cell 7. If you restarted your kernel, re-run all cells from the top by clicking Kernel then Restart & Run All.

Help me debug my chi-square test cell

Calculate point-biserial correlation for numeric features

Chi-square works for categorical features, but numeric features need a different approach. Point-biserial correlation measures the strength and direction of the linear relationship between a continuous variable and a binary variable (churned vs retained).

  • Create a new cell below Cell 9 and paste this code:
# Cell 10: Point-Biserial Correlation for Numeric Features
print("=" * 50)
print("POINT-BISERIAL CORRELATION WITH CHURN")
print("=" * 50)

correlation_results = []
for feature in numeric_features:
    corr, p_value = stats.pointbiserialr(df['Exited'], df[feature])
    correlation_results.append({
        'Feature': feature,
        'Correlation': round(corr, 4),
        'p-value': f"{p_value:.6f}",
        'Abs Correlation': round(abs(corr), 4)
    })

corr_df = pd.DataFrame(correlation_results).sort_values('Abs Correlation', ascending=False)
print("\nPoint-Biserial Correlation Results (ranked by strength):")
print(corr_df.to_string(index=False))

What does this code do?

For each numeric feature, stats.pointbiserialr() returns a correlation coefficient between -1 and 1. A positive value means the feature tends to be higher for churned customers. A negative value means the feature tends to be lower for churned customers.

The results are sorted by absolute correlation strength so the strongest churn predictors appear at the top of the table.

  • Run the cell with Shift+Enter.
  • You should see Age at the top with a positive correlation around 0.29. This confirms that older customers are more likely to churn.

Why point-biserial instead of Pearson?

Pearson correlation assumes both variables are continuous. Your target (Exited) is binary (0 or 1). Point-biserial is the mathematically correct version of Pearson for this situation. It produces the same numeric result but makes your methodology explicit.

All p-values showing 0.000000?

This is normal for a dataset with 10,000 rows. With this many observations, even weak correlations produce extremely small p-values. Focus on the correlation magnitude (the Abs Correlation column) to judge practical importance, not just statistical significance.

Help me interpret my correlation results

Create a correlation heatmap

Point-biserial correlation measures each feature against the target individually. A full correlation heatmap shows relationships between all numeric columns at once, helping you spot multicollinearity (features that are highly correlated with each other) in addition to churn drivers.

  • Create a new cell below Cell 10 and paste this code:
# Cell 11: Correlation Heatmap
numeric_df = df.select_dtypes(include=[np.number])
corr_matrix = numeric_df.corr()

fig, ax = plt.subplots(figsize=(12, 10))
mask = np.triu(np.ones_like(corr_matrix, dtype=bool))
sns.heatmap(corr_matrix, mask=mask, annot=True, cmap='coolwarm', center=0,
            fmt='.2f', square=True, linewidths=0.5, ax=ax)
ax.set_title('Correlation Heatmap - Bank Customer Churn Dataset')
plt.tight_layout()
plt.show()

What does this code do?

  • select_dtypes(include=[np.number]) filters down to only numeric columns so the correlation matrix can be computed.
  • The mask hides the upper triangle (which is a mirror of the lower triangle) to reduce visual clutter.
  • annot=True prints the actual correlation value inside each cell, and cmap='coolwarm' colours positive correlations red and negative correlations blue.
  • Run the cell with Shift+Enter.
  • You should see a colour-coded triangular heatmap. Look at the Exited row to confirm that Age has the strongest positive association with churn.

Heatmap looks squished or labels are cut off?

Make sure the figsize=(12, 10) is set correctly. If labels are still clipped, try adding plt.tight_layout() before plt.show() (it should already be there from the code above).

Help me fix my heatmap display

✔️ Awesome, I've got everything!

Great. Double check you have saved your notebook before moving on.

ⓧ I'd like to double check the full code

Here is the complete notebook code up to this point. Compare your cells against this:

# Cell 1: Imports and Setup
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
from scipy import stats

sns.set_style('whitegrid')
plt.rcParams['figure.figsize'] = (12, 6)

# Cell 2: Load Data
df = pd.read_csv('Churn_Modelling.csv')
print(f"Dataset shape: {df.shape}")
print(f"\nColumn types:\n{df.dtypes}")
print(f"\nFirst 5 rows:")
df.head()

# Cell 3: Data Quality Check
print("=" * 50)
print("DATA QUALITY ASSESSMENT")
print("=" * 50)
print(f"\nMissing values:\n{df.isnull().sum()}")
print(f"\nDuplicate rows: {df.duplicated().sum()}")
print(f"\nBasic statistics:")
df.describe()

# Cell 4: Class Imbalance Analysis
churn_counts = df['Exited'].value_counts()
churn_rate = df['Exited'].mean()
print(f"Churn rate: {churn_rate:.1%}")
print(f"Retained: {churn_counts[0]} ({churn_counts[0]/len(df):.1%})")
print(f"Churned: {churn_counts[1]} ({churn_counts[1]/len(df):.1%})")

fig, ax = plt.subplots(figsize=(8, 5))
sns.countplot(x='Exited', data=df, ax=ax)
ax.set_xticklabels(['Retained (0)', 'Churned (1)'])
ax.set_title('Class Distribution: Customer Churn')
for p in ax.patches:
    ax.annotate(f'{p.get_height()} ({p.get_height()/len(df):.1%})',
                (p.get_x() + p.get_width() / 2., p.get_height()),
                ha='center', va='bottom')
plt.tight_layout()
plt.show()

# Cell 5: Drop Irrelevant Columns
df = df.drop(columns=['RowNumber', 'CustomerId', 'Surname'])
print(f"Columns after cleanup: {list(df.columns)}")

# Cell 6: Numeric Feature Distributions by Churn Status
numeric_features = ['CreditScore', 'Age', 'Tenure', 'Balance', 'NumOfProducts', 'EstimatedSalary']

fig, axes = plt.subplots(2, 3, figsize=(18, 10))
axes = axes.flatten()

for i, feature in enumerate(numeric_features):
    sns.histplot(data=df, x=feature, hue='Exited', kde=True, ax=axes[i], stat='density')
    axes[i].set_title(f'{feature} Distribution by Churn Status')
    axes[i].legend(labels=['Retained', 'Churned'])

plt.tight_layout()
plt.show()

# Cell 7: Categorical Feature Churn Rates
categorical_features = ['Geography', 'Gender', 'HasCrCard', 'IsActiveMember']

fig, axes = plt.subplots(1, 4, figsize=(20, 5))

for i, feature in enumerate(categorical_features):
    churn_by_cat = df.groupby(feature)['Exited'].mean().reset_index()
    sns.barplot(x=feature, y='Exited', data=churn_by_cat, ax=axes[i])
    axes[i].set_title(f'Churn Rate by {feature}')
    axes[i].set_ylabel('Churn Rate')
    for p in axes[i].patches:
        axes[i].annotate(f'{p.get_height():.2%}',
                        (p.get_x() + p.get_width() / 2., p.get_height()),
                        ha='center', va='bottom', fontsize=9)

plt.tight_layout()
plt.show()

# Cell 8: Outlier Detection with IQR Method
print("=" * 50)
print("OUTLIER DETECTION (IQR Method)")
print("=" * 50)

outlier_summary = []
fig, axes = plt.subplots(2, 3, figsize=(18, 10))
axes = axes.flatten()

for i, feature in enumerate(numeric_features):
    Q1 = df[feature].quantile(0.25)
    Q3 = df[feature].quantile(0.75)
    IQR = Q3 - Q1
    lower_bound = Q1 - 1.5 * IQR
    upper_bound = Q3 + 1.5 * IQR
    outliers = df[(df[feature] < lower_bound) | (df[feature] > upper_bound)]
    outlier_summary.append({
        'Feature': feature,
        'Outlier Count': len(outliers),
        'Outlier %': f"{len(outliers)/len(df):.2%}",
        'Outlier Churn Rate': f"{outliers['Exited'].mean():.2%}" if len(outliers) > 0 else 'N/A'
    })
    sns.boxplot(x='Exited', y=feature, data=df, ax=axes[i])
    axes[i].set_title(f'{feature} - Outliers by Churn Status')
    axes[i].set_xticklabels(['Retained', 'Churned'])

plt.tight_layout()
plt.show()

outlier_df = pd.DataFrame(outlier_summary)
print("\nOutlier Summary:")
print(outlier_df.to_string(index=False))

# Cell 9: Chi-Square Tests for Categorical Features
print("=" * 50)
print("CHI-SQUARE TESTS OF INDEPENDENCE")
print("=" * 50)

chi2_results = []
for feature in categorical_features:
    contingency = pd.crosstab(df[feature], df['Exited'])
    chi2, p_value, dof, expected = stats.chi2_contingency(contingency)
    chi2_results.append({
        'Feature': feature,
        'Chi-Square': round(chi2, 4),
        'p-value': f"{p_value:.6f}",
        'Significant (p<0.05)': 'Yes' if p_value < 0.05 else 'No'
    })

chi2_df = pd.DataFrame(chi2_results)
print("\nChi-Square Test Results:")
print(chi2_df.to_string(index=False))

# Cell 10: Point-Biserial Correlation for Numeric Features
print("=" * 50)
print("POINT-BISERIAL CORRELATION WITH CHURN")
print("=" * 50)

correlation_results = []
for feature in numeric_features:
    corr, p_value = stats.pointbiserialr(df['Exited'], df[feature])
    correlation_results.append({
        'Feature': feature,
        'Correlation': round(corr, 4),
        'p-value': f"{p_value:.6f}",
        'Abs Correlation': round(abs(corr), 4)
    })

corr_df = pd.DataFrame(correlation_results).sort_values('Abs Correlation', ascending=False)
print("\nPoint-Biserial Correlation Results (ranked by strength):")
print(corr_df.to_string(index=False))

# Cell 11: Correlation Heatmap
numeric_df = df.select_dtypes(include=[np.number])
corr_matrix = numeric_df.corr()

fig, ax = plt.subplots(figsize=(12, 10))
mask = np.triu(np.ones_like(corr_matrix, dtype=bool))
sns.heatmap(corr_matrix, mask=mask, annot=True, cmap='coolwarm', center=0,
            fmt='.2f', square=True, linewidths=0.5, ax=ax)
ax.set_title('Correlation Heatmap - Bank Customer Churn Dataset')
plt.tight_layout()
plt.show()

You now have statistical proof backing up the visual patterns from your earlier analysis. Next, you will engineer new features that amplify these churn signals and produce a final ranked summary of churn drivers.

Engineer Features and Summarize Findings

Your statistical tests from the last step gave you hard evidence about which features matter. You know Age and NumOfProducts are the strongest correlators, and that Geography and IsActiveMember are statistically significant categorical predictors.

Raw features only tell part of the story. In this step, you will engineer new features that combine existing columns to capture richer churn signals. Then you will rank all features together and translate your statistical findings into actionable business recommendations.

In this step, get ready to:
  • Create four business-intuitive derived features (BalancePerProduct, TenurePerAge, IsHighBalance, AgeGroup).
  • Validate whether engineered features improve churn signal with point-biserial correlation and visualizations.
  • Produce a final ranked feature table and export the enriched dataset for downstream modelling.
Create business-intuitive derived features

Individual columns like Balance and NumOfProducts each carry some churn signal on their own. Combining them into ratios or categorical bins can surface patterns that raw columns miss.

  • Create a new cell in your notebook (Cell 12) and add the following feature engineering code:
# Cell 12: Feature Engineering
print("=" * 50)
print("FEATURE ENGINEERING")
print("=" * 50)

df['BalancePerProduct'] = df['Balance'] / (df['NumOfProducts'] + 1)
df['TenurePerAge'] = df['Tenure'] / df['Age']
df['IsHighBalance'] = (df['Balance'] > df['Balance'].median()).astype(int)
df['AgeGroup'] = pd.cut(df['Age'], bins=[17, 30, 40, 50, 60, 100],
                        labels=['18-30', '31-40', '41-50', '51-60', '60+'])

print("Engineered features created:")
print(f"  - BalancePerProduct: Balance / (NumOfProducts + 1)")
print(f"  - TenurePerAge: Tenure / Age")
print(f"  - IsHighBalance: 1 if Balance > median ({df['Balance'].median():.2f}), else 0")
print(f"  - AgeGroup: Age binned into 5 categories")

What does this code do?

Each engineered feature captures a different business intuition about churn risk:

  • BalancePerProduct divides Balance by the number of products (plus 1 to avoid division by zero). This measures wealth concentration per relationship.
  • TenurePerAge measures loyalty relative to the customer's lifetime. A young customer with high tenure is deeply embedded.
  • IsHighBalance is a binary flag marking customers above the median balance. Binary features often work better in downstream models.
  • AgeGroup uses pd.cut() to bin continuous ages into interpretable categories. This lets you see which age bracket churns most.
  • Run Cell 12.
  • You should see a confirmation listing all four engineered features with their definitions.

Seeing a KeyError?

If you see an error about a missing column like Balance or NumOfProducts, make sure you ran Cells 1 through 5 earlier. Cell 5 drops irrelevant columns but keeps the numeric ones you need here.

Help me debug this KeyError

Validate engineered features against churn

Creating features is only half the work. You need to verify they actually add predictive signal beyond what the original columns already provide.

  • Create a new cell (Cell 13) and add the following validation code:
# Cell 13: Validate Engineered Features
engineered_numeric = ['BalancePerProduct', 'TenurePerAge', 'IsHighBalance']

eng_corr_results = []
for feature in engineered_numeric:
    corr, p_value = stats.pointbiserialr(df['Exited'], df[feature])
    eng_corr_results.append({
        'Feature': feature,
        'Correlation': round(corr, 4),
        'p-value': f"{p_value:.6f}",
        'Abs Correlation': round(abs(corr), 4)
    })

eng_corr_df = pd.DataFrame(eng_corr_results).sort_values('Abs Correlation', ascending=False)
print("\nEngineered Feature Correlations with Churn:")
print(eng_corr_df.to_string(index=False))

# Visualize engineered features
fig, axes = plt.subplots(1, 3, figsize=(18, 5))
for i, feature in enumerate(engineered_numeric):
    sns.histplot(data=df, x=feature, hue='Exited', kde=True, ax=axes[i], stat='density')
    axes[i].set_title(f'{feature} Distribution by Churn Status')

plt.tight_layout()
plt.show()

# AgeGroup churn rate
age_churn = df.groupby('AgeGroup', observed=True)['Exited'].mean()
fig, ax = plt.subplots(figsize=(8, 5))
age_churn.plot(kind='bar', ax=ax, color='coral')
ax.set_title('Churn Rate by Age Group')
ax.set_ylabel('Churn Rate')
ax.set_xlabel('Age Group')
for i, v in enumerate(age_churn):
    ax.text(i, v + 0.01, f'{v:.1%}', ha='center', fontsize=10)
plt.tight_layout()
plt.show()

What does this code do?

This cell runs the same point-biserial correlation test you used in Step 4, but now on the engineered features. It then visualizes each engineered feature's distribution split by churn status.

The AgeGroup bar chart is especially useful. It groups all 10,000 customers into age bands and shows the churn rate for each. You should see the 41-50 and 51-60 brackets with dramatically higher churn than the younger groups.

  • Run Cell 13.
  • You should see a correlation table for the three engineered numeric features, followed by three density histograms and an AgeGroup bar chart with percentage annotations.

Bar chart showing no data?

If your AgeGroup bar chart is empty, confirm that Cell 12 ran successfully and created the AgeGroup column. Run df['AgeGroup'].value_counts() in a scratch cell to verify the column has values.

Help me fix my AgeGroup chart

Produce final rankings and business recommendations

Now you will combine both original and engineered features into one ranked table. This gives you a single view of which features matter most for predicting churn.

  • Create a new cell (Cell 14) and add the final ranking code:
# Cell 14: Final Summary and Rankings
print("=" * 50)
print("FINAL FEATURE RANKING (All Features vs Churn)")
print("=" * 50)

all_numeric_features = numeric_features + engineered_numeric
all_corr_results = []
for feature in all_numeric_features:
    corr, p_value = stats.pointbiserialr(df['Exited'], df[feature])
    all_corr_results.append({
        'Feature': feature,
        'Correlation': round(corr, 4),
        'Abs Correlation': round(abs(corr), 4),
        'Direction': 'Positive' if corr > 0 else 'Negative',
        'Significant': 'Yes' if p_value < 0.05 else 'No'
    })

final_ranking = pd.DataFrame(all_corr_results).sort_values('Abs Correlation', ascending=False)
print("\nFeature Importance Ranking (by correlation strength):")
print(final_ranking.to_string(index=False))

What does this code do?

This cell merges numeric_features (the 6 original columns) with engineered_numeric (the 3 new ones) into one list. It then runs point-biserial correlation on all 9 features and sorts the results by absolute correlation strength.

The Direction column tells you whether a feature increases or decreases churn risk. A positive direction means higher values correlate with more churn. Age should appear near the top with a positive direction.

  • Run Cell 14.
  • You should see a ranked table with all 9 features sorted from strongest to weakest churn predictor.
  • Create a new cell (Cell 15) and add the business recommendations:
# Cell 15: Business Recommendations
print("""
=====================================
BUSINESS RECOMMENDATIONS
=====================================

Based on this advanced EDA, the following retention strategies are recommended:

1. AGE-TARGETED CAMPAIGNS: Customers aged 40-60 show significantly higher churn rates.
   Develop loyalty programs specifically for this demographic segment.

2. GEOGRAPHIC FOCUS: Germany has approximately double the churn rate of France.
   Investigate regional service quality and competitive landscape in Germany.

3. ACTIVITY MONITORING: Inactive members churn at roughly 2x the rate of active members.
   Implement engagement triggers when activity drops below threshold.

4. PRODUCT STRATEGY: Customers with 3-4 products show extremely high churn (>80%).
   Review cross-selling practices to avoid customer overwhelm.

5. BALANCE-AWARE RETENTION: Customers with high balances who are inactive represent
   high-value churn risk. Prioritize personal outreach for this segment.
""")

Why write recommendations in a notebook?

In professional data science, your EDA is not complete until you translate statistical findings into plain-language recommendations. These five recommendations directly map to patterns you discovered: age separation (Cell 6), geographic churn rates (Cell 7), activity membership (Cell 9), and product count (Cell 7).

  • Run Cell 15 to print the recommendations.
  • You should see five numbered business strategies printed in your output.
  • Create a final cell (Cell 16) and add the export code:
# Cell 16: Export Cleaned Data
df.to_csv('churn_data_engineered.csv', index=False)
print("Engineered dataset exported to: churn_data_engineered.csv")
print(f"Final shape: {df.shape}")
print(f"Features: {list(df.columns)}")

What does this code do?

This exports your full DataFrame (now with 15 columns including the 4 engineered features) to a new CSV file. The index=False parameter prevents pandas from writing row numbers as an extra column.

This exported file is ready for a downstream modelling step. Any machine learning pipeline can load churn_data_engineered.csv and immediately use your engineered features.

  • Run Cell 16.
  • You should see the export confirmation showing a shape of (10000, 15) and the full column list.

Seeing a PermissionError on export?

If you get a permission error when writing churn_data_engineered.csv, make sure you do not have the file open in another program (like Excel). Close it and re-run the cell.

Help me fix this export error

✔️ Awesome, I've got everything!

Great. Double check you have saved your notebook with all 16 cells.

ⓧ I'd like to double check the full code

Here is the complete notebook code from Cell 1 through Cell 16:

# Cell 1: Imports and Setup
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
from scipy import stats

sns.set_style('whitegrid')
plt.rcParams['figure.figsize'] = (12, 6)

# Cell 2: Load Data
df = pd.read_csv('Churn_Modelling.csv')
print(f"Dataset shape: {df.shape}")
print(f"\nColumn types:\n{df.dtypes}")
print(f"\nFirst 5 rows:")
df.head()

# Cell 3: Data Quality Check
print("=" * 50)
print("DATA QUALITY ASSESSMENT")
print("=" * 50)
print(f"\nMissing values:\n{df.isnull().sum()}")
print(f"\nDuplicate rows: {df.duplicated().sum()}")
print(f"\nBasic statistics:")
df.describe()

# Cell 4: Class Imbalance Analysis
churn_counts = df['Exited'].value_counts()
churn_rate = df['Exited'].mean()
print(f"Churn rate: {churn_rate:.1%}")
print(f"Retained: {churn_counts[0]} ({churn_counts[0]/len(df):.1%})")
print(f"Churned: {churn_counts[1]} ({churn_counts[1]/len(df):.1%})")

fig, ax = plt.subplots(figsize=(8, 5))
sns.countplot(x='Exited', data=df, ax=ax)
ax.set_xticklabels(['Retained (0)', 'Churned (1)'])
ax.set_title('Class Distribution: Customer Churn')
for p in ax.patches:
    ax.annotate(f'{p.get_height()} ({p.get_height()/len(df):.1%})',
                (p.get_x() + p.get_width() / 2., p.get_height()),
                ha='center', va='bottom')
plt.tight_layout()
plt.show()

# Cell 5: Drop Irrelevant Columns
df = df.drop(columns=['RowNumber', 'CustomerId', 'Surname'])
print(f"Columns after cleanup: {list(df.columns)}")

# Cell 6: Numeric Feature Distributions by Churn Status
numeric_features = ['CreditScore', 'Age', 'Tenure', 'Balance', 'NumOfProducts', 'EstimatedSalary']

fig, axes = plt.subplots(2, 3, figsize=(18, 10))
axes = axes.flatten()

for i, feature in enumerate(numeric_features):
    sns.histplot(data=df, x=feature, hue='Exited', kde=True, ax=axes[i], stat='density')
    axes[i].set_title(f'{feature} Distribution by Churn Status')
    axes[i].legend(labels=['Retained', 'Churned'])

plt.tight_layout()
plt.show()

# Cell 7: Categorical Feature Churn Rates
categorical_features = ['Geography', 'Gender', 'HasCrCard', 'IsActiveMember']

fig, axes = plt.subplots(1, 4, figsize=(20, 5))

for i, feature in enumerate(categorical_features):
    churn_by_cat = df.groupby(feature)['Exited'].mean().reset_index()
    sns.barplot(x=feature, y='Exited', data=churn_by_cat, ax=axes[i])
    axes[i].set_title(f'Churn Rate by {feature}')
    axes[i].set_ylabel('Churn Rate')
    for p in axes[i].patches:
        axes[i].annotate(f'{p.get_height():.2%}',
                        (p.get_x() + p.get_width() / 2., p.get_height()),
                        ha='center', va='bottom', fontsize=9)

plt.tight_layout()
plt.show()

# Cell 8: Outlier Detection with IQR Method
print("=" * 50)
print("OUTLIER DETECTION (IQR Method)")
print("=" * 50)

outlier_summary = []
fig, axes = plt.subplots(2, 3, figsize=(18, 10))
axes = axes.flatten()

for i, feature in enumerate(numeric_features):
    Q1 = df[feature].quantile(0.25)
    Q3 = df[feature].quantile(0.75)
    IQR = Q3 - Q1
    lower_bound = Q1 - 1.5 * IQR
    upper_bound = Q3 + 1.5 * IQR
    outliers = df[(df[feature] < lower_bound) | (df[feature] > upper_bound)]
    outlier_summary.append({
        'Feature': feature,
        'Outlier Count': len(outliers),
        'Outlier %': f"{len(outliers)/len(df):.2%}",
        'Outlier Churn Rate': f"{outliers['Exited'].mean():.2%}" if len(outliers) > 0 else 'N/A'
    })
    sns.boxplot(x='Exited', y=feature, data=df, ax=axes[i])
    axes[i].set_title(f'{feature} - Outliers by Churn Status')
    axes[i].set_xticklabels(['Retained', 'Churned'])

plt.tight_layout()
plt.show()

outlier_df = pd.DataFrame(outlier_summary)
print("\nOutlier Summary:")
print(outlier_df.to_string(index=False))

# Cell 9: Chi-Square Tests for Categorical Features
print("=" * 50)
print("CHI-SQUARE TESTS OF INDEPENDENCE")
print("=" * 50)

chi2_results = []
for feature in categorical_features:
    contingency = pd.crosstab(df[feature], df['Exited'])
    chi2, p_value, dof, expected = stats.chi2_contingency(contingency)
    chi2_results.append({
        'Feature': feature,
        'Chi-Square': round(chi2, 4),
        'p-value': f"{p_value:.6f}",
        'Significant (p<0.05)': 'Yes' if p_value < 0.05 else 'No'
    })

chi2_df = pd.DataFrame(chi2_results)
print("\nChi-Square Test Results:")
print(chi2_df.to_string(index=False))

# Cell 10: Point-Biserial Correlation for Numeric Features
print("=" * 50)
print("POINT-BISERIAL CORRELATION WITH CHURN")
print("=" * 50)

correlation_results = []
for feature in numeric_features:
    corr, p_value = stats.pointbiserialr(df['Exited'], df[feature])
    correlation_results.append({
        'Feature': feature,
        'Correlation': round(corr, 4),
        'p-value': f"{p_value:.6f}",
        'Abs Correlation': round(abs(corr), 4)
    })

corr_df = pd.DataFrame(correlation_results).sort_values('Abs Correlation', ascending=False)
print("\nPoint-Biserial Correlation Results (ranked by strength):")
print(corr_df.to_string(index=False))

# Cell 11: Correlation Heatmap
numeric_df = df.select_dtypes(include=[np.number])
corr_matrix = numeric_df.corr()

fig, ax = plt.subplots(figsize=(12, 10))
mask = np.triu(np.ones_like(corr_matrix, dtype=bool))
sns.heatmap(corr_matrix, mask=mask, annot=True, cmap='coolwarm', center=0,
            fmt='.2f', square=True, linewidths=0.5, ax=ax)
ax.set_title('Correlation Heatmap - Bank Customer Churn Dataset')
plt.tight_layout()
plt.show()

# Cell 12: Feature Engineering
print("=" * 50)
print("FEATURE ENGINEERING")
print("=" * 50)

df['BalancePerProduct'] = df['Balance'] / (df['NumOfProducts'] + 1)
df['TenurePerAge'] = df['Tenure'] / df['Age']
df['IsHighBalance'] = (df['Balance'] > df['Balance'].median()).astype(int)
df['AgeGroup'] = pd.cut(df['Age'], bins=[17, 30, 40, 50, 60, 100],
                        labels=['18-30', '31-40', '41-50', '51-60', '60+'])

print("Engineered features created:")
print(f"  - BalancePerProduct: Balance / (NumOfProducts + 1)")
print(f"  - TenurePerAge: Tenure / Age")
print(f"  - IsHighBalance: 1 if Balance > median ({df['Balance'].median():.2f}), else 0")
print(f"  - AgeGroup: Age binned into 5 categories")

# Cell 13: Validate Engineered Features
engineered_numeric = ['BalancePerProduct', 'TenurePerAge', 'IsHighBalance']

eng_corr_results = []
for feature in engineered_numeric:
    corr, p_value = stats.pointbiserialr(df['Exited'], df[feature])
    eng_corr_results.append({
        'Feature': feature,
        'Correlation': round(corr, 4),
        'p-value': f"{p_value:.6f}",
        'Abs Correlation': round(abs(corr), 4)
    })

eng_corr_df = pd.DataFrame(eng_corr_results).sort_values('Abs Correlation', ascending=False)
print("\nEngineered Feature Correlations with Churn:")
print(eng_corr_df.to_string(index=False))

# Visualize engineered features
fig, axes = plt.subplots(1, 3, figsize=(18, 5))
for i, feature in enumerate(engineered_numeric):
    sns.histplot(data=df, x=feature, hue='Exited', kde=True, ax=axes[i], stat='density')
    axes[i].set_title(f'{feature} Distribution by Churn Status')

plt.tight_layout()
plt.show()

# AgeGroup churn rate
age_churn = df.groupby('AgeGroup', observed=True)['Exited'].mean()
fig, ax = plt.subplots(figsize=(8, 5))
age_churn.plot(kind='bar', ax=ax, color='coral')
ax.set_title('Churn Rate by Age Group')
ax.set_ylabel('Churn Rate')
ax.set_xlabel('Age Group')
for i, v in enumerate(age_churn):
    ax.text(i, v + 0.01, f'{v:.1%}', ha='center', fontsize=10)
plt.tight_layout()
plt.show()

# Cell 14: Final Summary and Rankings
print("=" * 50)
print("FINAL FEATURE RANKING (All Features vs Churn)")
print("=" * 50)

all_numeric_features = numeric_features + engineered_numeric
all_corr_results = []
for feature in all_numeric_features:
    corr, p_value = stats.pointbiserialr(df['Exited'], df[feature])
    all_corr_results.append({
        'Feature': feature,
        'Correlation': round(corr, 4),
        'Abs Correlation': round(abs(corr), 4),
        'Direction': 'Positive' if corr > 0 else 'Negative',
        'Significant': 'Yes' if p_value < 0.05 else 'No'
    })

final_ranking = pd.DataFrame(all_corr_results).sort_values('Abs Correlation', ascending=False)
print("\nFeature Importance Ranking (by correlation strength):")
print(final_ranking.to_string(index=False))

# Cell 15: Business Recommendations
print("""
=====================================
BUSINESS RECOMMENDATIONS
=====================================

Based on this advanced EDA, the following retention strategies are recommended:

1. AGE-TARGETED CAMPAIGNS: Customers aged 40-60 show significantly higher churn rates.
   Develop loyalty programs specifically for this demographic segment.

2. GEOGRAPHIC FOCUS: Germany has approximately double the churn rate of France.
   Investigate regional service quality and competitive landscape in Germany.

3. ACTIVITY MONITORING: Inactive members churn at roughly 2x the rate of active members.
   Implement engagement triggers when activity drops below threshold.

4. PRODUCT STRATEGY: Customers with 3-4 products show extremely high churn (>80%).
   Review cross-selling practices to avoid customer overwhelm.

5. BALANCE-AWARE RETENTION: Customers with high balances who are inactive represent
   high-value churn risk. Prioritize personal outreach for this segment.
""")

# Cell 16: Export Cleaned Data
df.to_csv('churn_data_engineered.csv', index=False)
print("Engineered dataset exported to: churn_data_engineered.csv")
print(f"Final shape: {df.shape}")
print(f"Features: {list(df.columns)}")

Your EDA notebook is complete. You have gone from raw data to statistically validated churn drivers and actionable business recommendations, all documented in one reproducible notebook.

Secret mission

SHAP-Based Feature Importance Validation

Your EDA ranked features by correlation strength. But correlation only captures linear relationships. What if a feature matters through non-linear interactions that point-biserial correlation misses entirely? Validate your statistical rankings against model-driven SHAP explanations to find out.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Project folder containing churn_eda.ipynb, Churn_Modelling.csv, and churn_data_engineered.csv.
  • Installed Python packages: pandas, numpy, seaborn, scipy, matplotlib, notebook, shap, scikit-learn.

Keep everything running

No action needed. Choose this if you are still actively building or want to revisit the analysis later.

  • Your project folder, notebook, and CSV files remain on your Desktop.
  • Installed Python packages stay available for future projects.
  • If you left the Jupyter server running, it continues to use a small amount of memory until you stop it.

Pause - I'll come back to this later

Shut down the Jupyter server to free up memory. Your files and packages stay intact.

  • Switch to the terminal where Jupyter Notebook is running.
  • Press Ctrl+C to stop the server. Confirm with y if prompted.
  • Your notebook and data files remain saved on disk. Run jupyter notebook again from the project folder when you are ready to resume.

Delete - I don't want to use this again

Remove all project files and uninstall the packages you installed for this project.

  • Stop the Jupyter server by pressing Ctrl+C in the terminal where it is running.
  • Delete your project folder (macOS/Linux) by running:
cd ~/Desktop
rm -rf churn-eda
  • Delete your project folder (Windows) by running:
cd ~/Desktop
rmdir /s /q churn-eda
  • Uninstall the Python packages you installed for this project by running:
pip uninstall pandas numpy seaborn scipy matplotlib notebook shap scikit-learn -y

Nice Work!

Nice Work!

Nice work! You have built a comprehensive advanced EDA notebook that extracts actionable churn insights from raw banking data using statistical rigour.

You've learned how to:

  • Applied chi-square tests and point-biserial correlation to statistically validate which features drive customer churn across 10,000 bank records.
  • Engineered 4 derived features (BalancePerProduct, TenurePerAge, IsHighBalance, AgeGroup) that amplify churn signal beyond raw columns.
  • Produced ranked feature importance tables and actionable business recommendations ready to inform a bank's retention strategy.
  • Validated your EDA findings with SHAP-based model explanations, bridging statistical analysis with machine learning interpretability.

Ready to quiz yourself?