Build a Credit Risk Scorecard

Build a bank-style credit scorecard with WoE binning and logistic regression.

Introduction

30 Second Summary

Every time someone applies for a credit card or a loan, a scorecard decides whether they get approved. Banks don't guess. They use a points-based system that assigns a credit score to each applicant based on their characteristics.

In this project, you will build a production-style credit risk scorecard using the same methodology banks use for regulatory-compliant lending decisions. You'll transform raw applicant data into interpretable risk signals, fit a model, and convert it into a scorecard that outputs credit scores in the 300-850 range.

What You'll Build

You'll score a sample applicant, trace exactly how their age, credit amount, and employment map to points, and watch the total land in a familiar credit score range. Then you'll prove the scorecard actually works by running the same validation metrics a bank's model risk team would demand.

By the end of this project, you'll have:

  • A points-based scorecard table that maps each variable bin to points, readable by a non-technical credit committee.
  • A model validation report with industry-standard KS, Gini, and AUC metrics proving your scorecard separates good from bad borrowers.
  • A challenger model comparison pitting your logistic regression scorecard against an XGBoost model to demonstrate the interpretability vs. performance trade-off.
  • Secret Mission: Build a PSI monitoring report to detect population drift between your training and test populations.

Are there any prerequisites?

You'll need Python 3.10 and VS Code installed on your machine. Basic familiarity with pandas and scikit-learn (importing libraries, working with DataFrames) is helpful but not required.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters.

You're building a credit risk scorecard that predicts probability of default for applicants, using the same methodology banks use for regulatory-compliant lending decisions.

Set Up Your Scorecard Environment

Credit risk scorecards rely on specialized statistical techniques like Weight of Evidence binning and PDO scaling that standard data science libraries do not provide out of the box. You need a purpose-built library to handle these transformations the way a bank's credit risk team would.

In this step, you will create an isolated Python environment and install the scorecard-specific libraries that power the rest of this project. By the end, you will have a working Jupyter notebook that proves your environment is ready.

In this step, get ready to:
  • Create a project folder and virtual environment.
  • Install the credit risk and machine learning packages.
  • Create a Jupyter notebook and confirm scorecardpy loads successfully.
Create a project folder and virtual environment

A virtual environment keeps your project's packages isolated from your system Python. This prevents version conflicts with other projects on your machine.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal and press Enter to open it.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir credit-scorecard
cd credit-scorecard
  • Create and activate a virtual environment by running:

macOS

python3 -m venv venv
source venv/bin/activate

Windows

python -m venv venv
.\venv\Scripts\Activate.ps1

You should see (venv) appear at the beginning of your terminal prompt. This confirms the virtual environment is active.

Don't see (venv) in your prompt?

Make sure you ran the activation command (not just the creation command). On macOS, the command starts with source. On Windows PowerShell, use .\venv\Scripts\Activate.ps1.

If you see an execution policy error on Windows, run Set-ExecutionPolicy -Scope CurrentUser RemoteSigned first, then try activating again.

help me activate my Python virtual environment

Install the required packages

You need scorecardpy for WoE binning and scorecard generation, scikit-learn for logistic regression, and XGBoost for the challenger model comparison.

  • Install all project dependencies by running this command:
pip install scorecardpy==0.1.9.7 scikit-learn==1.5.2 xgboost==2.1.3 shap pandas matplotlib seaborn

This will take a minute or two to download and install all packages. You should see Successfully installed followed by a list of package names at the end of the output.

Why pin specific versions?

Pinning scorecardpy==0.1.9.7, scikit-learn==1.5.2, and xgboost==2.1.3 ensures your results match this project exactly. Library updates can change default parameters or function signatures, which would cause code to break without warning.

Seeing installation errors?

Make sure (venv) is visible in your prompt before running pip install. If you see permission errors, you are likely installing into your system Python instead of the virtual environment.

If a specific package fails, try running the install command again. Network timeouts occasionally cause partial failures on the first attempt.

help me fix my pip install errors for scorecardpy

Create your notebook and verify the setup

Now you will open VS Code, install the Jupyter extension, and run a test cell to confirm everything works together.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code and press Enter to open it.
  • In VS Code, click File in the top menu bar.
  • Click Open Folder.
  • Navigate to your Desktop, select the credit-scorecard folder, and click Open.
  • Open the Extensions view by pressing Cmd+Shift+X (macOS) or Ctrl+Shift+X (Windows).
  • Type Jupyter in the search box.
  • Click Install on the Jupyter extension by Microsoft.

What is the Jupyter extension?

The Jupyter extension lets you run .ipynb notebook files directly in VS Code. Each cell can be executed independently, which makes it perfect for exploratory data work where you want to see results as you go.

  • Create the notebook file by pressing Cmd+Shift+P (macOS) or Ctrl+Shift+P (Windows) to open the Command Palette.
  • Type Create: New Jupyter Notebook and select it from the list.
  • Save the file immediately by pressing Cmd+S (macOS) or Ctrl+S (Windows).
  • Name the file scorecard.ipynb and save it in the credit-scorecard folder.
  • Click Select Kernel in the top-right corner of the notebook.
  • Select Python Environments, then choose the venv interpreter from your credit-scorecard folder.

Selecting the right kernel

Look for the option that shows the path to your credit-scorecard/venv folder. This ensures the notebook uses the packages you just installed, not your system Python.

  • In the first cell of your notebook, paste the following code to test the installation:
import scorecardpy as sc
print(sc.__version__)
data = sc.germancredit()
print(data.shape)

What does this code do?

  • import scorecardpy as sc loads the scorecard library and gives it the short alias sc for convenience.
  • print(sc.__version__) confirms the exact version you installed.
  • sc.germancredit() loads the German Credit dataset directly from the package. This is the dataset you will use for the entire scorecard build.
  • print(data.shape) shows you how many rows and columns the dataset has.

Before you run this cell, what do you expect to see printed for the version number?

  • Run the cell by clicking the play button to the left of the cell, or by pressing Shift+Enter.

You should see 0.1.9.7 printed on the first line, followed by (1000, 21) on the second line. This confirms scorecardpy is installed correctly and the German Credit dataset loaded with 1000 rows and 21 columns.

Seeing a ModuleNotFoundError?

This means the notebook is not using your virtual environment. Click Select Kernel in the top-right corner and make sure you have selected the Python interpreter from your credit-scorecard/venv folder.

If you do not see the venv option, close VS Code, reopen it from the credit-scorecard folder, and check the kernel picker again.

help me fix ModuleNotFoundError for scorecardpy in my Jupyter notebook

  • As a final confirmation, verify the version in your terminal by running:
python -c "import scorecardpy; print(scorecardpy.__version__)"

You should see 0.1.9.7 printed. Your environment is fully set up.

✔️ I see 0.1.9.7 in both places

Your environment is ready to go. Save your notebook with Cmd+S (macOS) or Ctrl+S (Windows).

ⓧ I see a different version or an error

If the terminal shows a different version, your (venv) may not be active. Run source venv/bin/activate (macOS) or .\venv\Scripts\Activate.ps1 (Windows) and try again.

If you see an import error, reinstall with:

pip install scorecardpy==0.1.9.7

Your environment is ready. Next up, you will load the German Credit dataset and figure out which variables actually predict whether someone defaults on their loan.

Explore and Filter Variables by Predictive Power

Your environment is set up and ready to go. Now you need to decide which features in your dataset actually predict whether someone will default on a loan.

Not every column carries useful signal. A credit risk team would never dump all 20 raw features into a model. They use a metric called Information Value (IV) to measure how well each variable separates good borrowers from bad ones, then drop the noise before modeling.

In this step, get ready to:
  • Load the German Credit dataset and check class balance.
  • Filter variables by IV to keep only predictive features.
  • Split the filtered data into train and test sets.
Load the German Credit dataset and explore class balance

Before filtering, you need to understand what you are working with. The German Credit dataset has 1000 applicants, each labeled as good or bad credit risk. You need to check how balanced that label is.

  • In scorecard.ipynb, replace the contents of your existing first cell with the following code:
import scorecardpy as sc
import pandas as pd

# Load dataset
data = sc.germancredit()
print(data.shape)
print(data.dtypes)
print(data.isnull().sum())
print(data['creditability'].value_counts(normalize=True))

What does this code do?

  • sc.germancredit() loads the built-in German Credit dataset directly from the scorecardpy package.
  • data.shape tells you how many rows and columns you have (expect 1000 rows, 21 columns).
  • data.dtypes shows which features are categorical (object) and which are numerical (int/float).
  • value_counts(normalize=True) shows the percentage of good vs. bad borrowers so you can check for class imbalance.
  • Run the cell by pressing Shift+Enter.

You should see the dataset shape printed as (1000, 21), a list of column dtypes, zero missing values, and the target split showing roughly 70% good (1) and 30% bad (0) borrowers.

Not seeing output?

  • Make sure you replaced the old cell contents entirely. The old print(scorecardpy.__version__) line should be gone.
  • Confirm the Jupyter kernel is running. Look for a solid circle next to the kernel name in the top right of VS Code.
  • help me debug my scorecardpy data loading
Filter variables by predictive power

Now you will use Information Value to decide which variables to keep. IV measures how well a feature separates defaulters from non-defaulters. Variables with IV below 0.02 are essentially noise.

  • Add a new cell below the first one and paste the following code:
# Filter variables by IV
dt_f = sc.var_filter(data, y='creditability')
print(dt_f.shape)

# IV values
iv = sc.iv(data, y='creditability')
print(iv)

What does this code do?

sc.var_filter() automatically removes variables with IV below 0.02 (useless predictive power), missing rates above 95%, or identical-value rates above 95%. It returns a cleaned DataFrame with only the surviving features.

sc.iv() calculates the raw Information Value for every variable. This gives you visibility into exactly how predictive each feature is.

Before you run this cell, think about what you expect. You started with 21 columns (20 features plus the target). How many do you think will survive the IV filter?

  • Run the cell by pressing Shift+Enter.

You should see the filtered shape showing roughly 14-16 columns (12-15 features plus the target). The IV table lists each variable with its information value score, showing you which features passed and which were dropped.

What do the IV thresholds mean?

  • IV < 0.02: useless predictor. Drop it.
  • IV 0.02 to 0.1: weak predictor. Keep but watch.
  • IV 0.1 to 0.3: medium predictor. Solid candidate.
  • IV 0.3 to 0.5: strong predictor. Definitely keep.
  • IV > 0.5: suspicious. Investigate for data leakage.

Seeing a warning or error?

  • If you see a deprecation warning from pandas, you can safely ignore it. The function still works correctly.
  • If var_filter is not recognized, check that your first cell has import scorecardpy as sc and that you ran it successfully.
  • help me understand why my variable filter is failing
Split into train and test sets

You have your filtered dataset. Now you need to split it into training and test sets so you can build on one and validate on the other. A 70/30 split is standard for credit scorecard development.

  • Add a new cell below and paste the following code:
# Train/test split
dt_list = sc.split_df(dt_f, y='creditability', ratio=0.7, seed=42)
train = dt_list['train']
test = dt_list['test']
print(train['creditability'].value_counts(normalize=True))
print(test['creditability'].value_counts(normalize=True))

What does this code do?

  • sc.split_df() creates a stratified split, keeping the same ratio of good-to-bad borrowers in both sets.
  • ratio=0.7 puts 70% of the data in train and 30% in test.
  • seed=42 makes the split reproducible so you get consistent results every time.
  • Printing value_counts(normalize=True) for both sets confirms the stratification worked correctly.
  • Run the cell by pressing Shift+Enter.

Both sets should show approximately the same default rate (around 30% bad, 70% good). If they match closely, your stratified split is working correctly.

Default rates look very different?

  • Make sure you passed dt_f (the filtered dataset) to sc.split_df(), not the original data variable.
  • Check that y='creditability' is spelled exactly as shown. The column name is case-sensitive.
  • help me fix my train/test split

✔️ Awesome, I've got everything!

Great. Double check you have saved scorecard.ipynb with all three cells.

ⓧ I'd like to double check the full code

Here are the three cells your notebook should contain at this point.

Cell 1:

import scorecardpy as sc
import pandas as pd

# Load dataset
data = sc.germancredit()
print(data.shape)
print(data.dtypes)
print(data.isnull().sum())
print(data['creditability'].value_counts(normalize=True))

Cell 2:

# Filter variables by IV
dt_f = sc.var_filter(data, y='creditability')
print(dt_f.shape)

# IV values
iv = sc.iv(data, y='creditability')
print(iv)

Cell 3:

# Train/test split
dt_list = sc.split_df(dt_f, y='creditability', ratio=0.7, seed=42)
train = dt_list['train']
test = dt_list['test']
print(train['creditability'].value_counts(normalize=True))
print(test['creditability'].value_counts(normalize=True))

You started with 20+ raw features and narrowed them down to only the ones with real predictive power. Next up, you will transform these features into Weight of Evidence values that make them interpretable for a credit committee and optimal for logistic regression.

Build WoE Bins and Transform Features

Your filtered dataset has the variables that carry real predictive signal. But raw feature values like "credit amount = 4,500" or "age = 32" mean nothing to a credit committee reviewing your model.

Weight of Evidence (WoE) binning converts each variable into monotonic risk signals that regulators and credit officers can read and approve. In this step, you'll generate bins, refine them with business judgment, and transform your data into WoE-encoded features ready for modeling.

In this step, get ready to:
  • Generate optimal WoE bins for all retained features.
  • Manually adjust bins for one variable to demonstrate business judgment.
  • Transform train and test data into WoE-encoded values for logistic regression.
Generate optimal WoE bins

Before building bins, consolidate your exploratory cells from the previous step into a single clean setup cell. This keeps your notebook organized as it grows.

  • In scorecard.ipynb, select all three existing cells and delete them.
  • Add a new cell at the top and paste the following consolidated setup code:
import scorecardpy as sc
import pandas as pd

data = sc.germancredit()
dt_f = sc.var_filter(data, y='creditability')
dt_list = sc.split_df(dt_f, y='creditability', ratio=0.7, seed=42)
train = dt_list['train']
test = dt_list['test']

Why consolidate?

The exploratory prints confirmed your data was loaded and filtered correctly. Now that you know it works, a single setup cell keeps the notebook clean and re-runnable from top to bottom.

  • Run the consolidated cell to reload all variables into memory.
  • Add a new cell below and generate WoE bins by running this code:
# Generate WoE bins
bins = sc.woebin(dt_f, y='creditability')
print(bins['credit.amount'])

What does this code do?

`sc.woebin()` automatically finds optimal bin boundaries for every variable in your filtered dataset. It calculates the WoE and Information Value (IV) for each bin.

The output is a dictionary where each key is a variable name and each value is a DataFrame showing bin boundaries, event counts, non-event counts, WoE values, and IV contribution per bin.

You should see a table for credit.amount showing columns like bin, count, woe, and total_iv. Each row is a bin with its boundaries and risk characteristics.

Now visualize the bins to confirm they produce a monotonic risk trend. Before you run the next cell, predict: do you expect the bad probability to rise or fall as credit amount increases?

  • Add a new cell and plot the WoE bins for credit.amount by running:
# Plot WoE bins for credit.amount
sc.woebin_plot(bins['credit.amount'])

You should see a chart with stacked bars showing the bin count distribution on the left axis, and a blue line showing the bad probability on the right axis. The blue line should trend upward as credit amount increases, confirming higher loan amounts carry higher default risk.

Not seeing a plot?

Make sure you passed bins['credit.amount'] (a single variable's bin DataFrame), not the entire bins dictionary. The function expects the binning result for one variable at a time.

If the plot appears but is too small to read, resize your Jupyter output area or add import matplotlib.pyplot as plt and plt.show() in a cell to force rendering.

Still stuck? Help me debug why my woebin_plot isn't rendering in Jupyter.

Manually adjust bins for age.in.years

Automated binning is a starting point. In practice, credit risk teams override statistical boundaries with domain expertise. A 26-year-old and a 40-year-old have fundamentally different risk profiles that business policy should reflect.

  • Add a new cell and override the automatic bins for age.in.years with custom cut-points by running:
# Manually adjust bins for age.in.years
breaks_adj = {'age.in.years': [26, 35, 40]}
bins_adj = sc.woebin(dt_f, y='creditability', breaks_list=breaks_adj)
print(bins_adj['age.in.years'])

What does this code do?

The breaks_adj dictionary specifies custom bin boundaries: applicants under 26, 26-35, 35-40, and over 40. These reflect life-stage risk differences that a credit committee would recognize.

Passing breaks_list=breaks_adj to sc.woebin() reruns the full binning process but uses your custom boundaries for age.in.years while keeping the automatic bins for all other variables. The result is stored in bins_adj which you'll use for the rest of the project.

You should see a table with four bins for age.in.years reflecting your custom boundaries. The WoE values should show younger applicants carrying higher risk (negative WoE) and older applicants carrying lower risk (positive WoE).

Seeing unexpected bin boundaries?

Check that your breaks_adj dictionary uses the exact variable name 'age.in.years' with dots, not underscores. The column names in the German Credit dataset use dots as separators.

Need help? Why are my manual bins not applying correctly in scorecardpy?

Transform train and test to WoE values

With your final bins locked in, you need to convert every raw feature value into its corresponding WoE number. This transforms the dataset from mixed types (categorical strings, numerical ranges) into a uniform set of continuous values that logistic regression can consume directly.

  • Add a new cell and transform both datasets by running:
# Transform train and test to WoE values
train_woe = sc.woebin_ply(train, bins_adj)
test_woe = sc.woebin_ply(test, bins_adj)
print(train_woe.head())
print(test_woe.head())

What does this code do?

`sc.woebin_ply()` looks up which bin each observation falls into and replaces the raw value with that bin's WoE score. The result is a DataFrame where every feature column now contains continuous WoE values instead of raw categories or numbers.

Notice the column names now end in _woe (e.g. credit.amount_woe). The target column creditability remains unchanged.

You should see a DataFrame where every feature column contains decimal WoE values. Positive values indicate lower risk. Negative values indicate higher risk.

Seeing an error about missing columns?

Make sure you're passing bins_adj (the manually adjusted bins) to sc.woebin_ply(), not the original bins variable. The adjusted version contains all variables including your custom age bins.

Still stuck? Help me troubleshoot why woebin_ply is failing on my train or test data.

✔️ Awesome, I've got everything!

Great. Double check you've saved your notebook file before moving on.

ⓧ I'd like to double check the full code

Here is the complete notebook at this point. Each section below represents one cell:

# Cell 1: Setup
import scorecardpy as sc
import pandas as pd

data = sc.germancredit()
dt_f = sc.var_filter(data, y='creditability')
dt_list = sc.split_df(dt_f, y='creditability', ratio=0.7, seed=42)
train = dt_list['train']
test = dt_list['test']
# Cell 2: Generate WoE bins
bins = sc.woebin(dt_f, y='creditability')
print(bins['credit.amount'])
# Cell 3: Plot WoE bins for credit.amount
sc.woebin_plot(bins['credit.amount'])
# Cell 4: Manually adjust bins for age.in.years
breaks_adj = {'age.in.years': [26, 35, 40]}
bins_adj = sc.woebin(dt_f, y='creditability', breaks_list=breaks_adj)
print(bins_adj['age.in.years'])
# Cell 5: Transform train and test to WoE values
train_woe = sc.woebin_ply(train, bins_adj)
test_woe = sc.woebin_ply(test, bins_adj)
print(train_woe.head())
print(test_woe.head())

Your features are now encoded as monotonic risk signals that a credit committee can interpret and a logistic regression can consume directly. Next up, you'll fit the model and convert it into a points-based scorecard that maps each bin to a specific number of credit score points.

Fit Logistic Regression and Generate the Scorecard

Your WoE-encoded features are ready, but right now they're just numbers in a table. You need a model to learn which features matter most, and a scorecard to translate those weights into business-readable credit scores.

This is the step where your analysis becomes deployable. You'll fit a logistic regression, convert its coefficients into a points-based scorecard, and score applicants on a 300-850 scale that any credit committee can read.

In this step, get ready to:
  • Fit a logistic regression on WoE-encoded features and confirm positive coefficients.
  • Generate a points-based scorecard with PDO scaling.
  • Score applicants and visualize the score distribution for good vs. bad borrowers.
Add the scikit-learn import

The logistic regression model comes from scikit-learn. You need to add this import to your notebook before fitting the model.

  • In scorecard.ipynb, scroll to the first cell where your imports live.
  • Find this line:
import pandas as pd
  • Add the following import directly below it, so both lines sit together:
from sklearn.linear_model import LogisticRegression
  • Run this cell to confirm no import errors appear.
Fit a logistic regression on WoE features

With WoE-encoded features, logistic regression is the industry standard for scorecards. The L1 penalty performs feature selection by shrinking weak predictors to zero.

  • Create a new cell below your existing work and add the following code to separate features from the target, fit the model, and generate predictions:
# Fit logistic regression
y_train = train_woe['creditability']
X_train = train_woe.drop('creditability', axis=1)
y_test = test_woe['creditability']
X_test = test_woe.drop('creditability', axis=1)

lr = LogisticRegression(penalty='l1', C=0.9, solver='saga', random_state=42)
lr.fit(X_train, y_train)
print(dict(zip(X_train.columns, lr.coef_[0])))

train_pred = lr.predict_proba(X_train)[:, 1]
test_pred = lr.predict_proba(X_test)[:, 1]

What does this code do?

The first four lines separate the target variable (creditability) from the WoE-encoded features for both train and test sets.

The LogisticRegression uses penalty='l1' (Lasso) to zero out features that add no value, C=0.9 as the inverse regularization strength, and solver='saga' which supports L1 penalties.

The predict_proba call generates probability-of-default estimates for every applicant. The [:, 1] selects the probability of the positive class (default).

  • Run the cell. You should see a dictionary printed with column names as keys and their coefficients as values.
  • Confirm that the coefficients are positive. WoE encoding ensures this when bins are well-constructed, because higher WoE already encodes lower risk.

Why should coefficients be positive?

WoE encoding transforms each feature so that higher values mean lower default risk. A positive coefficient means the model agrees: more WoE equals less risk. A negative coefficient would suggest the binning has issues or the feature is not separating borrowers properly.

Seeing a convergence warning?

The SAGA solver may need more iterations on small datasets. You can add max_iter=10000 to the LogisticRegression() call to suppress the warning. The results will be the same.

help me fix my LogisticRegression convergence warning in my credit scorecard notebook

Generate the points-based scorecard

A fitted model gives probabilities, but credit committees need points. The sc.scorecard() function converts your logistic regression into a scorecard where every 20 points doubles (or halves) the odds of default.

  • Create a new cell and generate the scorecard table by running:
# Generate scorecard
card = sc.scorecard(bins_adj, lr, X_train.columns.tolist(), points0=600, odds0=1/19, pdo=20)
print(card)

What do these parameters mean?

  • points0=600 sets the base score. An applicant at the target odds receives 600 points.
  • odds0=1/19 defines the target odds. This means at 600 points, the odds of default are 1:19 (about 5% probability).
  • pdo=20 means Points to Double the Odds. Every 20 points above 600 halves the default probability. Every 20 below doubles it.
  • Run the cell. You should see a dictionary of DataFrames printed, one per variable, showing bin boundaries and the points assigned to each bin.

This is the scorecard table a credit committee reviews. Each row maps a specific applicant characteristic (like credit amount between 1400 and 2500) to a number of points.

What is PDO scaling?

PDO stands for Points to Double the Odds. It controls the steepness of the score scale. With PDO=20, moving from 600 to 620 means the applicant is twice as likely to repay. Banks choose PDO values (typically 20 or 50) based on their risk appetite and how spread out they want scores to be.

The formula is: Score = Offset - Factor * ln(odds), where Factor = PDO / ln(2) and Offset = points0 + Factor * ln(odds0).

  • Now score both your train and test datasets. Create a new cell and run:
# Score train and test
train_score = sc.scorecard_ply(train, card, print_step=0)
test_score = sc.scorecard_ply(test, card, print_step=0)
print(train_score.describe())
print(test_score.describe())

What does this code do?

The sc.scorecard_ply() function takes raw applicant data (not WoE-transformed) and applies the scorecard to produce a credit score for each row. The print_step=0 suppresses progress output.

The describe() output shows you the distribution statistics. Check that the min and max scores fall roughly in the 300-850 range, which is the standard credit score band.

  • Run the cell. You should see summary statistics for both datasets showing mean, min, max, and quartile scores.
  • Confirm the score range makes sense. Scores should cluster between roughly 400 and 700, with the mean around 500-600.

Scores outside the expected range?

If scores are all identical or outside the 300-850 band, check that bins_adj was generated with the correct breaks_list parameter and that your logistic regression fitted without errors.

help me debug why my scorecard scores are outside the expected 300-850 range

Visualize the score distribution

A scorecard only works if it separates good borrowers from bad ones. The score distributions should show clear separation, with bad borrowers clustering at lower scores.

  • Create a new cell and plot the score distribution by running:
# Score distribution for good vs bad
import matplotlib.pyplot as plt
train_score['creditability'] = train['creditability'].values
train_score.groupby('creditability')['score'].plot(kind='kde')
plt.title('Score Distribution: Good vs Bad Borrowers')
plt.xlabel('Credit Score')
plt.legend(['Bad', 'Good'])
plt.show()

What does this code do?

This attaches the original target labels back to the scored DataFrame, then plots a kernel density estimate (KDE) for each group. The KDE shows the shape of the score distribution without binning into a histogram.

If your scorecard has good discriminatory power, you'll see two distinct curves. Bad borrowers (0) peak at lower scores. Good borrowers (1) peak at higher scores. The less overlap between curves, the better your scorecard separates risk.

  • Run the cell. You should see a plot with two KDE curves showing clear separation between good and bad borrowers.

Before you move on, think about what you expect to see. Do you think these curves will overlap a lot, or separate cleanly? A KS statistic above 0.30 means meaningful separation.

Plot not showing?

Make sure you ran the plt.show() call at the end. In some VS Code Jupyter configurations, plots render inline automatically. If the plot appears blank, try restarting the kernel and re-running all cells from the top.

help me get my matplotlib plot to display in VS Code Jupyter

✔️ Awesome, I've got everything!

Great. Double check you've saved your notebook.

ⓧ I'd like to double check the full code

Confirm the following cells exist in your notebook, in order. Scroll up and verify each one is present.

  • Cell 1: Imports (scorecardpy, pandas, LogisticRegression) plus data loading, filtering, splitting, binning, and WoE transformation.
  • Cell 2: Fit logistic regression, print coefficients, generate train_pred and test_pred.
  • Cell 3: Generate scorecard with sc.scorecard() and print it.
  • Cell 4: Score train and test with sc.scorecard_ply() and print describe().
  • Cell 5: Score distribution KDE plot for good vs. bad borrowers.

Your scorecard is now generating real credit scores. Next up, you'll validate it with the same metrics a model risk team uses and compare it against an XGBoost challenger.

Validate the Scorecard and Compare Against XGBoost

A scorecard without validation is just a hypothesis. You have a model that assigns credit scores, but you have not yet proved it can actually separate good borrowers from bad ones.

In this step, you will run the same validation metrics a model risk team would scrutinize. Then you will train an XGBoost challenger model to test whether a more powerful algorithm significantly outperforms your interpretable scorecard on this dataset.

In this step, get ready to:
  • Evaluate your scorecard with industry-standard KS and AUC metrics.
  • Train an XGBoost challenger model and compare its performance.
  • Document why the simpler scorecard is the better production choice.
Evaluate scorecard performance with industry metrics

Before you can deploy a scorecard, you need to prove it discriminates between defaulters and non-defaulters. The KS statistic measures the maximum separation between the cumulative distributions of good and bad borrowers. The AUC measures overall ranking accuracy.

  • In your first cell in scorecard.ipynb, find the existing import lines and add these new imports below them:
import xgboost as xgb
import shap
import matplotlib.pyplot as plt

What do these imports do?

  • xgboost provides the gradient boosting challenger model you will compare against your scorecard.
  • shap computes SHAP values to explain the XGBoost model's predictions.
  • matplotlib.pyplot handles plotting that sc.perf_eva() relies on for rendering KS and ROC curves.
  • Create a new cell below your existing code and evaluate the scorecard's performance by running:
# Evaluate scorecard performance
perf_train = sc.perf_eva(y_train, train_pred, title='Train')
perf_test = sc.perf_eva(y_test, test_pred, title='Test')
print('Train KS:', perf_train['KS'], 'Train AUC:', perf_train['AUC'])
print('Test KS:', perf_test['KS'], 'Test AUC:', perf_test['AUC'])

What does this code do?

sc.perf_eva() takes the true labels and predicted probabilities, then computes KS, AUC, and Gini metrics. It also generates KS and ROC plots inline.

You evaluate both train and test sets so you can check for overfitting. If the train metrics are vastly higher than test, the model memorized the training data rather than learning generalizable patterns.

  • Run the cell.

You should see KS and ROC plots render inline, plus printed KS and AUC values for both sets. The test KS should be above 0.30 and test AUC above 0.70, confirming acceptable discriminatory power.

How do I interpret these numbers?

  • KS above 40% is strong for application scorecards. 20-30% is acceptable.
  • AUC above 0.70 means the model ranks borrowers better than chance. Above 0.80 is strong.
  • A small gap between train and test metrics (less than 5-10 percentage points) confirms the model generalizes well.

Not seeing plots or getting an error?

  • Make sure you have matplotlib imported. The perf_eva function relies on it for plot rendering.
  • Check that y_train and train_pred are defined in a cell that ran before this one. Your earlier cells must have executed successfully.
  • If the dictionary key lookup fails, print the returned object directly with print(perf_test) to see the available keys.

help me debug my perf_eva results

Train an XGBoost challenger model

Now you will pit your interpretable scorecard against a gradient boosting model. This is the same comparison a model risk team would run to verify you are not sacrificing too much accuracy for interpretability.

  • Create a new cell and train the XGBoost challenger by running:
# Train XGBoost challenger
neg = (y_train == 0).sum()
pos = (y_train == 1).sum()
xgb_model = xgb.XGBClassifier(scale_pos_weight=neg/pos, random_state=42, eval_metric='auc')
xgb_model.fit(X_train, y_train)
xgb_test_pred = xgb_model.predict_proba(X_test)[:, 1]
xgb_perf = sc.perf_eva(y_test, xgb_test_pred, title='XGBoost Test')
print('XGBoost Test KS:', xgb_perf['KS'], 'XGBoost Test AUC:', xgb_perf['AUC'])

What does this code do?

scale_pos_weight=neg/pos corrects for the class imbalance in the German Credit dataset (more good borrowers than bad). Without this, XGBoost would bias toward predicting everyone as good.

You evaluate the challenger on the same test set using the same sc.perf_eva() function, so the comparison is apples-to-apples.

  • Run the cell.

You should see XGBoost's KS and AUC values printed alongside its ROC and KS plots. Compare these against your scorecard's test metrics from the previous cell.

Why is XGBoost not dramatically better?

On small structured datasets like the German Credit data (1000 records), logistic regression achieves comparable performance to gradient boosting. Research confirms no statistically significant advantage among LR, XGBoost, and LightGBM on this dataset.

This is exactly the finding you want. It validates the scorecard as the better production choice because you get nearly the same accuracy with full transparency.

Seeing a NameError?

  • Confirm that import xgboost as xgb ran successfully in your imports cell above.
  • Check that X_train, y_train, X_test, and y_test are all defined from your earlier cells.

help me fix my XGBoost challenger model

Explain why you would still deploy the scorecard

A model risk team will ask why you chose the simpler model. You need to show, not just claim, that the scorecard's interpretability advantage outweighs any marginal accuracy gain from XGBoost.

  • Create a new cell and run the SHAP explainability comparison:
# SHAP explainability for XGBoost
explainer = shap.TreeExplainer(xgb_model)
shap_values = explainer.shap_values(X_test)
shap.summary_plot(shap_values, X_test)

# Decision rationale
print('Scorecard AUC:', perf_test['AUC'])
print('XGBoost AUC:', xgb_perf['AUC'])
print('Rationale: Logistic regression scorecard is preferred for regulatory transparency, auditability, and competitive performance on this dataset size.')

What does this code do?

shap.TreeExplainer computes exact SHAP values for tree-based models. The summary plot shows each feature's impact on predictions, with dots colored by feature value (red = high, blue = low).

The printed rationale documents your production decision. In a real credit risk environment, this reasoning goes into the model documentation package reviewed by validators and regulators.

  • Run the cell.

Before you look at the output, think about this: which model do you expect to have higher AUC, and by how much?

You should see a SHAP summary plot showing which WoE features XGBoost relies on most, plus the side-by-side AUC comparison printed below it. The AUC values should be close, confirming the scorecard's competitive performance.

Why the scorecard wins for production

  • Regulatory transparency: every point in the scorecard maps directly to a bin of a single variable. A regulator can trace any decision.
  • Auditability: the scorecard table is a static document. XGBoost requires additional SHAP tooling to explain each decision.
  • Stable performance: on small datasets, simpler models generalize as well or better than complex ensembles.

SHAP plot not rendering?

  • Make sure import shap ran without errors. If you see an import failure, run pip install shap in your terminal and restart the kernel.
  • If the plot appears blank, try adding plt.show() after the summary_plot call.

help me get my SHAP summary plot working

✔️ Awesome, I've got everything!

Great. Make sure you have saved your notebook before moving on.

ⓧ I'd like to double check the full code

Here are the cells you should have added in this step. Scroll up and confirm each is present in order below your existing cells from previous steps.

  • Updated imports cell: import xgboost as xgb, import shap, and import matplotlib.pyplot as plt added to your existing imports.
  • Scorecard performance evaluation cell with sc.perf_eva() for train and test.
  • XGBoost challenger training and evaluation cell.
  • SHAP explainability and decision rationale cell.

You have validated your scorecard with industry metrics and proven it holds its own against a powerful challenger model. Next up, you will tackle the Secret Mission and build a PSI monitoring report to detect when your scorecard needs recalibration.

Secret mission

Build a PSI Monitoring Report

Your scorecard works today. But what happens when the applicant population shifts next quarter? Build a PSI monitoring report that detects drift before your model starts making bad decisions.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally using free, open-source libraries. There are no ongoing costs or cloud resources to worry about.

Decide whether to keep your project files for portfolio use or delete them to free up disk space.

Resources you used:

  • Project folder credit-scorecard on your Desktop (contains your notebook, virtual environment, and all generated outputs).
  • Python virtual environment venv inside the project folder (contains scorecardpy, scikit-learn, xgboost, shap, pandas, matplotlib, and seaborn).

Keep everything running

No action needed. Choose this if you want to keep your scorecard notebook as a portfolio piece or continue experimenting with different binning strategies and model parameters.

  • Your virtual environment and notebook remain on your machine inside ~/Desktop/credit-scorecard.
  • To reactivate the environment later, open a terminal, navigate to the project folder, and run source venv/bin/activate (macOS/Linux) or venv\Scripts\activate (Windows).

Pause - I'll come back to this later

Deactivate your virtual environment to free up terminal resources. Your files stay on disk and you can reactivate any time.

  • If your virtual environment is still active, deactivate it by running:
deactivate
  • Your project folder and all files remain at ~/Desktop/credit-scorecard.
  • To resume later, navigate back to the folder and reactivate the environment.

Delete - I don't want to use this again

Remove the entire project folder, including the virtual environment and all generated outputs. This frees up disk space completely.

  • If your virtual environment is still active, deactivate it first:
deactivate
  • Delete the entire project folder by running this command (macOS/Linux):
rm -rf ~/Desktop/credit-scorecard
  • On Windows, run this instead:
rmdir /s /q %USERPROFILE%\Desktop\credit-scorecard
  • Confirm the folder is gone by running:
ls ~/Desktop | grep credit-scorecard
  • You should see no output, confirming the folder has been removed.

Nice Work!

Nice Work!

You just built a production-style credit risk scorecard using the same methodology banks use for regulatory-compliant lending decisions. From raw applicant data to a validated, points-based scoring system, you now own the complete scorecard lifecycle.

Here's what you accomplished:

  • Built a WoE-binned logistic regression scorecard that assigns credit scores in the 300-850 range, with a human-readable scorecard table mapping each variable bin to points that a non-technical credit committee can approve.
  • Validated the scorecard with industry-standard metrics (KS statistic, Gini, AUC, ROC curves) proving its discriminatory power on both train and test sets.
  • Compared your scorecard against an XGBoost challenger model and documented the interpretability vs. performance trade-off, demonstrating why the simpler model is the better production choice for regulated credit decisions.
  • Implemented PSI monitoring to detect population drift between training and test distributions, with documented governance actions at each threshold.

Ready to quiz yourself?