Credit Risk Scoring with AI Reports

Train an XGBoost credit risk model and generate AI reports with Amazon Bedrock.

Introduction

30 Second Summary

When you apply for a loan, a bank decides within seconds whether you're a good risk. Behind that decision is a machine learning model that also needs to explain itself to regulators.

In this project, you will build a credit risk scoring system that trains an XGBoost model to predict loan defaults, explains each prediction with SHAP values, and generates professional risk assessment reports using Amazon Bedrock.

What You'll Build

A credit risk scoring system that combines traditional ML with generative AI to predict defaults on 32,581 real loan records, explain predictions for compliance, and auto-generate professional risk reports.

By the end of this project, you'll have:

  • A trained XGBoost credit risk model that predicts loan defaults with strong AUC-ROC performance.
  • SHAP explainability visualizations showing which factors drive each prediction for regulatory compliance.
  • A Python pipeline that calls Amazon Bedrock (Nova Micro) to auto-generate professional risk assessment reports from model predictions.
  • Secret Mission: Build a batch risk assessment pipeline that processes multiple applicants and generates a summary dashboard.

Are there any prerequisites?

You need an AWS account with free credits, basic Python familiarity, and VS Code installed. The project walks you through everything else, including installing libraries and enabling Bedrock model access.

Before We Start

Before you can train a credit risk model and generate AI-powered reports, you need a working development environment with the right Python libraries and AWS access configured.

In this step, you will set up your project directory, install all the machine learning and cloud libraries, connect to AWS, and download the dataset you will train on.

In this step, get ready to:
  • Install Python dependencies (XGBoost, SHAP, boto3, pandas, scikit-learn, matplotlib).
  • Configure the AWS CLI and enable Amazon Bedrock model access.
  • Download the credit risk dataset from Kaggle.
Install Python dependencies and verify versions

Your project needs several Python libraries for machine learning, explainability, and cloud connectivity. You will install them into an isolated virtual environment so they do not conflict with other Python projects on your machine.

  • Open your terminal. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal (macOS) or PowerShell (Windows) and press Enter to open it.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir credit-risk-model
cd credit-risk-model
  • Confirm you are inside the new folder by running:
pwd

You should see a path ending in Desktop/credit-risk-model.

  • Create a Python virtual environment by running this command:
python3 -m venv venv

What is a virtual environment?

A virtual environment is an isolated space for your Python project's dependencies. It keeps this project's libraries separate from other Python projects on your system.

  • Activate the virtual environment:

macOS

source venv/bin/activate

Windows

venv\Scripts\activate

You should see (venv) appear at the beginning of your terminal prompt. This confirms the virtual environment is active.

Don't see (venv) in your prompt?

On Windows, you may need to allow script execution first. Run this command in PowerShell, then try the activate command again:

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

On macOS, make sure you typed source before the path. If you see a permission denied error, check that the venv folder was created successfully with ls.

Help me troubleshoot my virtual environment activation.

  • Install all the project dependencies by running:
pip install xgboost==3.2.0 shap==0.46.0 boto3 pandas scikit-learn matplotlib

What are these libraries?

  • xgboost is the gradient boosting library you will use to train the credit risk model.
  • shap generates human-readable explanations of model predictions.
  • boto3 is the AWS SDK for Python, used to call Amazon Bedrock.
  • pandas and scikit-learn handle data processing and model evaluation.
  • matplotlib creates the SHAP visualization charts.

The installation may take a couple of minutes. When it finishes, you should see a message listing all the packages that were installed.

  • Verify the installations by running this quick import check:
python3 -c "import xgboost, shap, boto3, pandas, sklearn, matplotlib; print('All packages imported successfully')"

You should see All packages imported successfully printed in your terminal.

Seeing an import error?

Make sure your virtual environment is active (you should see (venv) in your prompt). If you ran pip install without the venv active, deactivate and reactivate it, then reinstall.

Help me fix my Python import error.

Configure AWS CLI and enable Bedrock model access

Your project calls Amazon Bedrock to generate risk reports. To authenticate those API calls from your terminal, you need the AWS CLI configured with your credentials.

  • Check whether the AWS CLI is installed by running:
aws --version

✔️ I see a version number

The AWS CLI is installed and ready to configure.

ⓧ Command not found

You need to install the AWS CLI first.

  • Follow the official installation guide for your operating system: AWS CLI installation guide.
  • After installing, close and reopen your terminal, then run aws --version again to confirm it works.
  • Configure the AWS CLI with your credentials by running:
aws configure

What does aws configure ask for?

The command prompts you for four values, one at a time. Enter each value and press Enter:

  • AWS Access Key ID - paste your access key.
  • AWS Secret Access Key - paste your secret key.
  • Default region name - type us-east-1.
  • Default output format - press Enter to accept the default.

Where do I find my access keys?

In the AWS Management Console, navigate to IAM, select Users, choose your user, then go to the Security credentials tab.

Click Create access key to generate a new key pair. Copy both the Access Key ID and Secret Access Key when prompted. You will not be able to see the secret key again after closing the dialog.

  • Now enable access to the Amazon Nova Micro model. Open your browser and navigate to the Amazon Bedrock console.
  • Make sure you are in the US East (N. Virginia) region. Check the region selector in the top right corner of the console. If it shows a different region, click it and select US East (N. Virginia).
  • In the left navigation pane, under Bedrock configurations, choose Model access.
  • Verify that Amazon Nova Micro shows Access granted status. Amazon models are enabled by default with the correct IAM permissions.

What if Nova Micro does not show Access granted?

Choose Enable specific models. Select the checkbox next to Amazon Nova Micro in the list, then choose Next.

On the Review and submit page, choose Submit. The change may take a few minutes to take effect.

  • Test your Bedrock connection by running this command back in your terminal:
python3 -c "
import boto3
client = boto3.client('bedrock-runtime', region_name='us-east-1')
response = client.converse(
    modelId='amazon.nova-micro-v1:0',
    messages=[{'role': 'user', 'content': [{'text': 'Say hello in one sentence.'}]}],
    inferenceConfig={'maxTokens': 50, 'temperature': 0.3}
)
print(response['output']['message']['content'][0]['text'])
"

You should see a short greeting printed by the Nova Micro model. This confirms your AWS credentials and Bedrock access are working correctly.

Seeing an access denied or model error?

Double-check that your region is set to us-east-1 in both aws configure and the Bedrock console. Also confirm that your IAM user has the AmazonBedrockFullAccess policy attached.

Help me debug my Bedrock access issue.

Download the dataset

The model you will train uses a publicly available credit risk dataset from Kaggle containing 32,581 real loan records. Each record includes features like the applicant's age, income, loan amount, and whether they defaulted.

  • Open your browser and navigate to the Kaggle Credit Risk Dataset page.
  • Sign in to Kaggle (or create a free account if you do not have one).
  • Click the Download button to download the dataset as a zip file.
  • Unzip the downloaded file. You should find a file called credit_risk_dataset.csv inside.
  • Move credit_risk_dataset.csv into your credit-risk-model folder on the Desktop.
  • Verify the dataset loads correctly by running this command in your terminal:
python3 -c "
import pandas as pd
df = pd.read_csv('credit_risk_dataset.csv')
print(f'Dataset loaded: {df.shape[0]} rows, {df.shape[1]} columns')
print(f'Columns: {list(df.columns)}')
"

You should see output showing 32,581 rows and 12 columns, along with column names including person_age, loan_status, and others.

Seeing a FileNotFoundError?

Make sure credit_risk_dataset.csv is in the same folder where your terminal is running. Run ls to check the current directory contents. The file should appear in the listing.

Help me fix my FileNotFoundError.

  • Open VS Code in your project directory. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code and press Enter to open it.
  • In VS Code, click File in the top menu bar.
  • Click Open Folder.
  • Navigate to your Desktop, select the credit-risk-model folder, and click Open.

You should see credit_risk_dataset.csv listed in the VS Code Explorer sidebar on the left.

Your environment is ready. Next up, you will build and train the XGBoost credit risk model on those 32,581 loan records.

Build the Credit Risk Model

Your environment is set up with all the libraries installed and the dataset ready to go. Now it is time to build the core of the project: a machine learning model that predicts whether a loan applicant will default.

Credit risk datasets are messy. They contain missing values, outliers, and categorical text that a model cannot process directly. In this step, you will clean the data, engineer features, and train an XGBoost classifier that learns patterns from 32,581 real loan records.

In this step, get ready to:
  • Load and preprocess the credit risk dataset (handle missing values, remove outliers, encode categories).
  • Train an XGBoost model with class imbalance handling and early stopping.
  • Evaluate model performance with AUC-ROC and a classification report.
Load and preprocess the data

Raw data needs cleaning before a model can learn from it. The credit risk dataset has missing interest rates, unrealistic ages, and text-based categories that need converting to numbers.

  • In VS Code, create a new file called credit_risk_model.py in your project directory.
  • Add the imports and data loading code by pasting the following into your new file:
import pandas as pd
import numpy as np
import xgboost as xgb
from sklearn.model_selection import train_test_split
from sklearn.metrics import roc_auc_score, classification_report, confusion_matrix
import shap
import matplotlib.pyplot as plt
import boto3
import json

# ============================================================
# STEP 2: Build the Credit Risk Model
# ============================================================

# Load data
df = pd.read_csv("credit_risk_dataset.csv")

# Handle missing values
df["loan_int_rate"].fillna(df["loan_int_rate"].median(), inplace=True)
df["person_emp_length"].fillna(df["person_emp_length"].median(), inplace=True)

# Remove outliers
df = df[df["person_age"] <= 80]
df = df[df["person_emp_length"] <= 60]

What does this code do?

The imports pull in every library you will use across the entire project (data handling, ML, explainability, AWS, and plotting).

  • pd.read_csv("credit_risk_dataset.csv") loads all 32,581 loan records into a DataFrame.
  • fillna(...median()) replaces missing values in loan_int_rate and person_emp_length with the column median, which is safer than dropping rows.
  • The outlier filters remove records where person_age > 80 or person_emp_length > 60 because these are likely data entry errors.
  • Save credit_risk_model.py.
  • Run the script to confirm it loads without errors by running this command in your terminal:
python credit_risk_model.py
  • The script should complete silently with no error messages. No output is expected yet.

Seeing a FileNotFoundError?

  • Make sure credit_risk_dataset.csv is in the same folder as credit_risk_model.py. Check for typos in the filename.
  • Confirm your terminal is in the project directory. Run ls and look for both files in the output.

Why can't Python find my CSV file?

  • Now add the feature encoding section below the outlier removal code in credit_risk_model.py:
# Separate features and target
target = "loan_status"
categorical_cols = ["person_home_ownership", "loan_intent", "loan_grade", "cb_person_default_on_file"]

# One-hot encode categorical features
df_encoded = pd.get_dummies(df, columns=categorical_cols, drop_first=True)

X = df_encoded.drop(columns=[target])
y = df_encoded[target]

What does this code do?

Machine learning models need numeric inputs. The four categorical columns (home ownership, loan intent, loan grade, and default history) contain text values like "RENT" or "MEDICAL".

  • pd.get_dummies() converts each category into separate binary (0/1) columns. The drop_first=True avoids redundancy by dropping one category per feature.
  • X holds all features (the inputs the model learns from). y holds the target column loan_status (0 = no default, 1 = default).
  • Save credit_risk_model.py. You will not see output yet. The next substep uses this data to train the model.
Train the XGBoost model

With clean, numeric features ready, you can now split the data and train the model. The dataset has a class imbalance: only about 22% of applicants default. Without compensation, the model would learn to predict "no default" for everyone and still look 78% accurate.

  • Add the train/test split below your encoding code in credit_risk_model.py:
# Train/test split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

# Calculate scale_pos_weight for class imbalance
scale_pos_weight = len(y_train[y_train == 0]) / len(y_train[y_train == 1])

What does this code do?

  • train_test_split holds back 20% of the data for testing. The stratify=y option keeps the same proportion of defaults in both sets.
  • scale_pos_weight tells XGBoost to pay more attention to the minority class (defaults). It equals the count of non-defaults divided by the count of defaults.

This is the standard XGBoost heuristic for imbalanced data: set scale_pos_weight to sum(negative instances) / sum(positive instances).

  • Now add the model configuration and training below the scale_pos_weight line:
# Train XGBoost model
model = xgb.XGBClassifier(
    objective="binary:logistic",
    eval_metric="auc",
    scale_pos_weight=scale_pos_weight,
    max_depth=5,
    learning_rate=0.05,
    n_estimators=300,
    subsample=0.8,
    colsample_bytree=0.8,
    min_child_weight=3,
    gamma=0.1,
    reg_alpha=0.1,
    reg_lambda=1.0,
    random_state=42,
    early_stopping_rounds=20,
)

model.fit(
    X_train, y_train,
    eval_set=[(X_test, y_test)],
    verbose=False
)

What does this code do?

  • objective="binary:logistic" outputs a probability of default between 0 and 1.
  • eval_metric="auc" measures how well the model separates defaults from non-defaults.
  • early_stopping_rounds=20 stops training if the AUC does not improve for 20 rounds. This prevents overfitting.
  • verbose=False suppresses per-round output so the console stays clean.

Why XGBoost for credit risk?

XGBoost is the industry standard for tabular financial ML. It handles mixed feature types, works well with class imbalance via scale_pos_weight, and has native SHAP support for fast, exact explainability computations you will use in the next step.

  • Save credit_risk_model.py and run it again:
python credit_risk_model.py
  • The script should complete without errors. Training takes a few seconds. No output is printed yet because verbose=False silences the training logs.

Seeing a ValueError or training error?

  • If you see a message about feature names, make sure the encoding step ran before the split. The order of your code blocks matters.
  • If training hangs for more than 30 seconds, check that n_estimators is set to 300 (not a much larger number).

Why is my XGBoost model throwing an error during training?

Evaluate model performance

A trained model means nothing until you measure how well it performs on unseen data. You will compute predictions on the test set and print key metrics that tell you whether the model is reliable enough for credit decisions.

  • Add the evaluation code below the model.fit() call in credit_risk_model.py:
# Evaluate
y_pred_proba = model.predict_proba(X_test)[:, 1]
y_pred = (y_pred_proba >= 0.5).astype(int)

print("=== Model Performance ===")
print(f"AUC-ROC: {roc_auc_score(y_test, y_pred_proba):.4f}")
print(f"\nClassification Report:\n{classification_report(y_test, y_pred)}")
print(f"Confusion Matrix:\n{confusion_matrix(y_test, y_pred)}")

What does this code do?

  • predict_proba(X_test)[:, 1] gets the probability of default for each test applicant.
  • A threshold of 0.5 converts probabilities into binary predictions (default or not).
  • AUC-ROC measures how well the model ranks defaulters above non-defaulters. A score above 0.85 is strong for credit risk.
  • The classification report shows precision, recall, and F1-score for each class.
  • The confusion matrix reveals exactly how many applicants were correctly or incorrectly classified.
  • Save credit_risk_model.py and run it:
python credit_risk_model.py
  • You should see the model performance output starting with === Model Performance === followed by the AUC-ROC score, classification report, and confusion matrix.
  • Your AUC-ROC should be approximately 0.90 or higher.

AUC-ROC score looks too low?

  • Make sure the outlier removal and missing value imputation ran before encoding. If outliers are still in the data, they can skew model performance.
  • Double check that stratify=y is present in your train_test_split call. Without it, the test set may not represent the true class distribution.

Help me understand why my AUC-ROC is lower than expected.

What is AUC-ROC?

AUC-ROC (Area Under the Receiver Operating Characteristic curve) measures how well the model distinguishes between defaulters and non-defaulters across all possible thresholds. A score of 1.0 means perfect separation. A score of 0.5 means no better than random guessing.

✔️ Awesome, I've got everything!

Great. Double check you have saved credit_risk_model.py and your AUC-ROC is printing above 0.85.

ⓧ I'd like to double check the full code

Here is the complete credit_risk_model.py up to this point:

import pandas as pd
import numpy as np
import xgboost as xgb
from sklearn.model_selection import train_test_split
from sklearn.metrics import roc_auc_score, classification_report, confusion_matrix
import shap
import matplotlib.pyplot as plt
import boto3
import json

# ============================================================
# STEP 2: Build the Credit Risk Model
# ============================================================

# Load data
df = pd.read_csv("credit_risk_dataset.csv")

# Handle missing values
df["loan_int_rate"].fillna(df["loan_int_rate"].median(), inplace=True)
df["person_emp_length"].fillna(df["person_emp_length"].median(), inplace=True)

# Remove outliers
df = df[df["person_age"] <= 80]
df = df[df["person_emp_length"] <= 60]

# Separate features and target
target = "loan_status"
categorical_cols = ["person_home_ownership", "loan_intent", "loan_grade", "cb_person_default_on_file"]

# One-hot encode categorical features
df_encoded = pd.get_dummies(df, columns=categorical_cols, drop_first=True)

X = df_encoded.drop(columns=[target])
y = df_encoded[target]

# Train/test split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

# Calculate scale_pos_weight for class imbalance
scale_pos_weight = len(y_train[y_train == 0]) / len(y_train[y_train == 1])

# Train XGBoost model
model = xgb.XGBClassifier(
    objective="binary:logistic",
    eval_metric="auc",
    scale_pos_weight=scale_pos_weight,
    max_depth=5,
    learning_rate=0.05,
    n_estimators=300,
    subsample=0.8,
    colsample_bytree=0.8,
    min_child_weight=3,
    gamma=0.1,
    reg_alpha=0.1,
    reg_lambda=1.0,
    random_state=42,
    early_stopping_rounds=20,
)

model.fit(
    X_train, y_train,
    eval_set=[(X_test, y_test)],
    verbose=False
)

# Evaluate
y_pred_proba = model.predict_proba(X_test)[:, 1]
y_pred = (y_pred_proba >= 0.5).astype(int)

print("=== Model Performance ===")
print(f"AUC-ROC: {roc_auc_score(y_test, y_pred_proba):.4f}")
print(f"\nClassification Report:\n{classification_report(y_test, y_pred)}")
print(f"Confusion Matrix:\n{confusion_matrix(y_test, y_pred)}")

Your credit risk model is trained and performing well. Next up, you will use SHAP to explain exactly why the model makes each prediction. This is critical for regulatory compliance in financial services.

Explain Predictions with SHAP

Your model is trained and producing strong predictions. But a credit risk model that only outputs a number is not enough for production use.

Regulators and credit committees need to understand why a model flagged someone as high-risk. SHAP values provide exactly that transparency by quantifying each feature's contribution to every individual prediction.

In this step, get ready to:
  • Generate SHAP values for the test set and visualize global feature importance.
  • Create individual waterfall plots for high-risk and low-risk applicants.
  • Build a formatter that structures SHAP explanations for the LLM report generator.
Generate SHAP explanations

SHAP (SHapley Additive exPlanations) works by calculating how much each feature contributed to pushing a prediction away from the average. A positive SHAP value means that feature increased the predicted default risk. A negative value means it decreased risk.

  • In credit_risk_model.py, scroll to the bottom of the file (below the last print statement from Step 2).
  • Add the following code to create a SHAP explainer and generate a global feature importance chart:
# ============================================================
# STEP 3: Explain Predictions with SHAP
# ============================================================

# Create SHAP explainer
explainer = shap.Explainer(model)
shap_values = explainer(X_test)

# Global feature importance
shap.plots.bar(shap_values, max_display=10, show=False)
plt.tight_layout()
plt.savefig("feature_importance.png", dpi=150, bbox_inches="tight")
plt.close()
print("\nSaved global feature importance plot to feature_importance.png")

What does this code do?

  • shap.Explainer(model) creates a TreeExplainer optimized for tree-based models like XGBoost. It computes exact SHAP values efficiently using the tree structure.
  • explainer(X_test) computes SHAP values for every feature of every applicant in the test set. This may take a few seconds.
  • shap.plots.bar() generates a horizontal bar chart ranking features by their average absolute impact across all predictions. This shows which features matter most overall.
  • Save credit_risk_model.py.
  • Run the script in your terminal. After the model performance output from Step 2, you should see the message Saved global feature importance plot to feature_importance.png.
  • Check your project folder. You should see a new feature_importance.png file. Open it to see which features drive risk the most.

Not seeing the plot file?

  • Make sure you saved credit_risk_model.py before running it.
  • If you see an import error for shap, confirm you activated your virtual environment before running.
  • Check that you placed this code below the existing Step 2 output, not inside any existing block.

Why isn't my SHAP feature importance plot generating?

What is model explainability?

In financial services, regulations require that lenders can explain why a loan application was denied. A model that just says "78% chance of default" is not sufficient.

SHAP values satisfy this requirement by showing exactly which factors (income, loan grade, employment history) contributed to each decision. This is called model governance.

Create individual explanations

The global bar chart shows overall feature importance. Now you need individual explanations for specific applicants. Waterfall plots show how each feature pushed one applicant's prediction up or down from the baseline.

  • Add the following code below the previous block to identify extreme-risk applicants and generate their waterfall plots:
# Find high-risk and low-risk applicants
high_risk_idx = np.argmax(y_pred_proba)
low_risk_idx = np.argmin(y_pred_proba)

# Waterfall plot for high-risk applicant
shap.plots.waterfall(shap_values[high_risk_idx], show=False)
plt.tight_layout()
plt.savefig("high_risk_waterfall.png", dpi=150, bbox_inches="tight")
plt.close()
print("Saved high-risk applicant waterfall plot to high_risk_waterfall.png")

# Waterfall plot for low-risk applicant
shap.plots.waterfall(shap_values[low_risk_idx], show=False)
plt.tight_layout()
plt.savefig("low_risk_waterfall.png", dpi=150, bbox_inches="tight")
plt.close()
print("Saved low-risk applicant waterfall plot to low_risk_waterfall.png")

What does this code do?

  • np.argmax(y_pred_proba) finds the index of the applicant with the highest predicted default probability. np.argmin finds the safest applicant.
  • shap.plots.waterfall() creates a chart showing how each feature moved one specific prediction. Red bars push the score toward default. Blue bars push it toward non-default.
  • You generate one waterfall for each extreme to compare what a risky profile looks like versus a safe one.
  • Save credit_risk_model.py and run the script again.
  • You should see two new messages confirming the waterfall plots were saved.
  • Open high_risk_waterfall.png and low_risk_waterfall.png in your project folder to see the individual explanations.

Seeing a blank or overlapping waterfall plot?

  • The plt.tight_layout() and bbox_inches="tight" parameters handle most layout issues. Make sure both lines are present.
  • If the image is blank, confirm plt.close() is called after each save. Without it, matplotlib may stack plots on top of each other.

Help me debug my SHAP waterfall plot generation.

Build the explanation formatter

The waterfall plots are great for human review. To feed explanations into an LLM, you need structured data instead of images. This function will extract the top contributing features into a dictionary that the report generator can consume in the next step.

  • Add the following function and its usage below the waterfall plot code:
def format_shap_explanation(shap_values_single, feature_names, risk_score):
    """Format SHAP values into structured explanation for LLM input."""
    values = shap_values_single.values
    feature_contributions = list(zip(feature_names, values))
    sorted_contributions = sorted(feature_contributions, key=lambda x: abs(x[1]), reverse=True)

    positive_factors = [(name, val) for name, val in sorted_contributions if val > 0][:3]
    negative_factors = [(name, val) for name, val in sorted_contributions if val < 0][:3]

    return {
        "risk_score": round(float(risk_score), 4),
        "top_positive_factors": positive_factors,
        "top_negative_factors": negative_factors,
    }


# Generate explanations for sample applicants
high_risk_explanation = format_shap_explanation(
    shap_values[high_risk_idx], X_test.columns.tolist(), y_pred_proba[high_risk_idx]
)
low_risk_explanation = format_shap_explanation(
    shap_values[low_risk_idx], X_test.columns.tolist(), y_pred_proba[low_risk_idx]
)

print(f"\nHigh-risk applicant explanation: {json.dumps(high_risk_explanation, indent=2, default=str)}")
print(f"\nLow-risk applicant explanation: {json.dumps(low_risk_explanation, indent=2, default=str)}")

What does this code do?

  • format_shap_explanation() takes raw SHAP values for one applicant and returns a clean dictionary. It sorts features by absolute impact, then separates them into risk-increasing (positive) and risk-decreasing (negative) groups.
  • top_positive_factors contains the top 3 features that pushed the applicant toward default. top_negative_factors contains the top 3 protective features.
  • This structured output is what Amazon Bedrock will consume in Step 4 to write professional risk assessment reports in plain English.
  • Save credit_risk_model.py and run the script.
  • At the bottom of the terminal output, you should see JSON-formatted dictionaries for both the high-risk and low-risk applicants. Each dictionary shows a risk_score, top_positive_factors, and top_negative_factors.

Seeing an attribute error on .values?

  • Make sure you are passing shap_values[high_risk_idx] (a single row from the SHAP explanation object), not a raw numpy array.
  • Confirm that shap_values was created with explainer(X_test) using the callable syntax, not the older explainer.shap_values() method.

Help me fix my format_shap_explanation function.

✔️ Awesome, I've got everything!

Great. Double check you have saved credit_risk_model.py and can see both explanation dictionaries printed in the terminal.

ⓧ I'd like to double check the full code

Here is the complete credit_risk_model.py file through Step 3:

import pandas as pd
import numpy as np
import xgboost as xgb
from sklearn.model_selection import train_test_split
from sklearn.metrics import roc_auc_score, classification_report, confusion_matrix
import shap
import matplotlib.pyplot as plt
import boto3
import json

# ============================================================
# STEP 2: Build the Credit Risk Model
# ============================================================

# Load data
df = pd.read_csv("credit_risk_dataset.csv")

# Handle missing values
df["loan_int_rate"].fillna(df["loan_int_rate"].median(), inplace=True)
df["person_emp_length"].fillna(df["person_emp_length"].median(), inplace=True)

# Remove outliers
df = df[df["person_age"] <= 80]
df = df[df["person_emp_length"] <= 60]

# Separate features and target
target = "loan_status"
categorical_cols = ["person_home_ownership", "loan_intent", "loan_grade", "cb_person_default_on_file"]

# One-hot encode categorical features
df_encoded = pd.get_dummies(df, columns=categorical_cols, drop_first=True)

X = df_encoded.drop(columns=[target])
y = df_encoded[target]

# Train/test split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

# Calculate scale_pos_weight for class imbalance
scale_pos_weight = len(y_train[y_train == 0]) / len(y_train[y_train == 1])

# Train XGBoost model
model = xgb.XGBClassifier(
    objective="binary:logistic",
    eval_metric="auc",
    scale_pos_weight=scale_pos_weight,
    max_depth=5,
    learning_rate=0.05,
    n_estimators=300,
    subsample=0.8,
    colsample_bytree=0.8,
    min_child_weight=3,
    gamma=0.1,
    reg_alpha=0.1,
    reg_lambda=1.0,
    random_state=42,
    early_stopping_rounds=20,
)

model.fit(
    X_train, y_train,
    eval_set=[(X_test, y_test)],
    verbose=False
)

# Evaluate
y_pred_proba = model.predict_proba(X_test)[:, 1]
y_pred = (y_pred_proba >= 0.5).astype(int)

print("=== Model Performance ===")
print(f"AUC-ROC: {roc_auc_score(y_test, y_pred_proba):.4f}")
print(f"\nClassification Report:\n{classification_report(y_test, y_pred)}")
print(f"Confusion Matrix:\n{confusion_matrix(y_test, y_pred)}")

# ============================================================
# STEP 3: Explain Predictions with SHAP
# ============================================================

# Create SHAP explainer
explainer = shap.Explainer(model)
shap_values = explainer(X_test)

# Global feature importance
shap.plots.bar(shap_values, max_display=10, show=False)
plt.tight_layout()
plt.savefig("feature_importance.png", dpi=150, bbox_inches="tight")
plt.close()
print("\nSaved global feature importance plot to feature_importance.png")

# Find high-risk and low-risk applicants
high_risk_idx = np.argmax(y_pred_proba)
low_risk_idx = np.argmin(y_pred_proba)

# Waterfall plot for high-risk applicant
shap.plots.waterfall(shap_values[high_risk_idx], show=False)
plt.tight_layout()
plt.savefig("high_risk_waterfall.png", dpi=150, bbox_inches="tight")
plt.close()
print("Saved high-risk applicant waterfall plot to high_risk_waterfall.png")

# Waterfall plot for low-risk applicant
shap.plots.waterfall(shap_values[low_risk_idx], show=False)
plt.tight_layout()
plt.savefig("low_risk_waterfall.png", dpi=150, bbox_inches="tight")
plt.close()
print("Saved low-risk applicant waterfall plot to low_risk_waterfall.png")


def format_shap_explanation(shap_values_single, feature_names, risk_score):
    """Format SHAP values into structured explanation for LLM input."""
    values = shap_values_single.values
    feature_contributions = list(zip(feature_names, values))
    sorted_contributions = sorted(feature_contributions, key=lambda x: abs(x[1]), reverse=True)

    positive_factors = [(name, val) for name, val in sorted_contributions if val > 0][:3]
    negative_factors = [(name, val) for name, val in sorted_contributions if val < 0][:3]

    return {
        "risk_score": round(float(risk_score), 4),
        "top_positive_factors": positive_factors,
        "top_negative_factors": negative_factors,
    }


# Generate explanations for sample applicants
high_risk_explanation = format_shap_explanation(
    shap_values[high_risk_idx], X_test.columns.tolist(), y_pred_proba[high_risk_idx]
)
low_risk_explanation = format_shap_explanation(
    shap_values[low_risk_idx], X_test.columns.tolist(), y_pred_proba[low_risk_idx]
)

print(f"\nHigh-risk applicant explanation: {json.dumps(high_risk_explanation, indent=2, default=str)}")
print(f"\nLow-risk applicant explanation: {json.dumps(low_risk_explanation, indent=2, default=str)}")

You now have a fully explainable credit risk model. Every prediction can be broken down into the specific factors that drove it. Next up, you will feed these structured explanations into Amazon Bedrock to automatically generate professional risk assessment reports.

Generate Risk Reports with Amazon Bedrock

In the previous step, you built SHAP explanations that reveal exactly why your model flags an applicant as high or low risk. Those structured explanations are useful for data scientists, but a credit committee needs a plain-English report.

In this step, you will connect your model's output to Amazon Bedrock and have a large language model write professional risk assessment reports automatically. The LLM takes your SHAP explanations as input and produces a structured report with a risk level, key findings, and a recommendation.

In this step, get ready to:
  • Connect to Amazon Bedrock's Nova Micro model via the Converse API.
  • Build a prompt that feeds SHAP explanation data to the LLM.
  • Generate and save professional risk assessment reports for both applicants.
Connect to Amazon Bedrock

Your script needs a way to send prompts to the Amazon Nova Micro model. The Converse API is AWS's recommended interface because it works across all Bedrock models without code changes.

  • In credit_risk_model.py, scroll to the bottom of the file (below the line that prints the low-risk explanation).
  • Add the Step 4 section header and Bedrock client setup by pasting the following code:
# ============================================================
# STEP 4: Generate Risk Reports with Amazon Bedrock
# ============================================================

# Create Bedrock client
bedrock_client = boto3.client("bedrock-runtime", region_name="us-east-1")

SYSTEM_PROMPT = """You are a senior credit risk analyst at a financial institution. 
Your role is to write professional, concise risk assessment reports based on model predictions and explainability data.
Each report should include: Risk Level, Key Findings, Recommendation (Approve/Review/Decline), and any Conditions.
Use clear, professional language suitable for a credit committee."""

What does this code do?

The boto3.client("bedrock-runtime", region_name="us-east-1") call creates a client that can send inference requests to Bedrock models in the us-east-1 region.

The SYSTEM_PROMPT constant tells the LLM to behave like a credit risk analyst. It specifies the exact output structure the model should follow: risk level, key findings, recommendation, and conditions.

Why the Converse API instead of InvokeModel?

AWS recommends the Converse API because it provides a unified interface across all Bedrock models. If you later upgrade from Nova Micro to a more powerful model, you only change the modelId string. The rest of your code stays the same.

Build the report generation prompt

The LLM needs context about each applicant to write a meaningful report. You will build a function that formats the SHAP explanation data into a structured prompt and sends it to Nova Micro.

  • Add the generate_risk_report function below the SYSTEM_PROMPT constant by pasting this code:
def generate_risk_report(explanation, bedrock_client):
    """Generate a risk assessment report using Amazon Bedrock Nova Micro."""
    risk_pct = explanation["risk_score"] * 100

    positive_factors_text = "\n".join(
        [f"  - {name}: contribution +{val:.3f}" for name, val in explanation["top_positive_factors"]]
    )
    negative_factors_text = "\n".join(
        [f"  - {name}: contribution {val:.3f}" for name, val in explanation["top_negative_factors"]]
    )

    user_prompt = f"""Generate a risk assessment report for this loan applicant:

Probability of Default: {risk_pct:.1f}%

Top Risk Factors (increasing default probability):
{positive_factors_text}

Top Protective Factors (decreasing default probability):
{negative_factors_text}

Please provide a structured risk assessment report."""

    response = bedrock_client.converse(
        modelId="amazon.nova-micro-v1:0",
        messages=[{"role": "user", "content": [{"text": user_prompt}]}],
        system=[{"text": SYSTEM_PROMPT}],
        inferenceConfig={"maxTokens": 512, "temperature": 0.3, "topP": 0.9},
    )

    return response["output"]["message"]["content"][0]["text"]

What does this function do?

  • It converts the risk_score from a decimal (e.g. 0.92) to a percentage (92.0%) for readability.
  • It formats the SHAP positive and negative factors into bullet-point text the LLM can easily parse.
  • The user_prompt feeds the applicant's probability of default, risk factors, and protective factors to the model.
  • The converse call sends the prompt with temperature=0.3 to keep responses factual and consistent.
  • It extracts the generated text from Bedrock's response structure.

Why low temperature for risk reports?

A temperature of 0.3 makes the model more deterministic. For financial reports, you want consistent, predictable language rather than creative variation. Higher temperatures (0.7+) would produce more varied phrasing on each run, which is inappropriate for regulatory documents.

Generate and display reports

Now you will call the function on both the high-risk and low-risk applicants from Step 3, then save the results as text files.

  • Add the report generation and saving code below the generate_risk_report function:
# Generate reports
print("\n" + "=" * 60)
print("RISK ASSESSMENT REPORT - HIGH RISK APPLICANT")
print("=" * 60)
high_risk_report = generate_risk_report(high_risk_explanation, bedrock_client)
print(high_risk_report)

print("\n" + "=" * 60)
print("RISK ASSESSMENT REPORT - LOW RISK APPLICANT")
print("=" * 60)
low_risk_report = generate_risk_report(low_risk_explanation, bedrock_client)
print(low_risk_report)

# Save reports
with open("high_risk_report.txt", "w") as f:
    f.write(high_risk_report)
with open("low_risk_report.txt", "w") as f:
    f.write(low_risk_report)

print("\nReports saved to high_risk_report.txt and low_risk_report.txt")

What does this code do?

  • It calls generate_risk_report once for the high-risk applicant and once for the low-risk applicant.
  • Each report is printed to the console with a header so you can compare them side by side.
  • Both reports are saved to .txt files in your project directory as portfolio artifacts.
  • Save credit_risk_model.py.
  • Run the full script in your terminal:
python credit_risk_model.py

The script will take a minute or two to run because it trains the model, computes SHAP values, and then makes two Bedrock API calls. At the end, you should see two formatted risk assessment reports printed to your terminal.

Seeing an AccessDeniedException?

This means your AWS account does not have access to the Nova Micro model. Return to the Amazon Bedrock console, navigate to Model access, and confirm that Amazon Nova Micro is listed as Access granted.

If you see a throttling error (HTTP 429), wait a few seconds and run the script again. Bedrock has per-account rate limits that can briefly throttle new accounts.

Help me debug my Bedrock connection error.

  • Verify that high_risk_report.txt and low_risk_report.txt were created in your project folder by running:
ls *.txt

You should see high_risk_report.txt and low_risk_report.txt in the output.

✔️ Awesome, I've got everything!

Great. Make sure you have saved credit_risk_model.py and that both report files exist in your project folder.

ⓧ I'd like to double check the full code

Here is the complete credit_risk_model.py file after Step 4. Compare your file against this:

import pandas as pd
import numpy as np
import xgboost as xgb
from sklearn.model_selection import train_test_split
from sklearn.metrics import roc_auc_score, classification_report, confusion_matrix
import shap
import matplotlib.pyplot as plt
import boto3
import json

# ============================================================
# STEP 2: Build the Credit Risk Model
# ============================================================

# Load data
df = pd.read_csv("credit_risk_dataset.csv")

# Handle missing values
df["loan_int_rate"].fillna(df["loan_int_rate"].median(), inplace=True)
df["person_emp_length"].fillna(df["person_emp_length"].median(), inplace=True)

# Remove outliers
df = df[df["person_age"] <= 80]
df = df[df["person_emp_length"] <= 60]

# Separate features and target
target = "loan_status"
categorical_cols = ["person_home_ownership", "loan_intent", "loan_grade", "cb_person_default_on_file"]

# One-hot encode categorical features
df_encoded = pd.get_dummies(df, columns=categorical_cols, drop_first=True)

X = df_encoded.drop(columns=[target])
y = df_encoded[target]

# Train/test split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

# Calculate scale_pos_weight for class imbalance
scale_pos_weight = len(y_train[y_train == 0]) / len(y_train[y_train == 1])

# Train XGBoost model
model = xgb.XGBClassifier(
    objective="binary:logistic",
    eval_metric="auc",
    scale_pos_weight=scale_pos_weight,
    max_depth=5,
    learning_rate=0.05,
    n_estimators=300,
    subsample=0.8,
    colsample_bytree=0.8,
    min_child_weight=3,
    gamma=0.1,
    reg_alpha=0.1,
    reg_lambda=1.0,
    random_state=42,
    early_stopping_rounds=20,
)

model.fit(
    X_train, y_train,
    eval_set=[(X_test, y_test)],
    verbose=False
)

# Evaluate
y_pred_proba = model.predict_proba(X_test)[:, 1]
y_pred = (y_pred_proba >= 0.5).astype(int)

print("=== Model Performance ===")
print(f"AUC-ROC: {roc_auc_score(y_test, y_pred_proba):.4f}")
print(f"\nClassification Report:\n{classification_report(y_test, y_pred)}")
print(f"Confusion Matrix:\n{confusion_matrix(y_test, y_pred)}")

# ============================================================
# STEP 3: Explain Predictions with SHAP
# ============================================================

# Create SHAP explainer
explainer = shap.Explainer(model)
shap_values = explainer(X_test)

# Global feature importance
shap.plots.bar(shap_values, max_display=10, show=False)
plt.tight_layout()
plt.savefig("feature_importance.png", dpi=150, bbox_inches="tight")
plt.close()
print("\nSaved global feature importance plot to feature_importance.png")

# Find high-risk and low-risk applicants
high_risk_idx = np.argmax(y_pred_proba)
low_risk_idx = np.argmin(y_pred_proba)

# Waterfall plot for high-risk applicant
shap.plots.waterfall(shap_values[high_risk_idx], show=False)
plt.tight_layout()
plt.savefig("high_risk_waterfall.png", dpi=150, bbox_inches="tight")
plt.close()
print("Saved high-risk applicant waterfall plot to high_risk_waterfall.png")

# Waterfall plot for low-risk applicant
shap.plots.waterfall(shap_values[low_risk_idx], show=False)
plt.tight_layout()
plt.savefig("low_risk_waterfall.png", dpi=150, bbox_inches="tight")
plt.close()
print("Saved low-risk applicant waterfall plot to low_risk_waterfall.png")


def format_shap_explanation(shap_values_single, feature_names, risk_score):
    """Format SHAP values into structured explanation for LLM input."""
    values = shap_values_single.values
    feature_contributions = list(zip(feature_names, values))
    sorted_contributions = sorted(feature_contributions, key=lambda x: abs(x[1]), reverse=True)

    positive_factors = [(name, val) for name, val in sorted_contributions if val > 0][:3]
    negative_factors = [(name, val) for name, val in sorted_contributions if val < 0][:3]

    return {
        "risk_score": round(float(risk_score), 4),
        "top_positive_factors": positive_factors,
        "top_negative_factors": negative_factors,
    }


# Generate explanations for sample applicants
high_risk_explanation = format_shap_explanation(
    shap_values[high_risk_idx], X_test.columns.tolist(), y_pred_proba[high_risk_idx]
)
low_risk_explanation = format_shap_explanation(
    shap_values[low_risk_idx], X_test.columns.tolist(), y_pred_proba[low_risk_idx]
)

print(f"\nHigh-risk applicant explanation: {json.dumps(high_risk_explanation, indent=2, default=str)}")
print(f"\nLow-risk applicant explanation: {json.dumps(low_risk_explanation, indent=2, default=str)}")

# ============================================================
# STEP 4: Generate Risk Reports with Amazon Bedrock
# ============================================================

# Create Bedrock client
bedrock_client = boto3.client("bedrock-runtime", region_name="us-east-1")

SYSTEM_PROMPT = """You are a senior credit risk analyst at a financial institution. 
Your role is to write professional, concise risk assessment reports based on model predictions and explainability data.
Each report should include: Risk Level, Key Findings, Recommendation (Approve/Review/Decline), and any Conditions.
Use clear, professional language suitable for a credit committee."""


def generate_risk_report(explanation, bedrock_client):
    """Generate a risk assessment report using Amazon Bedrock Nova Micro."""
    risk_pct = explanation["risk_score"] * 100

    positive_factors_text = "\n".join(
        [f"  - {name}: contribution +{val:.3f}" for name, val in explanation["top_positive_factors"]]
    )
    negative_factors_text = "\n".join(
        [f"  - {name}: contribution {val:.3f}" for name, val in explanation["top_negative_factors"]]
    )

    user_prompt = f"""Generate a risk assessment report for this loan applicant:

Probability of Default: {risk_pct:.1f}%

Top Risk Factors (increasing default probability):
{positive_factors_text}

Top Protective Factors (decreasing default probability):
{negative_factors_text}

Please provide a structured risk assessment report."""

    response = bedrock_client.converse(
        modelId="amazon.nova-micro-v1:0",
        messages=[{"role": "user", "content": [{"text": user_prompt}]}],
        system=[{"text": SYSTEM_PROMPT}],
        inferenceConfig={"maxTokens": 512, "temperature": 0.3, "topP": 0.9},
    )

    return response["output"]["message"]["content"][0]["text"]


# Generate reports
print("\n" + "=" * 60)
print("RISK ASSESSMENT REPORT - HIGH RISK APPLICANT")
print("=" * 60)
high_risk_report = generate_risk_report(high_risk_explanation, bedrock_client)
print(high_risk_report)

print("\n" + "=" * 60)
print("RISK ASSESSMENT REPORT - LOW RISK APPLICANT")
print("=" * 60)
low_risk_report = generate_risk_report(low_risk_explanation, bedrock_client)
print(low_risk_report)

# Save reports
with open("high_risk_report.txt", "w") as f:
    f.write(high_risk_report)
with open("low_risk_report.txt", "w") as f:
    f.write(low_risk_report)

print("\nReports saved to high_risk_report.txt and low_risk_report.txt")

Your credit risk system is now complete. You have a trained model that predicts defaults, SHAP explanations that satisfy regulatory transparency, and an LLM that writes professional reports automatically. Next up, the Secret Mission challenges you to scale this into a batch pipeline that processes multiple applicants at once.

Secret mission

Build a Batch Risk Assessment Pipeline

In real credit operations, analysts don't score one applicant at a time. They process entire batches and generate summary dashboards. Can you build a pipeline that scores multiple applicants, explains each prediction, and produces a formatted summary table?

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project uses Amazon Bedrock on-demand pricing, so there are no ongoing costs when you stop running the scripts.

Resources you used:

  • Project folder on your Desktop (Python files, CSV data, output images, text reports, virtual environment).
  • AWS CLI credentials stored locally (~/.aws/credentials and ~/.aws/config).
  • Amazon Bedrock model access (Nova Micro enabled in your AWS account).

Keep everything running

No action needed. Choose this if you plan to continue experimenting with the credit risk model or building on the batch pipeline.

  • Your project files remain on your Desktop.
  • Your AWS CLI credentials stay configured for future projects.
  • Bedrock model access stays enabled at no cost when idle.

Pause - I'll come back to this later

Free up terminal processes while keeping your files and configuration intact.

  • If your virtual environment is still active, deactivate it by running:
  • deactivate in your terminal.
  • Your project files, AWS credentials, and Bedrock access all remain in place for when you return.

Delete - I don't want to use this again

Remove all project resources and local configuration created during this project.

  • Delete your project folder by running this command in your terminal:

macOS

rm -rf ~/Desktop/credit-risk-scoring

Windows

Remove-Item -Recurse -Force "$env:USERPROFILE\Desktop\credit-risk-scoring"
  • Remove your AWS CLI credentials by deleting the configuration files:

macOS

rm ~/.aws/credentials
rm ~/.aws/config

Windows

Remove-Item "$env:USERPROFILE\.aws\credentials"
Remove-Item "$env:USERPROFILE\.aws\config"

Only remove AWS credentials if this is your only project

If you use AWS for other projects, skip deleting these files. They store credentials for all your AWS CLI work, not just this project.

  • Revoke Bedrock model access by opening the Amazon Bedrock console at https://console.aws.amazon.com/bedrock.
  • In the left navigation pane, under Bedrock configurations, choose Model access.
  • Choose Modify model access.
  • Deselect Amazon Nova Micro and submit your changes.

Nice Work!

Nice Work!

Nice work! You just built a complete credit risk scoring system that combines traditional machine learning with generative AI.

You've learned how to:

  • Train an XGBoost credit risk model on 32,581 loan records and evaluate it with AUC-ROC and classification metrics.
  • Implement SHAP-based model explainability with global feature importance plots and individual waterfall explanations for model governance.
  • Integrate Amazon Bedrock (Nova Micro) via the Converse API to auto-generate professional risk assessment reports from model predictions.
  • Secret Mission: Built a batch processing pipeline that scores multiple applicants, generates AI reports with retry logic, and outputs a summary dashboard as a CSV.

Ready to quiz yourself?