Predict Machine Failures with ML
Build a predictive maintenance ML pipeline using real industrial sensor data.
Introduction
30 Second Summary
Every factory has machines that break down without warning. A single unexpected failure can halt production for hours and cost thousands of dollars. What if you could predict that failure before it happens, turning a catastrophic breakdown into a cheap scheduled repair?
In this project, you will build a machine learning system that predicts equipment failures using real industrial sensor data. You will engineer physics-inspired features, handle severely imbalanced data, compare multiple models, and explain your predictions using SHAP.
What You'll Build
A complete predictive maintenance pipeline in a Google Colab notebook that processes 10,000 machine sensor records, engineers domain-driven features, and predicts failures before they happen.
By the end of this project, you'll have:
- A working ML pipeline that loads real sensor data, applies SMOTE for class imbalance, and trains multiple classifiers using scikit-learn.
- Physics-inspired engineered features (power output, strain index, thermal gradient) that outperform raw sensor readings.
- SHAP visualizations explaining why the model predicts each failure, connecting predictions back to real physical failure mechanisms.
- Secret Mission: Build a multi-class model that predicts which type of failure will occur, not just whether one will happen.
Are there any prerequisites?
You should be comfortable with basic Python (variables, loops, functions) and have some familiarity with basic statistics. No prior machine learning experience is required.
Before We Start
Before We Start...
Set Up and Load the Data
Before you can build a model that predicts machine failures, you need a workspace with the right tools and real industrial data to work with.
In this step, you'll set up a Google Colab notebook with all the libraries your project needs, then load the UCI AI4I 2020 Predictive Maintenance dataset directly into a DataFrame.
In this step, get ready to:
- Install and import all required Python libraries.
- Load the UCI AI4I 2020 dataset into your notebook.
- Confirm the dataset's shape and class distribution.
Install the required libraries
Google Colab comes with scikit-learn, pandas, matplotlib, and seaborn pre-installed. You only need to install three additional packages: ucimlrepo (to fetch datasets from the UCI Machine Learning Repository), imbalanced-learn (for handling class imbalance), and shap (for model explainability).
- Install the three packages by running this command in a new code cell:
!pip install ucimlrepo imbalanced-learn shapWhat does this command do?
The ! prefix tells Colab to run this as a terminal command instead of Python code.
It installs ucimlrepo for loading UCI datasets, imbalanced-learn for SMOTE oversampling, and shap for generating model explanations.
- Wait for the installation to complete. You should see output ending with lines like "Successfully installed" followed by package names and version numbers.
Seeing an error during installation?
- If you see dependency conflict warnings (yellow text), these are usually safe to ignore as long as the final line says "Successfully installed".
- If the install fails entirely, try restarting your Colab runtime (click **Runtime** then **Restart runtime**) and run the install cell again.
Why is my pip install failing in Google Colab?
Import all project libraries
Now that the packages are installed, you need to import everything your project will use. This single import cell loads data manipulation tools, visualization libraries, machine learning models, and explainability utilities.
- Create a new code cell and paste the following imports:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split, StratifiedKFold, RandomizedSearchCV
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.metrics import classification_report, confusion_matrix, ConfusionMatrixDisplay, precision_recall_curve, auc
from sklearn.dummy import DummyClassifier
from imblearn.over_sampling import SMOTE
import shap
import warnings
warnings.filterwarnings('ignore')What does this code do?
- numpy and pandas handle numerical operations and tabular data.
- matplotlib and seaborn create charts and visualizations.
- The sklearn imports bring in model training, preprocessing, evaluation, and cross-validation tools.
- SMOTE from imblearn generates synthetic samples to balance an imbalanced dataset.
- shap explains individual model predictions using game-theory-based feature attribution.
- warnings.filterwarnings('ignore') suppresses non-critical warnings so your notebook output stays clean.
- Run the cell. If everything is installed correctly, the cell completes with no output and no errors.
Seeing a ModuleNotFoundError?
- If you see an error mentioning imblearn or shap, your install cell may not have run successfully. Go back to the install cell, re-run it, then try the imports again.
- If imports still fail after reinstalling, click **Runtime** then **Restart runtime**, re-run the install cell, then re-run the import cell.
Why am I getting ModuleNotFoundError in Colab after installing?
Load and explore the dataset
The UCI AI4I 2020 Predictive Maintenance dataset contains 10,000 synthetic records modeled after real industrial sensor readings. Each row represents one machine operation with measurements like temperature, rotational speed, torque, and tool wear.
- Load the dataset and examine its structure by running this code in a new cell:
# Load the dataset
from ucimlrepo import fetch_ucirepo
dataset = fetch_ucirepo(id=601)
X = dataset.data.features
y = dataset.data.targets
# Combine into a single DataFrame for exploration
df = pd.concat([X, y], axis=1)
print(f"Dataset shape: {df.shape}")
print(f"\nColumn names: {df.columns.tolist()}")
print(f"\nFirst 5 rows:")
print(df.head())What does this code do?
- fetch_ucirepo(id=601) downloads the AI4I 2020 Predictive Maintenance dataset directly from the UCI Machine Learning Repository.
- dataset.data.features contains the sensor readings (air temperature, process temperature, rotational speed, torque, tool wear, and product type).
- dataset.data.targets contains the failure labels (Machine failure and individual failure mode columns).
- pd.concat([X, y], axis=1) merges features and targets into one DataFrame so you can explore everything together.
- You should see output showing a shape of (10000, 8) or similar, column names including Air temperature [K], Process temperature [K], Rotational speed [rpm], Torque [Nm], Tool wear [min], and the first five rows of data.
Seeing a connection error from fetch_ucirepo?
- The ucimlrepo package occasionally has SSL or connection issues. If fetching fails, try running the cell again after a few seconds.
- If repeated attempts fail, you can download the dataset CSV directly from the UCI Machine Learning Repository website for dataset ID 601 and load it with pd.read_csv() instead.
Why is fetch_ucirepo failing with a connection error?
The dataset has no missing values, which means you can skip data cleaning and move straight to checking the target variable.
- Check the class distribution of the Machine failure column by running this code in a new cell:
# Check class distribution
print(f"\nClass distribution:")
print(df['Machine failure'].value_counts())
print(f"\nFailure rate: {df['Machine failure'].mean()*100:.1f}%")What does this code do?
- value_counts() shows how many records belong to each class (0 = no failure, 1 = failure).
- The failure rate calculation reveals the class imbalance problem. Only about 3.4% of records are failures, which means a naive model could "cheat" by always predicting no failure and still look accurate.
- You should see approximately 9,661 records with no failure (class 0) and 339 records with failure (class 1), confirming the 3.4% failure rate.
Why does class imbalance matter?
With only 3.4% of records showing failures, a model that always predicts "no failure" would be 96.6% accurate. That sounds great on paper, but it catches zero actual failures.
This is exactly the trap that makes predictive maintenance challenging. You'll learn how to handle this imbalance in a later step using SMOTE and recall-focused evaluation.
✔️ Awesome, I've got everything!
Great. Double check you've saved your notebook by pressing **Cmd+S** (macOS) or **Ctrl+S** (Windows).
ⓧ I'd like to double check the full code
Here is the complete code you should have across your cells at this point:
# =============================================================================
# Predictive Maintenance: Predict Machine Failures Before They Happen
# =============================================================================
# --- Step 1: Setup and Data Loading ---
# Install required packages (run in Colab)
# !pip install ucimlrepo imbalanced-learn shap
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split, StratifiedKFold, RandomizedSearchCV
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.metrics import classification_report, confusion_matrix, ConfusionMatrixDisplay, precision_recall_curve, auc
from sklearn.dummy import DummyClassifier
from imblearn.over_sampling import SMOTE
import shap
import warnings
warnings.filterwarnings('ignore')
# Load the dataset
from ucimlrepo import fetch_ucirepo
dataset = fetch_ucirepo(id=601)
X = dataset.data.features
y = dataset.data.targets
# Combine into a single DataFrame for exploration
df = pd.concat([X, y], axis=1)
print(f"Dataset shape: {df.shape}")
print(f"\nColumn names: {df.columns.tolist()}")
print(f"\nFirst 5 rows:")
print(df.head())
# Check class distribution
print(f"\nClass distribution:")
print(df['Machine failure'].value_counts())
print(f"\nFailure rate: {df['Machine failure'].mean()*100:.1f}%")Your data is loaded and you've already spotted the core challenge of this project: severe class imbalance. Next up, you'll visualize the data and engineer physics-inspired features that actually capture how machines fail.
Explore Data and Engineer Features
Your dataset is loaded and you've confirmed the challenge: only 3.4% of records are machine failures. Now it's time to understand what the data is actually telling you before throwing it at a model.
In this step, you'll explore the sensor readings visually, then create new features inspired by the physics of how machines actually break. Raw sensor values alone often miss the real story. Combining them in domain-meaningful ways can reveal failure patterns that a model would never discover on its own.
In this step, get ready to:
- Visualize feature distributions and correlations.
- Engineer three physics-inspired features from raw sensor data.
- Validate that engineered features separate failure cases better than raw readings.
Visualize feature distributions and relationships
Before building any model, you need to see how the sensor readings behave for machines that fail versus machines that don't. This tells you which features already carry a failure signal and which need transformation.
- In a new code cell, define the feature column lists and plot histograms split by failure class by pasting this code:
# --- Step 2: EDA and Feature Engineering ---
# Separate numerical and categorical features
numerical_cols = ['Air temperature [K]', 'Process temperature [K]',
'Rotational speed [rpm]', 'Torque [Nm]', 'Tool wear [min]']
categorical_col = 'Type'
# Plot feature distributions by failure class
fig, axes = plt.subplots(2, 3, figsize=(15, 10))
for idx, col in enumerate(numerical_cols):
ax = axes[idx // 3, idx % 3]
df[df['Machine failure'] == 0][col].hist(ax=ax, alpha=0.5, label='No Failure', bins=30)
df[df['Machine failure'] == 1][col].hist(ax=ax, alpha=0.5, label='Failure', bins=30)
ax.set_title(col)
ax.legend()
axes[1, 2].axis('off')
plt.tight_layout()
plt.show()What does this code do?
The code defines two variables that group your columns by type. numerical_cols holds the five sensor readings. categorical_col holds the product quality type (L, M, or H).
The histogram loop creates one subplot per numerical feature, plotting the distribution for no-failure samples (blue) overlaid with failure samples (orange). This lets you visually compare where the two classes overlap and where they separate.
- Run the cell.
- You should see a 2x3 grid of histograms. Look for features where the orange (failure) distribution shifts noticeably away from the blue (no failure) distribution.
Seeing a NameError for df or plt?
Make sure you ran all cells from Step 1 first. Colab doesn't automatically re-run previous cells when you open a notebook. Go to Runtime then Run all to execute everything in order.
help me fix a NameError in my Colab notebook
Histograms show individual feature distributions, but you also need to understand how features relate to each other. A correlation heatmap reveals linear relationships between all pairs of features at a glance.
- In a new code cell, generate the correlation heatmap by pasting this code:
# Correlation heatmap
plt.figure(figsize=(10, 8))
sns.heatmap(df[numerical_cols + ['Machine failure']].corr(), annot=True, cmap='coolwarm', center=0)
plt.title('Feature Correlation Heatmap')
plt.tight_layout()
plt.show()What does the heatmap reveal?
Each cell shows the Pearson correlation between two features. Values near +1 or -1 mean a strong linear relationship. Values near 0 mean little linear connection.
You'll likely notice that no single feature has a strong correlation with Machine failure. This is exactly why feature engineering matters. The raw sensors alone don't capture the failure mechanisms cleanly.
- Run the cell.
- You should see a colored grid with correlation values annotated in each cell. Notice that Torque [Nm] and Rotational speed [rpm] are negatively correlated. This is a known physical relationship in electric motors.
Engineer physics-inspired features
The raw sensor columns don't directly map to how machines actually break. The UCI dataset documentation describes specific failure thresholds based on combinations of sensor values, not individual readings.
Why engineer features instead of letting the model figure it out?
Tree-based models can learn interactions between features, but you're making their job much easier by encoding known physics directly. For example, overstrain failure triggers when tool_wear * torque exceeds a threshold. Creating strain_index gives the model this relationship as a single, clear signal.
This is the difference between data science and just running code. Domain knowledge turns a mediocre model into one that catches real failures.
- In a new code cell, create three physics-inspired features by pasting this code:
# Engineer physics-inspired features
df['power'] = df['Torque [Nm]'] * (df['Rotational speed [rpm]'] * 2 * np.pi / 60)
df['thermal_gradient'] = df['Process temperature [K]'] - df['Air temperature [K]']
df['strain_index'] = df['Tool wear [min]'] * df['Torque [Nm]']What do these features represent?
- power: torque multiplied by angular velocity (RPM converted to rad/s). This is mechanical power in Watts. Power failure triggers when this falls below 3500 W or exceeds 9000 W.
- thermal_gradient: the temperature difference between the process and the surrounding air. Heat dissipation failure triggers when this drops below 8.6 K while rotational speed is low.
- strain_index: tool wear multiplied by torque. Overstrain failure triggers when this exceeds quality-specific thresholds (11,000 for L, 12,000 for M, 13,000 for H variants).
- Run the cell.
- You won't see visible output yet. The next chunk uses these features to produce plots that confirm they carry a stronger failure signal.
Validate engineered features and encode categories
You've created the features, but you need to prove they actually help. Plotting the engineered feature distributions by failure class shows whether they separate the two classes better than raw sensor values did.
- In a new code cell, visualize the engineered features and one-hot encode the Type column by pasting this code:
# Visualize engineered features by class
fig, axes = plt.subplots(1, 3, figsize=(15, 5))
engineered_features = ['power', 'thermal_gradient', 'strain_index']
for idx, col in enumerate(engineered_features):
ax = axes[idx]
df[df['Machine failure'] == 0][col].hist(ax=ax, alpha=0.5, label='No Failure', bins=30)
df[df['Machine failure'] == 1][col].hist(ax=ax, alpha=0.5, label='Failure', bins=30)
ax.set_title(f'{col} by class')
ax.legend()
plt.tight_layout()
plt.show()
# One-hot encode the Type column
df = pd.get_dummies(df, columns=['Type'], prefix='type')What does this code do?
The first section plots histograms for each engineered feature, split by failure class. You should see that strain_index and power show much clearer separation between failure and no-failure than the raw torque or speed histograms did earlier.
The last line converts the categorical Type column (L, M, H quality variants) into three binary columns: type_H, type_L, and type_M. This is called one-hot encoding. ML models need numerical inputs, so each category becomes its own 0/1 column.
- Run the cell.
- You should see a row of three histograms. Compare strain_index here to the raw Torque [Nm] histogram from earlier. The failure class (orange) should be more visibly separated in the engineered version.
Seeing a KeyError for the Type column?
If you ran this cell more than once, the Type column no longer exists because it was already converted to type_H, type_L, type_M. Re-run your notebook from the top to reset the DataFrame, or skip the encoding line if the columns already exist.
help me fix a KeyError with get_dummies
✔️ Awesome, I've got everything!
Great. Double check you've saved your notebook with all cells run in order.
ⓧ I'd like to double check the full code
# =============================================================================
# Predictive Maintenance: Predict Machine Failures Before They Happen
# =============================================================================
# --- Step 1: Setup and Data Loading ---
# Install required packages (run in Colab)
# !pip install ucimlrepo imbalanced-learn shap
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split, StratifiedKFold, RandomizedSearchCV
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.metrics import classification_report, confusion_matrix, ConfusionMatrixDisplay, precision_recall_curve, auc
from sklearn.dummy import DummyClassifier
from imblearn.over_sampling import SMOTE
import shap
import warnings
warnings.filterwarnings('ignore')
# Load the dataset
from ucimlrepo import fetch_ucirepo
dataset = fetch_ucirepo(id=601)
X = dataset.data.features
y = dataset.data.targets
# Combine into a single DataFrame for exploration
df = pd.concat([X, y], axis=1)
print(f"Dataset shape: {df.shape}")
print(f"\nColumn names: {df.columns.tolist()}")
print(f"\nFirst 5 rows:")
print(df.head())
# Check class distribution
print(f"\nClass distribution:")
print(df['Machine failure'].value_counts())
print(f"\nFailure rate: {df['Machine failure'].mean()*100:.1f}%")
# --- Step 2: EDA and Feature Engineering ---
# Separate numerical and categorical features
numerical_cols = ['Air temperature [K]', 'Process temperature [K]',
'Rotational speed [rpm]', 'Torque [Nm]', 'Tool wear [min]']
categorical_col = 'Type'
# Plot feature distributions by failure class
fig, axes = plt.subplots(2, 3, figsize=(15, 10))
for idx, col in enumerate(numerical_cols):
ax = axes[idx // 3, idx % 3]
df[df['Machine failure'] == 0][col].hist(ax=ax, alpha=0.5, label='No Failure', bins=30)
df[df['Machine failure'] == 1][col].hist(ax=ax, alpha=0.5, label='Failure', bins=30)
ax.set_title(col)
ax.legend()
axes[1, 2].axis('off')
plt.tight_layout()
plt.show()
# Correlation heatmap
plt.figure(figsize=(10, 8))
sns.heatmap(df[numerical_cols + ['Machine failure']].corr(), annot=True, cmap='coolwarm', center=0)
plt.title('Feature Correlation Heatmap')
plt.tight_layout()
plt.show()
# Engineer physics-inspired features
df['power'] = df['Torque [Nm]'] * (df['Rotational speed [rpm]'] * 2 * np.pi / 60)
df['thermal_gradient'] = df['Process temperature [K]'] - df['Air temperature [K]']
df['strain_index'] = df['Tool wear [min]'] * df['Torque [Nm]']
# Visualize engineered features by class
fig, axes = plt.subplots(1, 3, figsize=(15, 5))
engineered_features = ['power', 'thermal_gradient', 'strain_index']
for idx, col in enumerate(engineered_features):
ax = axes[idx]
df[df['Machine failure'] == 0][col].hist(ax=ax, alpha=0.5, label='No Failure', bins=30)
df[df['Machine failure'] == 1][col].hist(ax=ax, alpha=0.5, label='Failure', bins=30)
ax.set_title(f'{col} by class')
ax.legend()
plt.tight_layout()
plt.show()
# One-hot encode the Type column
df = pd.get_dummies(df, columns=['Type'], prefix='type')Your dataset now has three new features that encode real physics about how machines fail. Next up, you'll tackle the biggest challenge in this dataset: the severe class imbalance. With only 3.4% failures, a naive model could just predict "no failure" every time and still look 96.6% accurate.
Handle Class Imbalance and Build the Pipeline
In the previous step, you explored the data and engineered features that capture the physics of machine failure. Now you need to turn that prepared DataFrame into a proper ML training setup.
There is a hidden trap waiting for you. Only 3.4% of your data represents failures. A model that simply says "no failure" every single time would be 96.6% accurate. In this step, you will handle that imbalance, build a reproducible preprocessing pipeline, and train your first model with metrics that actually matter.
In this step, get ready to:
- Split data into training and test sets while preserving the class ratio.
- Apply SMOTE to synthetically balance the training data and build a scikit-learn Pipeline.
- Train a Logistic Regression baseline and prove that accuracy is a misleading metric.
Split the data into training and test sets
Before training any model, you need to separate your data into features (inputs) and a target (what you want to predict). You also need to hold out a test set that the model never sees during training.
- In your Colab notebook, create a new code cell and add the following code to prepare your features and split the data:
# --- Step 3: Handle Class Imbalance and Build Pipeline ---
# Prepare features and target
feature_cols = numerical_cols + engineered_features + ['type_H', 'type_L', 'type_M']
target_col = 'Machine failure'
X = df[feature_cols].copy()
y = df[target_col].copy()
# Train/test split with stratification
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y
)
print(f"Training set: {X_train.shape[0]} samples")
print(f"Test set: {X_test.shape[0]} samples")
print(f"Training failure rate: {y_train.mean()*100:.1f}%")What does this code do?
feature_cols combines your original sensor readings, engineered features, and one-hot encoded quality columns into a single list of input features.
stratify=y ensures the 3.4% failure ratio is preserved in both train and test sets. Without this, you might accidentally put all failures into one set.
- Run the cell. You should see the training set has 8,000 samples and the test set has 2,000 samples, with the failure rate preserved at approximately 3.4%.
Seeing a KeyError?
If you get an error about missing columns, make sure you ran the one-hot encoding cell from Step 2 (pd.get_dummies). The columns type_H, type_L, and type_M only exist after encoding.
Help me debug a KeyError in my train/test split
Apply SMOTE and build the preprocessing pipeline
Your training set has roughly 7,700 "no failure" samples and only about 270 "failure" samples. If you train a model on this, it will learn to always predict "no failure" because that is the safe bet. SMOTE (Synthetic Minority Over-sampling Technique) fixes this by generating synthetic failure examples until both classes are balanced.
- Add a new code cell and apply SMOTE to your training data:
# Apply SMOTE to training data only
smote = SMOTE(random_state=42)
X_train_resampled, y_train_resampled = smote.fit_resample(X_train, y_train)
print(f"\nAfter SMOTE:")
print(f"Training samples: {X_train_resampled.shape[0]}")
print(f"Class distribution: {pd.Series(y_train_resampled).value_counts().to_dict()}")Why apply SMOTE to training data only?
SMOTE creates synthetic samples by interpolating between existing minority-class examples. You only do this on the training set because your test set must represent the real world. If you oversample the test set, your evaluation metrics would be meaningless.
- Run the cell. You should see roughly 15,400 total training samples with both classes at approximately 7,700 each.
Now you need a preprocessing pipeline that scales your numerical features while leaving the one-hot encoded columns untouched.
- Add another code cell to build the preprocessing ColumnTransformer:
# Build preprocessing pipeline
numerical_features = numerical_cols + engineered_features
categorical_features = ['type_H', 'type_L', 'type_M']
preprocessor = ColumnTransformer(
transformers=[
('num', StandardScaler(), numerical_features),
('cat', 'passthrough', categorical_features)
]
)What is a ColumnTransformer?
A ColumnTransformer lets you apply different transformations to different columns. Here, StandardScaler normalizes numerical features (temperature, speed, power) to the same scale. The one-hot encoded columns (type_H, type_L, type_M) are already 0/1, so they pass through unchanged.
Why use a Pipeline?
A Pipeline chains your preprocessor and classifier into a single object. This prevents data leakage because the scaler is fitted only on training data within each cross-validation fold. It also makes your workflow reproducible.
- Run the cell. No output is expected here. The preprocessor is defined and ready to use in the next chunk.
Train your first model with Logistic Regression
Before training a real model, you will first prove WHY accuracy is a terrible metric for imbalanced data. A dummy classifier that always predicts "no failure" will score over 96% accuracy. Then you will train a real baseline and use classification_report to see the metrics that actually matter.
- Add a new code cell to train the dummy classifier and Logistic Regression baseline:
# Baseline: Dummy classifier (always predicts majority class)
dummy = DummyClassifier(strategy='most_frequent')
dummy.fit(X_train, y_train)
print(f"\nDummy classifier accuracy: {dummy.score(X_test, y_test)*100:.1f}%")
print("(This proves accuracy is USELESS for imbalanced data)\n")
# Logistic Regression baseline
lr_pipeline = Pipeline([
('preprocessor', preprocessor),
('classifier', LogisticRegression(max_iter=1000, random_state=42))
])
lr_pipeline.fit(X_train_resampled, y_train_resampled)
y_pred_lr = lr_pipeline.predict(X_test)
print("=== Logistic Regression ===")
print(classification_report(y_test, y_pred_lr, target_names=['No Failure', 'Failure']))What does this code do?
The DummyClassifier with strategy='most_frequent' always predicts "no failure." It scores approximately 96.6% accuracy. This proves that accuracy is completely useless when one class dominates.
The lr_pipeline chains the preprocessor with a Logistic Regression classifier. It trains on the SMOTE-balanced data but predicts on the original imbalanced test set. The classification_report shows precision, recall, and F1-score for each class separately.
- Run the cell. You should see the dummy classifier hitting approximately 96.6% accuracy, followed by the Logistic Regression classification report.
How do I read the classification report?
- Precision: of all predictions labeled "Failure," what percentage actually were failures. High precision means fewer false alarms.
- Recall: of all actual failures, what percentage did the model catch. High recall means fewer missed failures.
- F1-score: the harmonic mean of precision and recall. A single number balancing both.
For predictive maintenance, recall for the Failure class is the metric that matters most. A missed failure means a machine breaks down. A false alarm just means an unnecessary inspection.
Seeing all zeros for the Failure class?
If your classification report shows 0.00 for all Failure metrics, you may have trained on the original imbalanced data instead of the SMOTE-resampled data. Make sure you are using X_train_resampled and y_train_resampled in the .fit() call.
Help me debug my Logistic Regression classification report
✔️ Awesome, I've got everything!
Great. Double check you have saved your notebook.
ⓧ I'd like to double check the full code
Here is the complete code for your notebook up to and including this step:
# =============================================================================
# Predictive Maintenance: Predict Machine Failures Before They Happen
# =============================================================================
# --- Step 1: Setup and Data Loading ---
# Install required packages (run in Colab)
# !pip install ucimlrepo imbalanced-learn shap
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split, StratifiedKFold, RandomizedSearchCV
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.metrics import classification_report, confusion_matrix, ConfusionMatrixDisplay, precision_recall_curve, auc
from sklearn.dummy import DummyClassifier
from imblearn.over_sampling import SMOTE
import shap
import warnings
warnings.filterwarnings('ignore')
# Load the dataset
from ucimlrepo import fetch_ucirepo
dataset = fetch_ucirepo(id=601)
X = dataset.data.features
y = dataset.data.targets
# Combine into a single DataFrame for exploration
df = pd.concat([X, y], axis=1)
print(f"Dataset shape: {df.shape}")
print(f"\nColumn names: {df.columns.tolist()}")
print(f"\nFirst 5 rows:")
print(df.head())
# Check class distribution
print(f"\nClass distribution:")
print(df['Machine failure'].value_counts())
print(f"\nFailure rate: {df['Machine failure'].mean()*100:.1f}%")
# --- Step 2: EDA and Feature Engineering ---
# Separate numerical and categorical features
numerical_cols = ['Air temperature [K]', 'Process temperature [K]',
'Rotational speed [rpm]', 'Torque [Nm]', 'Tool wear [min]']
categorical_col = 'Type'
# Plot feature distributions by failure class
fig, axes = plt.subplots(2, 3, figsize=(15, 10))
for idx, col in enumerate(numerical_cols):
ax = axes[idx // 3, idx % 3]
df[df['Machine failure'] == 0][col].hist(ax=ax, alpha=0.5, label='No Failure', bins=30)
df[df['Machine failure'] == 1][col].hist(ax=ax, alpha=0.5, label='Failure', bins=30)
ax.set_title(col)
ax.legend()
axes[1, 2].axis('off')
plt.tight_layout()
plt.show()
# Correlation heatmap
plt.figure(figsize=(10, 8))
sns.heatmap(df[numerical_cols + ['Machine failure']].corr(), annot=True, cmap='coolwarm', center=0)
plt.title('Feature Correlation Heatmap')
plt.tight_layout()
plt.show()
# Engineer physics-inspired features
df['power'] = df['Torque [Nm]'] * (df['Rotational speed [rpm]'] * 2 * np.pi / 60)
df['thermal_gradient'] = df['Process temperature [K]'] - df['Air temperature [K]']
df['strain_index'] = df['Tool wear [min]'] * df['Torque [Nm]']
# Visualize engineered features by class
fig, axes = plt.subplots(1, 3, figsize=(15, 5))
engineered_features = ['power', 'thermal_gradient', 'strain_index']
for idx, col in enumerate(engineered_features):
ax = axes[idx]
df[df['Machine failure'] == 0][col].hist(ax=ax, alpha=0.5, label='No Failure', bins=30)
df[df['Machine failure'] == 1][col].hist(ax=ax, alpha=0.5, label='Failure', bins=30)
ax.set_title(f'{col} by class')
ax.legend()
plt.tight_layout()
plt.show()
# One-hot encode the Type column
df = pd.get_dummies(df, columns=['Type'], prefix='type')
# --- Step 3: Handle Class Imbalance and Build Pipeline ---
# Prepare features and target
feature_cols = numerical_cols + engineered_features + ['type_H', 'type_L', 'type_M']
target_col = 'Machine failure'
X = df[feature_cols].copy()
y = df[target_col].copy()
# Train/test split with stratification
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y
)
print(f"Training set: {X_train.shape[0]} samples")
print(f"Test set: {X_test.shape[0]} samples")
print(f"Training failure rate: {y_train.mean()*100:.1f}%")
# Apply SMOTE to training data only
smote = SMOTE(random_state=42)
X_train_resampled, y_train_resampled = smote.fit_resample(X_train, y_train)
print(f"\nAfter SMOTE:")
print(f"Training samples: {X_train_resampled.shape[0]}")
print(f"Class distribution: {pd.Series(y_train_resampled).value_counts().to_dict()}")
# Build preprocessing pipeline
numerical_features = numerical_cols + engineered_features
categorical_features = ['type_H', 'type_L', 'type_M']
preprocessor = ColumnTransformer(
transformers=[
('num', StandardScaler(), numerical_features),
('cat', 'passthrough', categorical_features)
]
)
# Baseline: Dummy classifier (always predicts majority class)
dummy = DummyClassifier(strategy='most_frequent')
dummy.fit(X_train, y_train)
print(f"\nDummy classifier accuracy: {dummy.score(X_test, y_test)*100:.1f}%")
print("(This proves accuracy is USELESS for imbalanced data)\n")
# Logistic Regression baseline
lr_pipeline = Pipeline([
('preprocessor', preprocessor),
('classifier', LogisticRegression(max_iter=1000, random_state=42))
])
lr_pipeline.fit(X_train_resampled, y_train_resampled)
y_pred_lr = lr_pipeline.predict(X_test)
print("=== Logistic Regression ===")
print(classification_report(y_test, y_pred_lr, target_names=['No Failure', 'Failure']))Your baseline is set. You now know that accuracy lies and that recall is what matters for catching failures. Next up, you will train more powerful models and tune them to maximize that recall score.
Compare Models and Tune Hyperparameters
Your Logistic Regression baseline gave you a starting point for measuring failure detection. Now it's time to find out whether more powerful models can catch more failures.
Tree-based ensemble models like Random Forest and Gradient Boosting often dominate tabular data problems because they capture non-linear relationships between features. You will train both, compare their recall scores, then tune the winner with cross-validation to squeeze out maximum performance.
In this step, get ready to:
- Train Random Forest and Gradient Boosting models and compare their recall.
- Tune the best model's hyperparameters using RandomizedSearchCV.
- Visualize performance with a confusion matrix and precision-recall curve.
Train and compare multiple models
A single model gives you no way to know if it's good or just okay. Training multiple algorithms on the same data lets you compare their strengths on this specific problem.
- In a new code cell, train a Random Forest classifier by running this code:
# --- Step 4: Model Comparison and Tuning ---
# Random Forest
rf_pipeline = Pipeline([
('preprocessor', preprocessor),
('classifier', RandomForestClassifier(n_estimators=100, random_state=42))
])
rf_pipeline.fit(X_train_resampled, y_train_resampled)
y_pred_rf = rf_pipeline.predict(X_test)
print("=== Random Forest ===")
print(classification_report(y_test, y_pred_rf, target_names=['No Failure', 'Failure']))What does this code do?
This follows the same Pipeline pattern as your Logistic Regression baseline. The preprocessor handles scaling and passthrough, then feeds into a Random Forest with 100 decision trees.
Random Forest works by training many decision trees on random subsets of the data, then averaging their predictions. This "wisdom of the crowd" approach reduces overfitting compared to a single tree.
- Run the cell. Look at the classification report and note the recall value for the Failure class.
Seeing all zeros for the Failure class?
Make sure you are fitting on X_train_resampled and y_train_resampled (the SMOTE-balanced data), not the original X_train and y_train.
Without SMOTE, the model may learn to predict "No Failure" for everything. Help me debug my Random Forest classification report
- In the next code cell, train a Gradient Boosting classifier by running this code:
# Gradient Boosting
gb_pipeline = Pipeline([
('preprocessor', preprocessor),
('classifier', GradientBoostingClassifier(n_estimators=100, random_state=42))
])
gb_pipeline.fit(X_train_resampled, y_train_resampled)
y_pred_gb = gb_pipeline.predict(X_test)
print("=== Gradient Boosting ===")
print(classification_report(y_test, y_pred_gb, target_names=['No Failure', 'Failure']))How is Gradient Boosting different from Random Forest?
Random Forest trains trees independently in parallel. Gradient Boosting trains trees sequentially, where each new tree corrects the mistakes of the previous ones.
This sequential correction often produces better accuracy, but at the cost of longer training time and higher sensitivity to hyperparameters.
- Run the cell and compare the Failure-class recall across all three models (Logistic Regression, Random Forest, Gradient Boosting).
- Note which model has the highest recall for the Failure class. This is the model that catches the most real failures.
Why focus on recall instead of accuracy?
In predictive maintenance, a missed failure (false negative) can mean a catastrophic breakdown costing thousands of dollars. A false alarm (false positive) just means an unnecessary inspection.
Recall measures the percentage of actual failures your model catches. Optimizing for recall accepts more false alarms to avoid missing real failures.
Tune the best model with cross-validation
Your best model's default settings are a starting point. Hyperparameter tuning searches for the combination of settings that maximizes your chosen metric.
- In a new code cell, define the hyperparameter search space and run the tuning by pasting this code:
# Hyperparameter tuning on the best model (Random Forest)
param_distributions = {
'classifier__n_estimators': [50, 100, 200, 300],
'classifier__max_depth': [5, 10, 15, 20, None],
'classifier__min_samples_split': [2, 5, 10],
'classifier__min_samples_leaf': [1, 2, 4]
}
rf_tuning_pipeline = Pipeline([
('preprocessor', preprocessor),
('classifier', RandomForestClassifier(random_state=42))
])
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
search = RandomizedSearchCV(
rf_tuning_pipeline,
param_distributions,
n_iter=20,
scoring='recall',
cv=cv,
random_state=42,
n_jobs=-1
)
search.fit(X_train_resampled, y_train_resampled)
print(f"\nBest parameters: {search.best_params_}")
print(f"Best CV recall: {search.best_score_:.3f}")What does this code do?
- param_distributions defines the values to try for each hyperparameter. The classifier__ prefix tells the Pipeline to pass these settings to the classifier step.
- StratifiedKFold splits training data into 5 folds while preserving the class ratio in each fold.
- RandomizedSearchCV randomly samples 20 combinations from the search space (n_iter=20) instead of trying every possible combination. This is faster than GridSearchCV when the search space is large.
- scoring='recall' tells the search to optimize for catching failures, not overall accuracy.
- n_jobs=-1 uses all available CPU cores to run the search in parallel.
- Run the cell. This may take 30-60 seconds as it trains 100 models (20 combinations x 5 folds).
- Note the best parameters and best CV recall printed in the output.
Cell taking too long?
The randomized search trains 100 models total. On Colab's free tier, this typically finishes within 60 seconds. If it takes longer than 2 minutes, check that you're using n_iter=20 (not a larger value) and n_jobs=-1 for parallel execution.
Help me speed up my RandomizedSearchCV
Evaluate the tuned model on the test set
The search found the best hyperparameters using cross-validation on the training data. Now you need to evaluate the tuned model on the held-out test set to see how it performs on data it has never seen.
- In a new code cell, extract the best model and generate the final classification report by running this code:
# Final evaluation
best_model = search.best_estimator_
y_pred_best = best_model.predict(X_test)
print("\n=== Best Model (Tuned Random Forest) ===")
print(classification_report(y_test, y_pred_best, target_names=['No Failure', 'Failure']))What does this code do?
search.best_estimator_ gives you the entire fitted Pipeline (preprocessor + classifier) using the best hyperparameters found during the search.
By predicting on X_test (data the model never saw during training or tuning), you get an honest estimate of real-world performance.
- Run the cell and compare the tuned model's Failure-class recall against the untuned Random Forest from earlier.
- In the next code cell, plot a confusion matrix heatmap by running this code:
# Confusion matrix
fig, ax = plt.subplots(figsize=(8, 6))
cm = confusion_matrix(y_test, y_pred_best)
disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=['No Failure', 'Failure'])
disp.plot(ax=ax, cmap='Blues')
plt.title('Confusion Matrix - Tuned Random Forest')
plt.tight_layout()
plt.show()How to read the confusion matrix
The confusion matrix is a 2x2 grid showing four outcomes. The top-left cell (true negatives) shows machines correctly predicted as healthy. The bottom-right cell (true positives) shows failures correctly caught.
The bottom-left cell (false negatives) is the most dangerous: failures your model missed. The top-right cell (false positives) shows healthy machines flagged as failing.
- Run the cell. You should see a blue heatmap with four quadrants.
- Check the bottom-left cell. The lower this number, the fewer failures your model is missing.
- In the next code cell, plot the precision-recall curve by running this code:
# Precision-Recall curve
y_scores = best_model.predict_proba(X_test)[:, 1]
precision, recall, thresholds = precision_recall_curve(y_test, y_scores)
pr_auc = auc(recall, precision)
plt.figure(figsize=(8, 6))
plt.plot(recall, precision, label=f'PR AUC = {pr_auc:.3f}')
plt.xlabel('Recall')
plt.ylabel('Precision')
plt.title('Precision-Recall Curve')
plt.legend()
plt.grid(True)
plt.tight_layout()
plt.show()Why precision-recall instead of ROC?
ROC curves can look optimistic with imbalanced data because the false positive rate stays low even with many false alarms (since the majority class is so large).
The precision-recall curve focuses on the minority class directly. A higher PR AUC means the model maintains good precision even as you push recall higher.
- Run the cell. You should see a curve that starts high on the left (high precision at low recall) and drops as recall increases.
- Note the PR AUC value in the legend. Higher values mean better overall performance across all thresholds.
Curve looks flat or unexpected?
Make sure y_scores uses [:, 1] to select the probability of the Failure class (class 1), not the No Failure class. If you use [:, 0], the curve will be inverted.
My precision-recall curve looks wrong
✔️ Awesome, I've got everything!
Great. Double check you have saved your notebook (Colab auto-saves, but you can press Ctrl+S to be sure).
ⓧ I'd like to double check the full code
Here is the complete code up to this point. Compare it against your notebook cells:
# =============================================================================
# Predictive Maintenance: Predict Machine Failures Before They Happen
# =============================================================================
# --- Step 1: Setup and Data Loading ---
# Install required packages (run in Colab)
# !pip install ucimlrepo imbalanced-learn shap
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split, StratifiedKFold, RandomizedSearchCV
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.metrics import classification_report, confusion_matrix, ConfusionMatrixDisplay, precision_recall_curve, auc
from sklearn.dummy import DummyClassifier
from imblearn.over_sampling import SMOTE
import shap
import warnings
warnings.filterwarnings('ignore')
# Load the dataset
from ucimlrepo import fetch_ucirepo
dataset = fetch_ucirepo(id=601)
X = dataset.data.features
y = dataset.data.targets
# Combine into a single DataFrame for exploration
df = pd.concat([X, y], axis=1)
print(f"Dataset shape: {df.shape}")
print(f"\nColumn names: {df.columns.tolist()}")
print(f"\nFirst 5 rows:")
print(df.head())
# Check class distribution
print(f"\nClass distribution:")
print(df['Machine failure'].value_counts())
print(f"\nFailure rate: {df['Machine failure'].mean()*100:.1f}%")
# --- Step 2: EDA and Feature Engineering ---
# Separate numerical and categorical features
numerical_cols = ['Air temperature [K]', 'Process temperature [K]',
'Rotational speed [rpm]', 'Torque [Nm]', 'Tool wear [min]']
categorical_col = 'Type'
# Plot feature distributions by failure class
fig, axes = plt.subplots(2, 3, figsize=(15, 10))
for idx, col in enumerate(numerical_cols):
ax = axes[idx // 3, idx % 3]
df[df['Machine failure'] == 0][col].hist(ax=ax, alpha=0.5, label='No Failure', bins=30)
df[df['Machine failure'] == 1][col].hist(ax=ax, alpha=0.5, label='Failure', bins=30)
ax.set_title(col)
ax.legend()
axes[1, 2].axis('off')
plt.tight_layout()
plt.show()
# Correlation heatmap
plt.figure(figsize=(10, 8))
sns.heatmap(df[numerical_cols + ['Machine failure']].corr(), annot=True, cmap='coolwarm', center=0)
plt.title('Feature Correlation Heatmap')
plt.tight_layout()
plt.show()
# Engineer physics-inspired features
df['power'] = df['Torque [Nm]'] * (df['Rotational speed [rpm]'] * 2 * np.pi / 60)
df['thermal_gradient'] = df['Process temperature [K]'] - df['Air temperature [K]']
df['strain_index'] = df['Tool wear [min]'] * df['Torque [Nm]']
# Visualize engineered features by class
fig, axes = plt.subplots(1, 3, figsize=(15, 5))
engineered_features = ['power', 'thermal_gradient', 'strain_index']
for idx, col in enumerate(engineered_features):
ax = axes[idx]
df[df['Machine failure'] == 0][col].hist(ax=ax, alpha=0.5, label='No Failure', bins=30)
df[df['Machine failure'] == 1][col].hist(ax=ax, alpha=0.5, label='Failure', bins=30)
ax.set_title(f'{col} by class')
ax.legend()
plt.tight_layout()
plt.show()
# One-hot encode the Type column
df = pd.get_dummies(df, columns=['Type'], prefix='type')
# --- Step 3: Handle Class Imbalance and Build Pipeline ---
# Prepare features and target
feature_cols = numerical_cols + engineered_features + ['type_H', 'type_L', 'type_M']
target_col = 'Machine failure'
X = df[feature_cols].copy()
y = df[target_col].copy()
# Train/test split with stratification
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y
)
print(f"Training set: {X_train.shape[0]} samples")
print(f"Test set: {X_test.shape[0]} samples")
print(f"Training failure rate: {y_train.mean()*100:.1f}%")
# Apply SMOTE to training data only
smote = SMOTE(random_state=42)
X_train_resampled, y_train_resampled = smote.fit_resample(X_train, y_train)
print(f"\nAfter SMOTE:")
print(f"Training samples: {X_train_resampled.shape[0]}")
print(f"Class distribution: {pd.Series(y_train_resampled).value_counts().to_dict()}")
# Build preprocessing pipeline
numerical_features = numerical_cols + engineered_features
categorical_features = ['type_H', 'type_L', 'type_M']
preprocessor = ColumnTransformer(
transformers=[
('num', StandardScaler(), numerical_features),
('cat', 'passthrough', categorical_features)
]
)
# Baseline: Dummy classifier (always predicts majority class)
dummy = DummyClassifier(strategy='most_frequent')
dummy.fit(X_train, y_train)
print(f"\nDummy classifier accuracy: {dummy.score(X_test, y_test)*100:.1f}%")
print("(This proves accuracy is USELESS for imbalanced data)\n")
# Logistic Regression baseline
lr_pipeline = Pipeline([
('preprocessor', preprocessor),
('classifier', LogisticRegression(max_iter=1000, random_state=42))
])
lr_pipeline.fit(X_train_resampled, y_train_resampled)
y_pred_lr = lr_pipeline.predict(X_test)
print("=== Logistic Regression ===")
print(classification_report(y_test, y_pred_lr, target_names=['No Failure', 'Failure']))
# --- Step 4: Model Comparison and Tuning ---
# Random Forest
rf_pipeline = Pipeline([
('preprocessor', preprocessor),
('classifier', RandomForestClassifier(n_estimators=100, random_state=42))
])
rf_pipeline.fit(X_train_resampled, y_train_resampled)
y_pred_rf = rf_pipeline.predict(X_test)
print("=== Random Forest ===")
print(classification_report(y_test, y_pred_rf, target_names=['No Failure', 'Failure']))
# Gradient Boosting
gb_pipeline = Pipeline([
('preprocessor', preprocessor),
('classifier', GradientBoostingClassifier(n_estimators=100, random_state=42))
])
gb_pipeline.fit(X_train_resampled, y_train_resampled)
y_pred_gb = gb_pipeline.predict(X_test)
print("=== Gradient Boosting ===")
print(classification_report(y_test, y_pred_gb, target_names=['No Failure', 'Failure']))
# Hyperparameter tuning on the best model (Random Forest)
param_distributions = {
'classifier__n_estimators': [50, 100, 200, 300],
'classifier__max_depth': [5, 10, 15, 20, None],
'classifier__min_samples_split': [2, 5, 10],
'classifier__min_samples_leaf': [1, 2, 4]
}
rf_tuning_pipeline = Pipeline([
('preprocessor', preprocessor),
('classifier', RandomForestClassifier(random_state=42))
])
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
search = RandomizedSearchCV(
rf_tuning_pipeline,
param_distributions,
n_iter=20,
scoring='recall',
cv=cv,
random_state=42,
n_jobs=-1
)
search.fit(X_train_resampled, y_train_resampled)
print(f"\nBest parameters: {search.best_params_}")
print(f"Best CV recall: {search.best_score_:.3f}")
# Final evaluation
best_model = search.best_estimator_
y_pred_best = best_model.predict(X_test)
print("\n=== Best Model (Tuned Random Forest) ===")
print(classification_report(y_test, y_pred_best, target_names=['No Failure', 'Failure']))
# Confusion matrix
fig, ax = plt.subplots(figsize=(8, 6))
cm = confusion_matrix(y_test, y_pred_best)
disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=['No Failure', 'Failure'])
disp.plot(ax=ax, cmap='Blues')
plt.title('Confusion Matrix - Tuned Random Forest')
plt.tight_layout()
plt.show()
# Precision-Recall curve
y_scores = best_model.predict_proba(X_test)[:, 1]
precision, recall, thresholds = precision_recall_curve(y_test, y_scores)
pr_auc = auc(recall, precision)
plt.figure(figsize=(8, 6))
plt.plot(recall, precision, label=f'PR AUC = {pr_auc:.3f}')
plt.xlabel('Recall')
plt.ylabel('Precision')
plt.title('Precision-Recall Curve')
plt.legend()
plt.grid(True)
plt.tight_layout()
plt.show()Your tuned model now catches the majority of machine failures before they happen. Next up, you will use SHAP to open the model's black box and explain exactly why it predicts each failure.
Explain Predictions with SHAP
You now have a tuned model that catches the majority of machine failures. But a prediction alone is not enough in industrial settings. A maintenance engineer needs to know why the model flagged a specific machine.
In this step, you will use SHAP (SHapley Additive exPlanations) to explain your model's decisions. SHAP uses game theory to assign each feature a contribution score for every prediction. You will generate global importance plots and drill into individual predictions to see which sensor readings drove each failure call.
In this step, get ready to:
- Generate SHAP values and a global feature importance plot.
- Explain individual predictions for a caught failure and a missed failure.
- Compare SHAP rankings with your domain knowledge of failure physics.
Generate SHAP values for the best model
Your tuned Random Forest is wrapped inside a scikit-learn Pipeline. SHAP's TreeExplainer needs direct access to the tree-based classifier, not the full pipeline. You also need the preprocessed (scaled) version of your test data so the SHAP values align with what the classifier actually sees.
- In a new code cell in your notebook, extract the classifier and transform the test data by running this code:
# --- Step 5: SHAP Explanations ---
# Extract the trained classifier from the pipeline
classifier = best_model.named_steps['classifier']
X_test_transformed = best_model.named_steps['preprocessor'].transform(X_test)
# Create SHAP explainer
explainer = shap.TreeExplainer(classifier)
shap_values = explainer.shap_values(X_test_transformed)
# Feature names for display
feature_names = numerical_features + categorical_features
# SHAP summary plot (global feature importance)
plt.figure(figsize=(10, 8))
shap.summary_plot(shap_values[1], X_test_transformed, feature_names=feature_names, show=False)
plt.title('SHAP Feature Importance (Failure Class)')
plt.tight_layout()
plt.show()What does this code do?
The code extracts the Random Forest classifier from your pipeline using best_model.named_steps['classifier']. It then transforms the test data through the same preprocessor the model was trained with.
The TreeExplainer is optimized for tree-based models and runs much faster than the generic KernelExplainer. shap_values[1] gives the SHAP values for the Failure class (class index 1). The summary plot shows how each feature pushes predictions toward or away from failure across all test samples.
- Run the cell.
- You should see a SHAP summary plot where each dot represents one test sample. Features are ranked top-to-bottom by importance. Red dots pushed to the right indicate high feature values driving the prediction toward failure.
Not seeing the SHAP plot?
If you get an error about shap_values shape, make sure you are indexing shap_values[1] (class 1 = Failure). The output of shap_values is a list of arrays, one per class.
If the cell is taking a long time, SHAP is computing values for all 2,000 test samples. This is normal and may take 30-60 seconds in Colab.
help me debug my SHAP summary plot
Explain individual predictions
Global importance tells you which features matter overall. But to explain a specific machine's failure prediction, you need a local explanation. SHAP force plots show exactly which features pushed one particular prediction toward failure or toward safety.
- In a new code cell, generate force plots for a correctly caught failure and a missed failure by running this code:
# Individual prediction explanations
# Find a correctly predicted failure
failure_indices = np.where((y_test.values == 1) & (y_pred_best == 1))[0]
if len(failure_indices) > 0:
idx = failure_indices[0]
print(f"\n--- Explaining a correctly predicted FAILURE (test sample {idx}) ---")
print(f"Features: {X_test.iloc[idx].to_dict()}")
shap.force_plot(
explainer.expected_value[1],
shap_values[1][idx],
X_test_transformed[idx],
feature_names=feature_names,
matplotlib=True
)
plt.tight_layout()
plt.show()
# Find a missed failure (false negative)
missed_indices = np.where((y_test.values == 1) & (y_pred_best == 0))[0]
if len(missed_indices) > 0:
idx = missed_indices[0]
print(f"\n--- Explaining a MISSED failure (test sample {idx}) ---")
print(f"Features: {X_test.iloc[idx].to_dict()}")
shap.force_plot(
explainer.expected_value[1],
shap_values[1][idx],
X_test_transformed[idx],
feature_names=feature_names,
matplotlib=True
)
plt.tight_layout()
plt.show()What does this code do?
The code finds two interesting test samples: one where the model correctly predicted a failure (true positive), and one where it missed a failure (false negative).
For each sample, shap.force_plot creates a horizontal bar chart showing which features pushed the prediction. Red bars push toward failure. Blue bars push toward safety. The length of each bar shows how strongly that feature contributed.
Comparing the caught failure against the missed failure reveals what made the missed one harder to detect. This is exactly the kind of analysis a maintenance engineer would use to improve the system.
- Run the cell.
- You should see two force plots and the raw feature values printed above each one. For the correctly predicted failure, look for red bars on strain_index or power pushing the prediction right (toward failure).
Force plot not rendering?
Make sure you included matplotlib=True in the shap.force_plot() call. Without this parameter, SHAP tries to render an interactive JavaScript plot that may not display in Colab.
help me fix my SHAP force plot in Colab
Compare SHAP feature importance with your domain knowledge
The real test of good feature engineering is whether your engineered features actually outperform the raw sensor readings. If SHAP ranks strain_index and power higher than raw Torque [Nm] or Rotational speed [rpm], your physics-driven features captured failure mechanisms better than the individual measurements alone.
- In a new code cell, add these summary print statements and run the cell:
print("\n=== Project Complete ===")
print("Your predictive maintenance model can catch machine failures before they happen.")
print("Key insight: Engineered features (strain_index, power) outperform raw sensors.")
print("Key metric: Recall > Accuracy for imbalanced classification.")What should I look for in the SHAP results?
Check your SHAP summary plot from earlier. If strain_index and power appear in the top features, your domain-driven engineering paid off. These features encode the actual physics of overstrain failure (tool wear multiplied by torque) and power failure (torque times angular velocity).
Also compare the force plots from the two individual predictions. The caught failure likely has extreme values on engineered features. The missed failure may have subtler signals that the model could not confidently separate from normal operation.
- Add a new markdown cell below the code cell by clicking **+ Text** in the Colab toolbar.
- Document your findings: which features ranked highest in SHAP, whether engineered features outperformed raw ones, and what the force plots revealed about the caught versus missed failure.
Write for a non-technical audience
In a real maintenance setting, your audience is the plant engineer, not the data scientist. Practice writing explanations like: "This machine was flagged because its strain index (tool wear 215 min times torque 55 Nm) is extremely high, indicating overstrain." SHAP turns a black-box model into a tool that builds trust.
✔️ Awesome, I've got everything!
Great. Double check you have saved your notebook.
ⓧ I'd like to double check the full code
Here is the complete notebook code from Step 1 through Step 5:
# =============================================================================
# Predictive Maintenance: Predict Machine Failures Before They Happen
# =============================================================================
# --- Step 1: Setup and Data Loading ---
# Install required packages (run in Colab)
# !pip install ucimlrepo imbalanced-learn shap
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split, StratifiedKFold, RandomizedSearchCV
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.metrics import classification_report, confusion_matrix, ConfusionMatrixDisplay, precision_recall_curve, auc
from sklearn.dummy import DummyClassifier
from imblearn.over_sampling import SMOTE
import shap
import warnings
warnings.filterwarnings('ignore')
# Load the dataset
from ucimlrepo import fetch_ucirepo
dataset = fetch_ucirepo(id=601)
X = dataset.data.features
y = dataset.data.targets
# Combine into a single DataFrame for exploration
df = pd.concat([X, y], axis=1)
print(f"Dataset shape: {df.shape}")
print(f"\nColumn names: {df.columns.tolist()}")
print(f"\nFirst 5 rows:")
print(df.head())
# Check class distribution
print(f"\nClass distribution:")
print(df['Machine failure'].value_counts())
print(f"\nFailure rate: {df['Machine failure'].mean()*100:.1f}%")
# --- Step 2: EDA and Feature Engineering ---
# Separate numerical and categorical features
numerical_cols = ['Air temperature [K]', 'Process temperature [K]',
'Rotational speed [rpm]', 'Torque [Nm]', 'Tool wear [min]']
categorical_col = 'Type'
# Plot feature distributions by failure class
fig, axes = plt.subplots(2, 3, figsize=(15, 10))
for idx, col in enumerate(numerical_cols):
ax = axes[idx // 3, idx % 3]
df[df['Machine failure'] == 0][col].hist(ax=ax, alpha=0.5, label='No Failure', bins=30)
df[df['Machine failure'] == 1][col].hist(ax=ax, alpha=0.5, label='Failure', bins=30)
ax.set_title(col)
ax.legend()
axes[1, 2].axis('off')
plt.tight_layout()
plt.show()
# Correlation heatmap
plt.figure(figsize=(10, 8))
sns.heatmap(df[numerical_cols + ['Machine failure']].corr(), annot=True, cmap='coolwarm', center=0)
plt.title('Feature Correlation Heatmap')
plt.tight_layout()
plt.show()
# Engineer physics-inspired features
df['power'] = df['Torque [Nm]'] * (df['Rotational speed [rpm]'] * 2 * np.pi / 60)
df['thermal_gradient'] = df['Process temperature [K]'] - df['Air temperature [K]']
df['strain_index'] = df['Tool wear [min]'] * df['Torque [Nm]']
# Visualize engineered features by class
fig, axes = plt.subplots(1, 3, figsize=(15, 5))
engineered_features = ['power', 'thermal_gradient', 'strain_index']
for idx, col in enumerate(engineered_features):
ax = axes[idx]
df[df['Machine failure'] == 0][col].hist(ax=ax, alpha=0.5, label='No Failure', bins=30)
df[df['Machine failure'] == 1][col].hist(ax=ax, alpha=0.5, label='Failure', bins=30)
ax.set_title(f'{col} by class')
ax.legend()
plt.tight_layout()
plt.show()
# One-hot encode the Type column
df = pd.get_dummies(df, columns=['Type'], prefix='type')
# --- Step 3: Handle Class Imbalance and Build Pipeline ---
# Prepare features and target
feature_cols = numerical_cols + engineered_features + ['type_H', 'type_L', 'type_M']
target_col = 'Machine failure'
X = df[feature_cols].copy()
y = df[target_col].copy()
# Train/test split with stratification
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y
)
print(f"Training set: {X_train.shape[0]} samples")
print(f"Test set: {X_test.shape[0]} samples")
print(f"Training failure rate: {y_train.mean()*100:.1f}%")
# Apply SMOTE to training data only
smote = SMOTE(random_state=42)
X_train_resampled, y_train_resampled = smote.fit_resample(X_train, y_train)
print(f"\nAfter SMOTE:")
print(f"Training samples: {X_train_resampled.shape[0]}")
print(f"Class distribution: {pd.Series(y_train_resampled).value_counts().to_dict()}")
# Build preprocessing pipeline
numerical_features = numerical_cols + engineered_features
categorical_features = ['type_H', 'type_L', 'type_M']
preprocessor = ColumnTransformer(
transformers=[
('num', StandardScaler(), numerical_features),
('cat', 'passthrough', categorical_features)
]
)
# Baseline: Dummy classifier (always predicts majority class)
dummy = DummyClassifier(strategy='most_frequent')
dummy.fit(X_train, y_train)
print(f"\nDummy classifier accuracy: {dummy.score(X_test, y_test)*100:.1f}%")
print("(This proves accuracy is USELESS for imbalanced data)\n")
# Logistic Regression baseline
lr_pipeline = Pipeline([
('preprocessor', preprocessor),
('classifier', LogisticRegression(max_iter=1000, random_state=42))
])
lr_pipeline.fit(X_train_resampled, y_train_resampled)
y_pred_lr = lr_pipeline.predict(X_test)
print("=== Logistic Regression ===")
print(classification_report(y_test, y_pred_lr, target_names=['No Failure', 'Failure']))
# --- Step 4: Model Comparison and Tuning ---
# Random Forest
rf_pipeline = Pipeline([
('preprocessor', preprocessor),
('classifier', RandomForestClassifier(n_estimators=100, random_state=42))
])
rf_pipeline.fit(X_train_resampled, y_train_resampled)
y_pred_rf = rf_pipeline.predict(X_test)
print("=== Random Forest ===")
print(classification_report(y_test, y_pred_rf, target_names=['No Failure', 'Failure']))
# Gradient Boosting
gb_pipeline = Pipeline([
('preprocessor', preprocessor),
('classifier', GradientBoostingClassifier(n_estimators=100, random_state=42))
])
gb_pipeline.fit(X_train_resampled, y_train_resampled)
y_pred_gb = gb_pipeline.predict(X_test)
print("=== Gradient Boosting ===")
print(classification_report(y_test, y_pred_gb, target_names=['No Failure', 'Failure']))
# Hyperparameter tuning on the best model (Random Forest)
param_distributions = {
'classifier__n_estimators': [50, 100, 200, 300],
'classifier__max_depth': [5, 10, 15, 20, None],
'classifier__min_samples_split': [2, 5, 10],
'classifier__min_samples_leaf': [1, 2, 4]
}
rf_tuning_pipeline = Pipeline([
('preprocessor', preprocessor),
('classifier', RandomForestClassifier(random_state=42))
])
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
search = RandomizedSearchCV(
rf_tuning_pipeline,
param_distributions,
n_iter=20,
scoring='recall',
cv=cv,
random_state=42,
n_jobs=-1
)
search.fit(X_train_resampled, y_train_resampled)
print(f"\nBest parameters: {search.best_params_}")
print(f"Best CV recall: {search.best_score_:.3f}")
# Final evaluation
best_model = search.best_estimator_
y_pred_best = best_model.predict(X_test)
print("\n=== Best Model (Tuned Random Forest) ===")
print(classification_report(y_test, y_pred_best, target_names=['No Failure', 'Failure']))
# Confusion matrix
fig, ax = plt.subplots(figsize=(8, 6))
cm = confusion_matrix(y_test, y_pred_best)
disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=['No Failure', 'Failure'])
disp.plot(ax=ax, cmap='Blues')
plt.title('Confusion Matrix - Tuned Random Forest')
plt.tight_layout()
plt.show()
# Precision-Recall curve
y_scores = best_model.predict_proba(X_test)[:, 1]
precision, recall, thresholds = precision_recall_curve(y_test, y_scores)
pr_auc = auc(recall, precision)
plt.figure(figsize=(8, 6))
plt.plot(recall, precision, label=f'PR AUC = {pr_auc:.3f}')
plt.xlabel('Recall')
plt.ylabel('Precision')
plt.title('Precision-Recall Curve')
plt.legend()
plt.grid(True)
plt.tight_layout()
plt.show()
# --- Step 5: SHAP Explanations ---
# Extract the trained classifier from the pipeline
classifier = best_model.named_steps['classifier']
X_test_transformed = best_model.named_steps['preprocessor'].transform(X_test)
# Create SHAP explainer
explainer = shap.TreeExplainer(classifier)
shap_values = explainer.shap_values(X_test_transformed)
# Feature names for display
feature_names = numerical_features + categorical_features
# SHAP summary plot (global feature importance)
plt.figure(figsize=(10, 8))
shap.summary_plot(shap_values[1], X_test_transformed, feature_names=feature_names, show=False)
plt.title('SHAP Feature Importance (Failure Class)')
plt.tight_layout()
plt.show()
# Individual prediction explanations
# Find a correctly predicted failure
failure_indices = np.where((y_test.values == 1) & (y_pred_best == 1))[0]
if len(failure_indices) > 0:
idx = failure_indices[0]
print(f"\n--- Explaining a correctly predicted FAILURE (test sample {idx}) ---")
print(f"Features: {X_test.iloc[idx].to_dict()}")
shap.force_plot(
explainer.expected_value[1],
shap_values[1][idx],
X_test_transformed[idx],
feature_names=feature_names,
matplotlib=True
)
plt.tight_layout()
plt.show()
# Find a missed failure (false negative)
missed_indices = np.where((y_test.values == 1) & (y_pred_best == 0))[0]
if len(missed_indices) > 0:
idx = missed_indices[0]
print(f"\n--- Explaining a MISSED failure (test sample {idx}) ---")
print(f"Features: {X_test.iloc[idx].to_dict()}")
shap.force_plot(
explainer.expected_value[1],
shap_values[1][idx],
X_test_transformed[idx],
feature_names=feature_names,
matplotlib=True
)
plt.tight_layout()
plt.show()
print("\n=== Project Complete ===")
print("Your predictive maintenance model can catch machine failures before they happen.")
print("Key insight: Engineered features (strain_index, power) outperform raw sensors.")
print("Key metric: Recall > Accuracy for imbalanced classification.")Secret mission
Multi-Class Failure Mode Prediction
Your binary classifier knows IF a machine will fail. But in a real factory, the maintenance crew needs to know WHICH failure mode is coming. A heat dissipation failure requires a coolant check. An overstrain failure requires a torque reduction. Can you build a model that predicts the specific failure type?
Clean Up Your Resources
Clean Up Your Resources
Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely in Google Colab, so there are no ongoing costs.
Resources you used:
- Google Colab notebook (stored in your Google Drive).
- Python packages installed during the Colab session (ucimlrepo, imbalanced-learn, shap). These are session-scoped and disappear when the Colab runtime disconnects.
Keep everything running
No action needed. Choose this if you're still actively building or want to revisit the notebook later.
- Your notebook is saved in Google Drive automatically.
- If your Colab runtime disconnects, you can reconnect and re-run all cells from the top.
- Installed packages will need to be re-installed when you start a new session (just re-run the first cell).
Pause - I'll come back to this later
Disconnect the Colab runtime to free up resources. Your notebook file stays safely in Google Drive.
- In Colab, click Runtime in the top menu bar.
- Click Disconnect and delete runtime.
- This frees the GPU/CPU resources allocated to your session. Your notebook code and outputs remain saved.
- When you come back, click Runtime then Run all to re-execute all cells from the top.
Delete - I don't want to use this again
Remove the notebook from your Google Drive entirely.
- In Colab, click File in the top menu bar.
- Click Locate in Drive. This opens Google Drive and highlights your notebook file.
- Right-click the notebook file and select Move to trash.
- The installed packages (ucimlrepo, imbalanced-learn, shap) are session-scoped and already gone once the runtime disconnects. No additional cleanup needed.
Nice Work!
Nice Work!
You just built a production-grade predictive maintenance system from scratch. Your notebook takes raw industrial sensor data and turns it into actionable failure predictions, complete with explanations a maintenance engineer could act on.
You've learned how to:
- Built a complete ML pipeline that predicts machine failures from raw sensor data, using physics-inspired feature engineering (power, thermal gradient, strain index) that outperforms raw readings.
- Handled severe class imbalance (3.4% failures) with SMOTE, proved why accuracy is misleading, and optimised for recall to catch failures before they happen.
- Explained model decisions using SHAP, connecting individual predictions back to physical failure mechanisms that non-technical stakeholders can understand.
- Secret Mission: Extended to multi-class failure mode prediction, identifying which specific failure type will occur across multiple imbalanced categories simultaneously.
Ready to quiz yourself?