Adaptive Email Spam Detection with AI

Build a spam detector that fuses email content with metadata signals.

Introduction

30 Second Summary

Every day, your inbox quietly filters out dozens of scam offers, phishing attempts, and junk messages before you ever see them. Behind the scenes, a spam filter reads each email and makes a split-second decision: safe or suspicious.

In this project, you will build a spam detection model in Python that goes beyond simple text analysis. You will fuse email content features with metadata signals like sender domains, subject line patterns, and structural cues to create a measurably stronger classifier.

What You'll Build

Picture running a single command, watching your model process 75,000 real emails, and opening side-by-side confusion matrices that prove your fused model catches spam the baseline missed.

By the end of this project, you'll have:

  • Side-by-side confusion matrices that let you point at the exact cells where metadata fusion reduced misclassifications compared to a text-only baseline.
  • An adaptive retraining script where you drop in a CSV of new labeled emails, run one command, and the model updates itself on the combined data.
  • Professional classification reports with per-class precision, recall, and F1 scores you can pull straight into a work presentation.
  • Secret Mission: Add class-weighted training and ROC curve analysis to push your model even further.

Are there any prerequisites?

You'll need basic Python knowledge (variables, functions, loops, if statements). No machine learning experience is required.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters.

Set Up Your Environment and Dataset

You've defined the spam detection system you're building and why combining content with metadata makes it stronger. To make that idea real, you need a dataset that has both email body text and metadata like sender addresses and subject lines.

Most spam detection tutorials use a simple two-column dataset with just a label and a message. In this step, you'll set up a Python environment, install the machine learning libraries, and download the TREC 2007 email dataset with over 75,000 emails across five metadata-rich columns.

In this step, get ready to:
  • Create a project folder with a Python virtual environment.
  • Install scikit-learn, pandas, matplotlib, and beautifulsoup4.
  • Download the TREC 2007 email dataset and verify it loads correctly.
Create your project folder and virtual environment

A virtual environment keeps your project's packages isolated from the rest of your system. Every package you install stays inside this project, so nothing conflicts with other Python work on your machine.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal (macOS) or PowerShell (Windows) and press Enter to open it.
  • Set up your project folder and virtual environment by running these commands:

macOS

cd ~/Desktop
mkdir spam-detector
cd spam-detector
python3 -m venv venv
source venv/bin/activate

Windows

cd ~\Desktop
mkdir spam-detector
cd spam-detector
python -m venv venv
venv\Scripts\activate

You should see (venv) appear at the start of your terminal prompt. This confirms your virtual environment is active.

Not seeing (venv)?

Make sure you are inside the spam-detector folder. Run pwd (macOS) or cd (Windows) to check your current directory.

On Windows, if PowerShell blocks the activation script with a red error about execution policies, run the following command first, then try activating again:

Set-ExecutionPolicy -ExecutionPolicy Bypass -Scope Process

Still stuck? Help me activate my Python virtual environment

  • Install the required packages by running this command:
pip install scikit-learn pandas matplotlib beautifulsoup4

You should see output showing each package being downloaded and installed along with its dependencies.

What are these packages?

  • scikit-learn provides machine learning algorithms and tools like TF-IDF vectorization and logistic regression.
  • pandas handles loading and manipulating tabular data like your email dataset.
  • matplotlib creates visualizations like confusion matrices and charts.
  • beautifulsoup4 strips HTML tags from email content during text preprocessing.

Seeing install errors?

Make sure your virtual environment is active. You should see (venv) in your terminal prompt before running pip install.

If you see a network error, check your internet connection and try the command again.

Still stuck? Help me install Python packages with pip

Download the TREC 2007 dataset

The dataset you need is the Preprocessed TREC 2007 Public Corpus hosted on Kaggle. It contains 75,419 emails with five columns: label, subject, email_to, email_from, and message.

Why this dataset?

Many spam tutorials use a simple two-column dataset with just a label and message body. That format makes content-metadata fusion impossible because there is no metadata to fuse.

The TREC 2007 dataset includes sender addresses, subject lines, and recipient fields alongside the message body. These extra columns are exactly the metadata signals you will engineer features from in later steps.

✔️ I already have a Kaggle account

Sign in and continue to the download step below.

ⓧ I need to create one

  • Click Register at the top right of the page.
  • Sign up with your Google account or email address.
  • Once signed in, return to the dataset page linked above.
  • Click the Download button on the dataset page. The file is about 338 MB, so the download may take a minute.
  • Extract the downloaded zip file by double-clicking it (macOS) or right-clicking and selecting Extract All (Windows).
  • Back in your terminal, create the data folder and move the extracted CSV into it by running:

macOS

mkdir data
mv ~/Downloads/processed_data.csv data/
ls data/

Windows

mkdir data
Move-Item ~\Downloads\processed_data.csv data
ls data

You should see processed_data.csv listed in the output.

File not found?

If the move command says the file does not exist, the CSV may be inside a subfolder after extraction. Check your Downloads folder for a subfolder named archive or similar, and adjust the path in the move command to match.

Make sure your terminal is still inside the spam-detector folder before running mkdir data.

Still stuck? Help me move my Kaggle dataset into my project folder

Write and run the verification script

Before building any models, you need to confirm the dataset loaded correctly and inspect its structure. A quick verification script prints the shape, columns, and class balance so you know exactly what you're working with.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Visual Studio Code and press Enter to open it.
  • In VS Code, click File in the top menu bar.
  • Click Open Folder.
  • Select the spam-detector folder on your Desktop and click Open.

You should see your project in the Explorer sidebar on the left, with the data and venv folders visible.

  • In the Explorer sidebar, right-click in the empty area below the folders.
  • Select New File and name it verify_setup.py.
  • Create the verification script by pasting this code into verify_setup.py:
import pandas as pd

# Load the dataset from the data folder
df = pd.read_csv("data/processed_data.csv")

# Print the dataset dimensions and column names
print(f"Dataset shape: {df.shape}")
print(f"Columns: {df.columns.tolist()}")

# Show how many spam vs ham emails exist
print(f"\nClass distribution:")
print(f"  Spam (label=1): {(df['label'] == 1).sum()}")
print(f"  Ham  (label=0): {(df['label'] == 0).sum()}")

# Preview the first few rows
print(f"\nFirst 3 rows:")
print(df.head(3))

What does this code do?

  • pd.read_csv("data/processed_data.csv") loads the entire CSV into a pandas DataFrame.
  • df.shape returns the number of rows and columns, confirming nothing was lost during the download.
  • The class distribution lines count how many emails are labeled spam (1) versus ham (0), showing whether the dataset is balanced or skewed.
  • df.head(3) previews the first three rows so you can see the actual column values.
  • Save verify_setup.py by pressing Cmd+S (macOS) or Ctrl+S (Windows).
  • Open VS Code's integrated terminal by pressing Ctrl+` (the backtick key above Tab).

The terminal opens at your project root. You should see (venv) at the start of your prompt, meaning your virtual environment is active.

No (venv) in VS Code's terminal?

VS Code may not auto-detect your virtual environment. Activate it manually by running the command for your platform.

On macOS or Linux:

source venv/bin/activate

On Windows:

venv\Scripts\activate

Before you run the script, what do you predict the balance between spam and ham emails will be?

  • Verify your dataset loaded correctly by running this command:
python verify_setup.py

You should see output showing the dataset has 75,419 rows across 5 columns. The class distribution reveals approximately 50,199 spam emails and 25,220 ham emails.

What does this output tell you?

The dataset is roughly two-thirds spam and one-third legitimate email. This imbalance means a model that always guesses "spam" would already be right about 66% of the time.

That is exactly why you need a confusion matrix (coming in the next step) to measure real performance. Raw accuracy alone would hide whether your model actually learned anything useful.

Seeing a FileNotFoundError?

Check that processed_data.csv is inside the data folder in your project. Run ls data/ (macOS) or ls data (Windows PowerShell) to confirm.

If you see a ModuleNotFoundError for pandas, make sure your virtual environment is active. You should see (venv) in your prompt.

Still stuck? Help me run my verify_setup.py script

✔️ Awesome, I've got everything!

Great. Double check you've saved verify_setup.py and move on to the next step.

ⓧ I'd like to double check the full code

Here is the complete verify_setup.py for comparison:

import pandas as pd

df = pd.read_csv("data/processed_data.csv")

print(f"Dataset shape: {df.shape}")
print(f"Columns: {df.columns.tolist()}")
print(f"\nClass distribution:")
print(f"  Spam (label=1): {(df['label'] == 1).sum()}")
print(f"  Ham  (label=0): {(df['label'] == 0).sum()}")
print(f"\nFirst 3 rows:")
print(df.head(3))

Your environment is ready and the dataset is loaded. Next up, you'll train a baseline spam detector on just the message text and use a confusion matrix to expose exactly where it fails.

Run a Baseline and Expose Its Blind Spots

Your environment is ready and the TREC dataset is loaded. Now it is time to build your first spam detector and measure how well it performs.

Before improving anything, you need to see what a content-only model can do on its own. High accuracy can mask serious problems. A confusion matrix reveals exactly which emails the model gets wrong, giving you a clear target for improvement.

In this step, get ready to:
  • Train a baseline spam detection model using TF-IDF and Logistic Regression.
  • Visualize the model's mistakes with a confusion matrix and classification report.
Create the baseline evaluation script

The simplest approach to spam detection is to look at the email body text alone. You will convert raw email text into numbers using TF-IDF, then train a classifier on those numbers.

  • Create a new file called baseline_evaluation.py in VS Code.
  • Add the data loading and train-test split by pasting this code:
import pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, classification_report, ConfusionMatrixDisplay
import matplotlib.pyplot as plt

# Load the email dataset and drop rows with missing messages
df = pd.read_csv("data/processed_data.csv")
df = df.dropna(subset=["message"])

# Separate the email text (features) and spam/ham labels (target)
X = df["message"]
y = df["label"]

# Split into training and test sets (70% train, 30% test)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.3, random_state=42, stratify=y
)

What does this code do?

This sets up the data pipeline for the baseline model.

  • pd.read_csv loads the 75,419-row TREC dataset. dropna removes any rows where the message column is empty, since you cannot classify an email with no text.
  • train_test_split reserves 30% of the data as a test set the model never trains on. stratify=y keeps the spam-to-ham ratio the same in both sets.
  • Save baseline_evaluation.py and run it to confirm the data loads without error:
python baseline_evaluation.py

The script finishes silently with no errors. That confirms your imports and dataset path are correct.

Seeing a FileNotFoundError?

  • Make sure you are running the command from your project folder (the one containing the data/ folder).
  • Check that the file is named exactly processed_data.csv inside the data/ folder.

Still stuck? Help me debug a FileNotFoundError when loading my CSV dataset.

  • Add the model training and accuracy check below the existing code in baseline_evaluation.py:
# Convert email text into numerical TF-IDF features
tfidf = TfidfVectorizer(lowercase=True, stop_words="english")
X_train_tfidf = tfidf.fit_transform(X_train)
X_test_tfidf = tfidf.transform(X_test)

# Train a logistic regression classifier
model = LogisticRegression(max_iter=1000, random_state=42)
model.fit(X_train_tfidf, y_train)

# Generate predictions and print accuracy
y_pred = model.predict(X_test_tfidf)

print(f"Baseline Accuracy: {accuracy_score(y_test, y_pred):.4f}")

What does this code do?

  • TfidfVectorizer converts each email into a vector of word-importance scores. TF-IDF stands for Term Frequency-Inverse Document Frequency. Words that appear often in one email but rarely across all emails get higher scores.
  • fit_transform learns the vocabulary from the training set and converts it in one step. transform applies the same vocabulary to the test set without learning new words from it.
  • LogisticRegression learns a decision boundary between spam and ham based on those TF-IDF scores. max_iter=1000 gives the algorithm enough iterations to converge on a large dataset.
  • Save baseline_evaluation.py and run it again:
python baseline_evaluation.py

You should see a line like Baseline Accuracy: 0.98XX printed to the terminal. The accuracy looks impressive.

Seeing a convergence warning?

  • Check that max_iter=1000 is set in the LogisticRegression() call. The default of 100 is too low for this dataset.
  • Make sure stop_words="english" is set in TfidfVectorizer(). Without it, common words add noise and slow convergence.

Need more help? Help me fix a convergence warning from LogisticRegression.

Add a confusion matrix and classification report

That accuracy number looks great, but a single number hides a lot. How many legitimate emails is the model incorrectly flagging as spam? How many spam emails are slipping through undetected?

  • Add this code at the bottom of baseline_evaluation.py, below the accuracy print line:
# Print detailed precision, recall, and F1 per class
print("\nClassification Report:")
print(classification_report(y_test, y_pred, target_names=["Ham", "Spam"]))

# Generate and save a confusion matrix visualization
ConfusionMatrixDisplay.from_predictions(
    y_test, y_pred, display_labels=["Ham", "Spam"]
)
plt.title("Baseline Model - Confusion Matrix")
plt.savefig("baseline_confusion_matrix.png", dpi=150, bbox_inches="tight")
print("Confusion matrix saved to baseline_confusion_matrix.png")
plt.show()

What does this code do?

  • classification_report prints precision, recall, and F1-score for each class. Precision tells you what fraction of emails labeled spam were actually spam. Recall tells you what fraction of actual spam emails were caught.
  • ConfusionMatrixDisplay.from_predictions creates a grid showing exact counts: how many ham emails were correctly identified, how many were misclassified as spam (false positives), how many spam emails slipped through (false negatives), and how many spam emails were caught.
  • plt.savefig saves the chart as an image file so you can compare it against improved models later.
  • Save baseline_evaluation.py.

Before you run the script, think about this: the accuracy was around 98%. Do you think the model is equally good at catching spam and protecting legitimate email?

  • Run the script to find out:
python baseline_evaluation.py

You should see the accuracy, the full classification report with per-class precision, recall, and F1, and a matplotlib window showing the confusion matrix. The image is also saved to baseline_confusion_matrix.png.

How do I read the confusion matrix?

The confusion matrix is a 2x2 grid. The top-left cell shows ham emails correctly identified. The bottom-right shows spam emails correctly caught. The interesting cells are the off-diagonal ones.

  • Top-right (false positives): legitimate emails the model flagged as spam. In a real inbox, these are important emails you would never see.
  • Bottom-left (false negatives): spam emails that slipped through. These are the junk messages cluttering your inbox.

Look at which type of mistake the model makes more often. That gap is what you will fix in the next steps by adding text preprocessing and metadata features.

Close the chart window to continue

plt.show() opens a matplotlib window and pauses the script. Close the chart window to let the script finish and return to your terminal prompt.

No chart window appearing?

  • Some environments (like remote SSH sessions) do not support GUI windows. The chart is still saved to baseline_confusion_matrix.png. Open that file directly to view it.
  • Make sure import matplotlib.pyplot as plt is at the top of your file.

Still stuck? Help me troubleshoot matplotlib not displaying a chart window.

✔️ Awesome, I've got everything!

Great. Double check you have saved baseline_evaluation.py and that baseline_confusion_matrix.png exists in your project folder.

ⓧ I'd like to double check the full code

Your baseline_evaluation.py file should have three sections in order. Scroll up and confirm each one is present:

  • Imports and data loading (6 import lines, then pd.read_csv and dropna, then train_test_split).
  • TF-IDF and model training (TfidfVectorizer, fit_transform, LogisticRegression, predict, and the accuracy print).
  • Classification report and confusion matrix (classification_report, ConfusionMatrixDisplay.from_predictions, plt.savefig, and plt.show()).

You have a baseline that looks accurate on the surface, but the confusion matrix tells the real story. Next up, you will clean the raw email text and tune the TF-IDF settings to squeeze stronger signals out of the same content.

Improve Text Preprocessing and TF-IDF Features

Your baseline model read raw email text straight into TF-IDF. That means every HTML tag, URL, and chunk of formatting noise counted as a "feature" the model learned from.

Spam emails are packed with this noise. Cleaning the text first and tuning how TF-IDF extracts features can pull much stronger signals from the exact same content. In this step, you will build a preprocessing pipeline and an upgraded vectorizer, then compare the results against your baseline.

In this step, get ready to:
  • Write a text preprocessing function that strips HTML, normalizes URLs and numbers, and removes noise.
  • Configure an improved TF-IDF vectorizer and compare the improved model against your baseline.
Write the text preprocessing function

Spam emails often contain raw HTML markup, embedded URLs, email addresses, and random numbers. If you feed all of that into TF-IDF, the model wastes capacity learning noise instead of genuine spam patterns.

Why clean the text instead of using it raw?

A raw email body might contain <table><tr><td>Click here</td></tr></table>. The baseline treats every HTML tag as a word. Stripping that markup and replacing URLs with a single placeholder like urlplaceholder lets the model focus on the actual language instead.

  • Create a new file called fused_model.py in your project folder (the same folder where baseline_evaluation.py lives).
  • Add the following imports and preprocess_text function by pasting this code into the file:
import re
import numpy as np
import pandas as pd
from bs4 import BeautifulSoup
from scipy.sparse import hstack, csr_matrix
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, classification_report, ConfusionMatrixDisplay
import matplotlib.pyplot as plt
import pickle


def preprocess_text(text):
    # Return empty string for non-text values
    if not isinstance(text, str):
        return ""
    # Strip HTML tags from email content
    text = BeautifulSoup(text, "html.parser").get_text()
    text = text.lower()
    # Replace URLs and emails with generic placeholders
    text = re.sub(r"http\S+|www\.\S+", "urlplaceholder", text)
    text = re.sub(r"\S+@\S+", "emailplaceholder", text)
    # Replace numbers with a placeholder
    text = re.sub(r"\d+", "numplaceholder", text)
    # Remove special characters and collapse whitespace
    text = re.sub(r"[^a-z\s]", " ", text)
    text = re.sub(r"\s+", " ", text).strip()
    return text

What does this code do?

The imports pull in everything you will need for this step and the next. Some (like hstack and pickle) are not used yet but will be needed when you add metadata features later.

The preprocess_text function runs each email through six cleaning steps: strip HTML with BeautifulSoup, lowercase everything, swap URLs and email addresses for generic placeholders, replace numbers, remove special characters, and collapse extra whitespace. Every email that comes out is pure lowercase text with consistent tokens.

  • Save fused_model.py.
  • Verify the function works by running this quick test in your terminal:
python -c "from fused_model import preprocess_text; print(preprocess_text('Hello World 123'))"

You should see hello world numplaceholder printed. The function lowercased the text and replaced the number with a placeholder.

Seeing a ModuleNotFoundError?

Make sure your virtual environment is activated before running the test. You should see the environment name in parentheses at the start of your terminal prompt.

If you see an import error for bs4, double check that you installed beautifulsoup4 in the previous step.

Help me debug my preprocess_text import error

Configure improved TF-IDF and train the model

Your baseline vectorizer used default TF-IDF settings with only single words (unigrams). Spam phrases like "click now" or "free offer" are two-word patterns that a unigram model cannot capture.

What are these new TF-IDF settings?

Bigrams (ngram_range=(1, 2)) let the vectorizer learn two-word phrases alongside single words. Sublinear TF scaling (sublinear_tf=True) uses a logarithmic count so a word appearing 100 times does not dominate one that appears 10 times.

The frequency thresholds (max_df=0.95 and min_df=2) drop words that appear in nearly every email (too common to be useful) or only once (too rare to generalize). max_features=50000 caps the vocabulary so the model stays fast on 75,000 emails.

  • In fused_model.py, add the following training and evaluation code below the preprocess_text function:
# Load the dataset and apply text preprocessing
df = pd.read_csv("data/processed_data.csv")
df = df.dropna(subset=["message"])
df["cleaned_message"] = df["message"].apply(preprocess_text)
X = df["cleaned_message"]
y = df["label"]
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.3, random_state=42, stratify=y
)

# Improved TF-IDF with bigrams and sublinear scaling
tfidf = TfidfVectorizer(
    lowercase=True, stop_words="english", ngram_range=(1, 2),
    max_df=0.95, min_df=2, sublinear_tf=True, max_features=50000,
)
X_train_tfidf = tfidf.fit_transform(X_train)
X_test_tfidf = tfidf.transform(X_test)

# Train and evaluate the improved model
model = LogisticRegression(max_iter=1000, random_state=42)
model.fit(X_train_tfidf, y_train)
y_pred = model.predict(X_test_tfidf)
print(f"Improved Model Accuracy: {accuracy_score(y_test, y_pred):.4f}")
print("\nClassification Report:")
print(classification_report(y_test, y_pred, target_names=["Ham", "Spam"]))

# Save confusion matrix to compare against baseline
ConfusionMatrixDisplay.from_predictions(y_test, y_pred, display_labels=["Ham", "Spam"])
plt.title("Improved Model - Confusion Matrix")
plt.savefig("improved_confusion_matrix.png", dpi=150, bbox_inches="tight")
print("Confusion matrix saved to improved_confusion_matrix.png")
plt.show()

What does this code do?

This code mirrors the baseline flow from baseline_evaluation.py but with two key upgrades. First, every email runs through your preprocess_text function before vectorization. Second, the TfidfVectorizer now uses bigrams, sublinear TF scaling, and frequency thresholds.

The same 70/30 stratified split and LogisticRegression setup are used so the comparison is fair. The confusion matrix is saved to improved_confusion_matrix.png so you can compare it side by side with baseline_confusion_matrix.png.

  • Save fused_model.py.

Before you run the script, think about this: will cleaning the text and adding bigrams improve precision, recall, or both compared to the baseline?

  • Run the improved model by executing this command:
python fused_model.py

The script will take a moment to preprocess all 75,000 emails. When it finishes, you will see the accuracy, the full classification report with per-class precision, recall, and F1 scores, and a confusion matrix window will pop up.

Compare the numbers against your baseline. Both precision and recall should be higher, and improved_confusion_matrix.png should show fewer misclassifications than baseline_confusion_matrix.png.

Script taking a very long time?

Preprocessing 75,000 emails and building bigram features can take 30 to 90 seconds depending on your machine. This is normal. If the script hangs for more than a few minutes, check that your dataset file at data/processed_data.csv is the correct TREC dataset (it should be around 338 MB).

If you see a NameError for preprocess_text, make sure the training code is pasted below the function definition, not inside it. Check the indentation.

Help me debug my fused_model.py script

✔️ Awesome, I've got everything!

Great. Double check you have saved fused_model.py and that improved_confusion_matrix.png was saved to your project folder.

ⓧ I'd like to double check the full code

Scroll up and confirm your fused_model.py has these two sections in order:

  • All 11 import lines at the top (from import re through import pickle), followed by the preprocess_text function definition.
  • The training and evaluation block starting with df = pd.read_csv("data/processed_data.csv") and ending with plt.show().

Make sure the training block sits outside the preprocess_text function (no indentation under def).

Your cleaned text and upgraded TF-IDF settings already beat the baseline. But text is only half the story. Next up, you will engineer features from email metadata (sender domains, subject lines, structural patterns) and fuse them with the content features to build a model that catches what text alone misses.

Engineer Metadata Features and Build the Fused Model

Your improved text preprocessing already cut down misclassifications. But text content alone has a ceiling.

Spam emails carry telltale metadata patterns: suspicious sender domains, urgently worded subject lines, excessive URLs, and unusual capitalization. In this step, you will engineer 15 features from those signals and fuse them with your TF-IDF text features to create a measurably stronger model.

In this step, get ready to:
  • Write a metadata feature engineering function that extracts 15 signals from subject lines, sender addresses, and email bodies.
  • Fuse content and metadata features into a combined matrix, train a model on it, and compare results against the text-only baseline.
Write the metadata feature engineering function

Each email in the TREC dataset has a subject line, a sender address, and a message body. You can extract numeric signals from all three without ever reading the words themselves.

  • In fused_model.py, find the comment line # [engineer_metadata_features and train_fused_model to be added in next step] and delete it.
  • In its place, add the first part of the engineer_metadata_features function by copying the following code. This covers the six subject-line features:
def engineer_metadata_features(df):
    features = pd.DataFrame()

    # Subject line features: length, word count, reply/forward flags,
    # uppercase ratio, and exclamation marks
    features["subject_length"] = df["subject"].fillna("").str.len()
    features["subject_word_count"] = (
        df["subject"].fillna("").str.split().str.len().fillna(0).astype(int)
    )
    features["subject_has_re"] = (
        df["subject"].fillna("").str.lower().str.startswith("re:").astype(int)
    )
    features["subject_has_fw"] = (
        df["subject"].fillna("").str.lower().str.startswith("fw").astype(int)
    )
    features["subject_uppercase_ratio"] = df["subject"].fillna("").apply(
        lambda x: sum(1 for c in x if c.isupper()) / max(len(x), 1)
    )
    features["subject_has_exclamation"] = (
        df["subject"].fillna("").str.contains("!", regex=False).astype(int)
    )

What does this code do?

Each line extracts a single numeric signal from the subject column. Spam subjects tend to be longer, contain more uppercase letters, and use exclamation marks more often than legitimate emails.

The subject_has_re and subject_has_fw flags check if the email is a reply or forward. Legitimate emails are far more likely to be part of a thread than spam blasts.

  • Directly below the subject features you just added, copy the sender and body features to complete the function:
    # Sender features: domain length and noreply detection
    features["sender_domain_length"] = df["email_from"].fillna("").apply(
        lambda x: len(x.split("@")[-1]) if "@" in str(x) else 0
    )
    features["sender_has_noreply"] = (
        df["email_from"]
        .fillna("")
        .str.lower()
        .str.contains("noreply|no-reply", regex=True)
        .astype(int)
    )

    # Body features: length, word count, URL count, uppercase/special/digit
    # ratios, and exclamation count
    features["body_length"] = df["message"].fillna("").str.len()
    features["body_word_count"] = (
        df["message"].fillna("").str.split().str.len().fillna(0).astype(int)
    )
    features["url_count"] = (
        df["message"].fillna("").str.count(r"http\S+|www\.\S+")
    )
    features["uppercase_ratio"] = df["message"].fillna("").apply(
        lambda x: sum(1 for c in x if c.isupper()) / max(len(x), 1)
    )
    features["special_char_ratio"] = df["message"].fillna("").apply(
        lambda x: sum(1 for c in x if not c.isalnum() and not c.isspace())
        / max(len(x), 1)
    )
    features["digit_ratio"] = df["message"].fillna("").apply(
        lambda x: sum(1 for c in x if c.isdigit()) / max(len(x), 1)
    )
    features["exclamation_count"] = (
        df["message"].fillna("").str.count("!")
    )

    return features

What does this code do?

The sender features check the domain length (spammers often use long, random domains) and whether the address contains "noreply" (common in automated spam).

The body features measure structural patterns that text analysis misses: how many URLs are embedded, the ratio of uppercase letters, special characters, and digits. Spam emails tend to be heavy on all of these compared to normal conversation.

  • Save fused_model.py.
  • Verify the function loads without errors by running this command:
python -c "from fused_model import engineer_metadata_features; print('Function loaded successfully')"

You should see Function loaded successfully printed with no errors.

Seeing a SyntaxError or IndentationError?

  • Make sure the second code block is indented at the same level as the subject features above it. Every line inside the function needs consistent indentation.
  • Check that the return features line is indented one level inside the function, not at the top level of the file.
  • Confirm you deleted the old placeholder comment before pasting.
  • Help me fix a syntax error in my engineer_metadata_features function in fused_model.py.
Fuse content and metadata features into a training pipeline

You now have two types of features: the TF-IDF vectors from email text and the 15 numeric metadata features. The fusion step combines them into a single feature matrix so the model sees both signals at once.

  • Below the engineer_metadata_features function, add the first part of the train_fused_model function. This section loads the data, preprocesses it, engineers metadata features, and splits everything into train/test sets:
def train_fused_model(data_path="data/processed_data.csv"):
    # Load and clean the dataset
    print("Loading dataset...")
    df = pd.read_csv(data_path)
    df = df.dropna(subset=["message"])
    print(f"Dataset loaded: {len(df)} emails")

    # Apply text preprocessing and metadata feature engineering
    print("Preprocessing text...")
    df["cleaned_message"] = df["message"].apply(preprocess_text)

    print("Engineering metadata features...")
    metadata_features = engineer_metadata_features(df)

    X_text = df["cleaned_message"]
    X_meta = metadata_features
    y = df["label"]

    # Split text and metadata together to keep rows aligned
    X_text_train, X_text_test, X_meta_train, X_meta_test, y_train, y_test = (
        train_test_split(X_text, X_meta, y, test_size=0.3, random_state=42, stratify=y)
    )

What does this code do?

The function takes a CSV path as input, loads it, and runs both your preprocess_text function and the new engineer_metadata_features function on the data.

The train_test_split call passes both X_text and X_meta together. This keeps the text and metadata rows aligned after the split, so email #42's text features always pair with email #42's metadata.

  • Directly below the train/test split you just added, copy the TF-IDF vectorization, feature fusion, model training, and evaluation code:
    # Build improved TF-IDF features from cleaned text
    print("Building TF-IDF features...")
    tfidf = TfidfVectorizer(
        lowercase=True,
        stop_words="english",
        ngram_range=(1, 2),
        max_df=0.95,
        min_df=2,
        sublinear_tf=True,
        max_features=50000,
    )
    X_train_tfidf = tfidf.fit_transform(X_text_train)
    X_test_tfidf = tfidf.transform(X_text_test)

    # Fuse TF-IDF content features with metadata features
    print("Fusing content and metadata features...")
    X_train_meta_sparse = csr_matrix(X_meta_train.values)
    X_test_meta_sparse = csr_matrix(X_meta_test.values)

    X_train_fused = hstack([X_train_tfidf, X_train_meta_sparse])
    X_test_fused = hstack([X_test_tfidf, X_test_meta_sparse])

    # Train and evaluate the fused model
    print("Training fused model...")
    model = LogisticRegression(max_iter=1000, random_state=42)
    model.fit(X_train_fused, y_train)

    y_pred = model.predict(X_test_fused)
    accuracy = accuracy_score(y_test, y_pred)
    print(f"\nFused Model Accuracy: {accuracy:.4f}")
    print("\nClassification Report:")
    print(classification_report(y_test, y_pred, target_names=["Ham", "Spam"]))

What is feature fusion?

TF-IDF produces a sparse matrix (mostly zeros, because most words do not appear in most emails). The metadata features are a dense 15-column table. csr_matrix converts the metadata into sparse format so both feature sets speak the same language.

Then hstack glues them side by side. The model now trains on every TF-IDF column plus all 15 metadata columns in one pass, seeing both what the email says and how it is structured.

  • Save fused_model.py and verify the file has no syntax errors by running:
python -c "import fused_model; print('No syntax errors')"

You should see No syntax errors printed. The function is not yet complete, so do not run it fully yet.

Seeing an IndentationError?

  • The code you just added is a continuation of train_fused_model. Every line must be indented one level inside the function body, matching the indentation of the code above it.
  • Make sure there is no blank line or unindented code between the two chunks inside the function.
  • Help me fix an indentation error in my train_fused_model function.
  • Finally, add the confusion matrix visualization, model saving, and the main entry point directly below the classification report print statements:
    # Save confusion matrix as an image
    ConfusionMatrixDisplay.from_predictions(
        y_test, y_pred, display_labels=["Ham", "Spam"]
    )
    plt.title("Fused Model - Confusion Matrix")
    plt.savefig("fused_confusion_matrix.png", dpi=150, bbox_inches="tight")
    print("Confusion matrix saved to fused_confusion_matrix.png")
    plt.show()

    # Save trained model and vectorizer for reuse
    print("Saving model artifacts...")
    with open("fused_model.pkl", "wb") as f:
        pickle.dump(model, f)
    with open("tfidf_vectorizer.pkl", "wb") as f:
        pickle.dump(tfidf, f)
    print("Model saved to fused_model.pkl")
    print("Vectorizer saved to tfidf_vectorizer.pkl")

    return model, tfidf


if __name__ == "__main__":
    train_fused_model()

What does this code do?

The confusion matrix image lets you visually compare misclassifications against your baseline and improved-text-only results from earlier steps.

The pickle.dump calls save the trained model and the fitted TF-IDF vectorizer as .pkl files. These files let you load the model later without retraining from scratch.

  • Save fused_model.py.

Before you run the script, what do you expect to happen to the accuracy and misclassification counts now that the model has both text content and metadata features?

  • Run the fused model training pipeline by running this command:
python fused_model.py

You should see progress messages followed by the fused model's accuracy and a full classification report. The confusion matrix window will pop up, and after you close it the script saves it to fused_confusion_matrix.png along with two pickle files: fused_model.pkl and tfidf_vectorizer.pkl.

  • Confirm the output files were created by running:
ls *.pkl *.png

You should see fused_model.pkl, tfidf_vectorizer.pkl, fused_confusion_matrix.png, alongside your earlier baseline_confusion_matrix.png and improved_confusion_matrix.png.

How do the three models compare?

Open all three confusion matrix images side by side. The baseline (content-only with minimal TF-IDF) has the most misclassifications. The improved text-only model cuts those down. The fused model should show further improvement, especially in correctly identifying spam that previously slipped through.

The classification report's per-class precision and recall numbers tell the same story in numbers. Higher recall on the Spam class means fewer spam emails sneaking into the inbox.

Script runs but accuracy didn't improve?

  • Check that engineer_metadata_features returns all 15 features. If any were accidentally omitted, the fused matrix has less information to work with.
  • Make sure hstack is combining X_train_tfidf with X_train_meta_sparse, not doubling the text features or the metadata features.
  • My fused model accuracy didn't improve over the text-only model. Help me debug.

✔️ Awesome, I've got everything!

Great. Double check you have saved fused_model.py and that all three confusion matrix images plus both pickle files are present in your project folder.

ⓧ I'd like to double check the full code

Your fused_model.py is long, so scroll through it and confirm these sections are present, in order:

  • Imports at the top (re, numpy, pandas, BeautifulSoup, hstack, csr_matrix, scikit-learn classes, matplotlib, pickle).
  • The preprocess_text function (from Step 3).
  • The engineer_metadata_features function with 15 features: 6 subject, 2 sender, 7 body.
  • The train_fused_model function: loads data, preprocesses text, engineers metadata, splits, builds TF-IDF, fuses with hstack, trains LogisticRegression, prints report, saves confusion matrix, and pickles model and vectorizer.
  • The if __name__ == "__main__": block calling train_fused_model().

Your fused model now combines email content with structural metadata for stronger spam detection. Next up, you will build an adaptive retraining script that lets you feed new labeled emails into the system and keep the model current as spam tactics evolve.

Build the Adaptive Retraining Mechanism

Your fused model proves that metadata features add real value to spam detection. But spam evolves constantly.

A model trained once on today's data will lose accuracy as spammers change tactics and adopt new patterns. In this step, you'll build an adaptive retraining script that feeds new labeled emails into the system and updates the model automatically.

In this step, get ready to:
  • Create retrain.py that combines original and new labeled data, then retrains the fused model.
  • Test the retraining pipeline end-to-end with sample spam and ham emails.
Write the retraining script

The retraining script needs to run the same pipeline as fused_model.py, but on a combined dataset of original and new emails. Instead of duplicating the preprocessing and feature engineering code, you'll import those functions directly.

  • Create a new file called retrain.py in your project folder and add the following code:
import sys
import pandas as pd
import pickle
from scipy.sparse import hstack, csr_matrix
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, classification_report
from fused_model import preprocess_text, engineer_metadata_features

# Reuse preprocessing and feature engineering from fused_model.py
def retrain(original_data_path, new_data_path):
    # Load and merge both datasets
    print("Loading original dataset...")
    original = pd.read_csv(original_data_path)
    original = original.dropna(subset=["message"])
    print("Loading new labeled data...")
    new_data = pd.read_csv(new_data_path)
    new_data = new_data.dropna(subset=["message"])
    combined = pd.concat([original, new_data], ignore_index=True)
    print(f"Combined dataset: {len(combined)} emails ({len(original)} original + {len(new_data)} new)")
    # Apply same text cleaning and metadata feature engineering
    print("Preprocessing...")
    combined["cleaned_message"] = combined["message"].apply(preprocess_text)
    metadata_features = engineer_metadata_features(combined)

What does this code do?

The import line from fused_model import preprocess_text, engineer_metadata_features pulls in the exact functions you already wrote. No code duplication needed.

The retrain function loads two CSV files (original data and new data), drops any rows with missing messages, and concatenates them into a single combined dataset. It then applies the same preprocessing and feature engineering pipeline from your fused model.

  • Save retrain.py.
  • The function isn't complete yet. The next chunk adds the training, evaluation, and saving logic.
  • In retrain.py, add this code directly below the metadata_features line:
    # Split, vectorize, fuse, train, and save the updated model
    X_text = combined["cleaned_message"]
    X_meta = metadata_features
    y = combined["label"]
    X_text_train, X_text_test, X_meta_train, X_meta_test, y_train, y_test = train_test_split(X_text, X_meta, y, test_size=0.3, random_state=42, stratify=y)
    tfidf = TfidfVectorizer(lowercase=True, stop_words="english", ngram_range=(1, 2), max_df=0.95, min_df=2, sublinear_tf=True, max_features=50000)
    X_train_tfidf = tfidf.fit_transform(X_text_train)
    X_test_tfidf = tfidf.transform(X_text_test)
    X_train_fused = hstack([X_train_tfidf, csr_matrix(X_meta_train.values)])
    X_test_fused = hstack([X_test_tfidf, csr_matrix(X_meta_test.values)])
    print("Training updated model...")
    model = LogisticRegression(max_iter=1000, random_state=42)
    model.fit(X_train_fused, y_train)
    y_pred = model.predict(X_test_fused)
    print(f"\nUpdated Model Accuracy: {accuracy_score(y_test, y_pred):.4f}")
    print("\nClassification Report:")
    print(classification_report(y_test, y_pred, target_names=["Ham", "Spam"]))
    # Overwrite saved model files with retrained versions
    with open("fused_model.pkl", "wb") as f:
        pickle.dump(model, f)
    with open("tfidf_vectorizer.pkl", "wb") as f:
        pickle.dump(tfidf, f)
    print("\nUpdated model saved to fused_model.pkl")
    print("Updated vectorizer saved to tfidf_vectorizer.pkl")

if __name__ == "__main__":
    if len(sys.argv) != 3:
        print("Usage: python retrain.py <original_data.csv> <new_data.csv>")
        print("Example: python retrain.py data/processed_data.csv data/new_emails.csv")
        sys.exit(1)
    retrain(sys.argv[1], sys.argv[2])

What does this code do?

  • The train-test split, TfidfVectorizer settings, and hstack fusion are identical to fused_model.py. The model trains on the combined dataset instead of the original alone.
  • After evaluation, the script overwrites fused_model.pkl and tfidf_vectorizer.pkl with the retrained versions.
  • The if __name__ == "__main__" block reads two command-line arguments: the path to the original dataset and the path to the new labeled data CSV.
  • Save retrain.py.

Seeing a syntax error when you save?

  • Make sure the second code block starts at the correct indentation level (one tab or four spaces inside the retrain function body). The comment line should align with the print("Preprocessing...") line above it.
  • Check that the if __name__ block is at the top level (no indentation), not inside the function.

Help me fix a syntax error in my retrain.py file

✔️ Awesome, I've got everything!

Make sure you've saved retrain.py. Time to test it.

ⓧ I'd like to double check the full code

Your retrain.py should contain the two code blocks you added above, in order. Scroll up and confirm:

  • Block 1 (imports and data loading): starts with import sys and ends with metadata_features = engineer_metadata_features(combined).
  • Block 2 (training and saving): starts with the comment # Split, vectorize, fuse, train, and save the updated model and ends with retrain(sys.argv[1], sys.argv[2]).
Test the retraining pipeline

To test the retraining script, you need a small CSV of new labeled emails in the same 5-column format as the TREC dataset.

  • Create a new file called data/new_emails.csv and add the following test data:
label,subject,email_to,email_from,message
1,URGENT: You've Won $1000000!!!,victim@email.com,winner-notify@prize-central.com,"Congratulations! You have been selected as the winner of our grand prize. Click here immediately to claim your $1000000. This offer expires in 24 hours. Send your bank details to process the transfer."
0,Team Meeting Tomorrow,john@company.com,manager@company.com,"Hi team, just a reminder that we have our weekly standup meeting tomorrow at 10am in the main conference room. Please prepare your status updates. Thanks."
1,Your Account Has Been Compromised,user@email.com,security-alert@bankk-verify.com,"We have detected suspicious activity on your account. Your account will be suspended unless you verify your identity immediately. Click the link below and enter your password and social security number."
0,Re: Project deadline update,alice@work.com,bob@work.com,"Thanks for the update Alice. I will have the report ready by Friday. Let me know if you need anything else from my side."
1,FREE FREE FREE - Buy One Get Ten Free!!!,shopper@email.com,deals-noreply@super-savings.com,"AMAZING DEAL!!! Buy one get TEN free!!! Limited time only!!! Click NOW before it's too late!!! FREE shipping FREE returns FREE everything!!!"

What is in this test data?

This CSV contains 5 sample emails: 3 spam and 2 ham. The spam emails have classic patterns your model should recognize, like urgency keywords, suspicious sender domains, and excessive exclamation marks. The ham emails are normal workplace messages.

The columns match the TREC dataset format (label, subject, email_to, email_from, message) so the retraining script can process them identically.

  • Save data/new_emails.csv.
  • Confirm the file exists by running:
ls data/

You should see both processed_data.csv and new_emails.csv listed.

Before you run the retraining script, think about what you expect to happen. You are adding just 5 emails to a dataset of over 75,000. Will the model's accuracy change significantly?

  • Run the retraining script by passing both CSV file paths:
python retrain.py data/processed_data.csv data/new_emails.csv

The script prints the combined dataset size, trains on the merged data, and outputs the updated accuracy and classification report. The accuracy should be nearly identical to the fused model from Step 4 because 5 emails barely shift a 75,000-email dataset. The important thing is that the pipeline works end-to-end.

Why doesn't the accuracy change much?

Five new emails in a dataset of 75,419 is a 0.007% increase. The model's statistical patterns barely shift. In a real production system, you would accumulate hundreds or thousands of new labeled emails before retraining. The point of this step is to prove the pipeline works, not to change the model's performance.

Seeing a ModuleNotFoundError?

  • If the error mentions fused_model, make sure retrain.py and fused_model.py are in the same folder. Python looks for imports relative to the script's location.
  • If the error mentions a package like sklearn or scipy, check that your virtual environment is activated before running the script.
  • If you see a FileNotFoundError, confirm you are running the command from the project folder (where data/ lives).

Help me fix an import or file error when running retrain.py

Your spam detection system now adapts. Any time new labeled spam arrives, you feed it into retrain.py and the model updates itself with the combined data. Next up, put your skills to the test with a Secret Mission.

Secret mission

Add Class-Weighted Training and ROC Curve Analysis

Your fused model treats every email equally during training. But the TREC dataset has nearly twice as many spam emails as ham. Class weighting corrects this imbalance, and a ROC curve reveals how well your model discriminates between spam and ham across every possible decision threshold.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely on your local machine. There are no cloud resources or paid services to shut down, so there are no ongoing costs.

Resources you used:

  • Trained model files (fused_model.pkl, tfidf_vectorizer.pkl) stored in your project folder.
  • Evaluation images (baseline_confusion_matrix.png, improved_confusion_matrix.png, fused_confusion_matrix.png, roc_curve.png) stored in your project folder.
  • TREC 2007 dataset CSV (data/processed_data.csv, ~338 MB) and sample emails (data/new_emails.csv) in the data/ folder.
  • Python scripts (verify_setup.py, baseline_evaluation.py, fused_model.py, retrain.py) and Python virtual environment in your project folder.

Decide whether to keep your resources, pause them to come back later, or delete them entirely.

Keep everything running

No action needed. Choose this if you plan to keep experimenting with your spam detection model or want to retrain it with more data.

Your model files, scripts, dataset, and virtual environment all stay on your machine as they are. You can re-run fused_model.py or retrain.py at any time.

Kaggle account

If you created a Kaggle account for this project, it remains active. You can use it to download other datasets for future projects.

Pause - I'll come back to this later

Free up disk space by deleting the large dataset file while keeping your trained model and scripts. The dataset is ~338 MB, which is the bulk of this project's storage.

What stays and what goes?

Your trained model (fused_model.pkl and tfidf_vectorizer.pkl) will still work for classifying new emails without the dataset.

You will not be able to retrain the model without the dataset. You can re-download it from Kaggle later if needed.

  • Delete the dataset and confirm it is gone by running these commands from inside your project folder:

macOS

rm data/processed_data.csv
ls data/

Windows

del data\processed_data.csv
dir data\

You should see only new_emails.csv remaining in the data/ folder.

Delete - I don't want to use this again

Remove the entire project folder to delete all scripts, model files, evaluation images, the dataset, and the virtual environment.

  • Delete the project folder from your Desktop by running these commands:

macOS

cd ~/Desktop
rm -rf [[PROJECT_FOLDER="your project folder name"]]

Windows

cd ~\Desktop
rmdir /s /q [[PROJECT_FOLDER="your project folder name"]]

Your machine is back to its original state. If you created a Kaggle account for this project, it remains active separately.

Nice Work!

Nice Work!

Nice work! You just built a professional-grade spam detection system that proves the measurable impact of fusing email content with metadata signals.

You learned how to:

  • Built a baseline spam detector using TF-IDF on email body text, exposed its blind spots with a confusion matrix, then improved it with text preprocessing (HTML stripping, URL normalization) and upgraded TF-IDF settings (bigrams, sublinear scaling).
  • Engineered 15 metadata features from sender domains, subject lines, and email structure, then fused them with content features using scipy.sparse.hstack to build a measurably stronger content-metadata fusion model with side-by-side confusion matrix proof.
  • Created an adaptive retraining script that combines new labeled emails with the original dataset, retrains the model on the fused features, and saves updated pickle files so the detector stays current as spam evolves.
  • Secret Mission: Added class-weighted training to handle class imbalance and generated a ROC curve with an AUC score to evaluate the model's discrimination ability at different thresholds.

Ready to quiz yourself?