Adaptive Email Spam Detection with AI

Build a spam classifier that fuses email content with metadata features.

Introduction

30 Second Summary

Every day, your email inbox quietly filters out dozens of scam offers, phishing attempts, and junk messages before you even see them. But some spam still slips through, disguised as normal emails that a simple keyword filter cannot catch.

In this project, you will build a spam detection system that classifies emails by fusing text content with structural metadata features. You will train a text-only classifier on 33,716 real emails, watch it fail on tricky messages, then engineer features from email structure that fix those blind spots.

What You'll Build

Paste any email into your script and watch it instantly classify the message as spam or ham, backed by a confidence score from a model that reads both the words and the structural patterns of the email.

By the end of this project, you'll have:

  • A text-only baseline classifier using TF-IDF and Random Forest you can test to see exactly which emails it gets wrong, revealing the limits of content-only spam detection.
  • A fused pipeline combining text and metadata features with scikit-learn's ColumnTransformer, delivering measurably higher accuracy than text alone.
  • A comparison bar chart proving the improvement, and a prediction function that classifies any email you paste in with a confidence score.
  • Secret Mission: Experiment with different classifiers and TF-IDF hyperparameters to push your model's F1-score even higher.

Do I need machine learning experience?

No prior machine learning experience is required. You just need Python and VS Code installed on your machine.

Before We Start

Before diving in, take a moment to lock in what you are about to build and why it matters.

Set Up Your Environment and Libraries

Building a spam detector requires more than Python alone. You need specialized libraries for machine learning, data processing, and visualization.

In this step, you'll create your project folder and install three libraries: scikit-learn for machine learning, pandas for data handling, and matplotlib for visualization.

In this step, get ready to:
  • Create a spam-detector project folder with a spam_detector.py file.
  • Install scikit-learn, pandas, and matplotlib.
  • Verify all three libraries are installed and working.
Create your project folder and file
  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code and press Enter to open it.
  • Open the integrated terminal by pressing Ctrl+` (backtick key, usually above Tab).
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir spam-detector
cd spam-detector
  • Click File in the top menu bar.
  • Click Open Folder.
  • Navigate to your Desktop, select the spam-detector folder, and click Open.

The Explorer sidebar on the left now shows the spam-detector folder as the workspace root.

  • In the Explorer sidebar, hover over SPAM-DETECTOR and click the New File icon.
  • Type spam_detector.py and press Enter.

You should see spam_detector.py listed in the Explorer sidebar and open as an empty tab in the editor.

Install the required libraries
  • Grab all three libraries at once by running this command in your terminal:
pip install scikit-learn==1.9.0 pandas==3.0.5 matplotlib==3.11.1

What do these libraries do?

  • scikit-learn provides machine learning algorithms, text vectorization, and model evaluation tools.
  • pandas handles loading, cleaning, and manipulating data as tables called DataFrames.
  • matplotlib creates charts and visualizations so you can compare model performance at a glance.

The install takes a minute or two. When it finishes, you should see Successfully installed followed by the package names near the bottom of the terminal output.

Seeing an install error?

  • If you see a permissions error, try adding --user to the end of the install command.
  • If pip is not recognized, try pip3 instead.
  • scikit-learn 1.9.0 requires Python 3.11 or higher. Run python --version to check yours.

Help me troubleshoot my pip install error for scikit-learn, pandas, or matplotlib.

Verify the installation

A successful install doesn't always mean the libraries load correctly. A quick version check confirms everything imported without errors.

  • Add the following version check code to spam_detector.py:
# Import the three libraries
import sklearn
import pandas
import matplotlib

# Print each library's version to confirm installation
print(sklearn.__version__)
print(pandas.__version__)
print(matplotlib.__version__)

What does this code do?

  • Each import line loads a library into Python so your script can use it.
  • __version__ is a built-in attribute that every Python library exposes. Printing it confirms the exact version that installed.
  • Save spam_detector.py by pressing Cmd+S (macOS) or Ctrl+S (Windows).

Before you run this, what three values do you expect to see printed in your terminal?

  • Run the script by entering this command in your terminal:
python spam_detector.py

You should see three version numbers printed, one per line: 1.9.0, 3.0.5, and 3.11.1.

Seeing a ModuleNotFoundError?

  • Make sure you saved the file before running it.
  • Check that your terminal is inside the spam-detector folder. Run pwd to confirm.
  • If python opens the wrong version, try python3 instead.

Help me fix a ModuleNotFoundError when running my version check script.

✔️ Awesome, I've got everything!

Great. Double check you have saved spam_detector.py and move on to the next step.

ⓧ I'd like to double check the full code

Here is the complete spam_detector.py at this point:

# Import the three libraries
import sklearn
import pandas
import matplotlib

# Print each library's version to confirm installation
print(sklearn.__version__)
print(pandas.__version__)
print(matplotlib.__version__)

Your environment is ready. Next up, you'll load a real dataset of 33,716 Enron emails and explore what spam actually looks like.

Load and Explore the Enron Spam Dataset

Your libraries are installed and verified. Before training any spam detector, you need to understand the raw material you are working with.

In this step, you will load 33,716 real emails from the Enron corpus, compare what spam and ham actually look like, and prepare the data for machine learning by splitting it into training and test sets.

In this step, get ready to:
  • Download and load the Enron spam dataset into a pandas DataFrame.
  • Explore sample emails and print statistics about spam vs. ham.
  • Create binary labels and split the data into training and test sets.
Download and load the Enron dataset

The Enron Spam Dataset contains 33,716 real emails labeled as spam or ham. It is hosted on GitHub as a single CSV file inside a zip archive.

  • Make sure your terminal is open inside your spam-detector folder. Download the dataset zip file by running this command:
curl -L -o enron_spam_data.zip https://github.com/MWiechmann/enron_spam_data/raw/master/enron_spam_data.zip
  • Extract the CSV file from the zip by running:
python -c "import zipfile; zipfile.ZipFile('enron_spam_data.zip').extractall('.')"
  • Confirm the CSV file exists by running:
ls enron_spam_data.csv

You should see enron_spam_data.csv printed in the output.

Seeing 'curl: command not found' or download issues?

On Windows PowerShell, the curl alias conflicts with a built-in command. Use curl.exe instead (with the .exe extension).

Alternatively, download the zip manually from the GitHub repository. Click the green Code button, select Download ZIP, then extract the CSV file into your spam-detector folder.

Help me download the Enron spam dataset

Now that the dataset file is in your project folder, you need to write the code that loads and explores it. The existing version check code in spam_detector.py has served its purpose.

  • In spam_detector.py, select all the existing code and replace it with the following:
import pandas as pd
import matplotlib.pyplot as plt
import re
from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.ensemble import RandomForestClassifier
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.metrics import classification_report, accuracy_score

# ============================================================
# STEP 2: Load and Explore the Enron Spam Dataset
# ============================================================

df = pd.read_csv("enron_spam_data.csv")

df["Subject"] = df["Subject"].fillna("")
df["Message"] = df["Message"].fillna("")

print("Dataset shape:", df.shape)
print("\nClass distribution:")
print(df["Spam/Ham"].value_counts())

What does this code do?

  • The imports at the top bring in every library you will use across the whole project. pandas handles data loading, matplotlib handles charts, and the sklearn imports cover ML tools you will use in later steps.
  • pd.read_csv("enron_spam_data.csv") loads the entire CSV into a DataFrame called df.
  • fillna("") replaces any missing values in the Subject and Message columns with empty strings, so text processing does not crash on missing data later.
  • The three print statements show the dataset dimensions and how many emails are spam vs. ham.
  • Save spam_detector.py.
  • Run the script by executing:
python spam_detector.py

You should see the dataset shape printed as (33716, 4) and a class distribution showing roughly 17,171 spam and 16,545 ham emails.

Seeing a FileNotFoundError?

This means Python cannot find enron_spam_data.csv in the current directory. Make sure the CSV file is inside your spam-detector folder (not inside a subfolder created during extraction).

Also check that your terminal is running from the spam-detector folder. You can confirm by running ls and looking for both spam_detector.py and enron_spam_data.csv in the output.

Help me fix my FileNotFoundError when loading the CSV

Inspect sample emails and print statistics

Numbers alone do not tell the full story. Reading actual spam and ham emails reveals patterns that statistics miss, like how spam subjects shout in ALL CAPS or stuff URLs into the body.

  • In spam_detector.py, add the following code below the print(df["Spam/Ham"].value_counts()) line:
print("\n--- Sample SPAM email ---")
spam_sample = df[df["Spam/Ham"] == "spam"].iloc[0]
print(f"Subject: {spam_sample['Subject'][:100]}")
print(f"Message: {spam_sample['Message'][:200]}")

print("\n--- Sample HAM email ---")
ham_sample = df[df["Spam/Ham"] == "ham"].iloc[0]
print(f"Subject: {ham_sample['Subject'][:100]}")
print(f"Message: {ham_sample['Message'][:200]}")

What does this code do?

  • df[df["Spam/Ham"] == "spam"].iloc[0] filters the DataFrame to only spam rows, then grabs the first one.
  • [:100] and [:200] truncate the subject and message so the output stays readable in the terminal.
  • Save spam_detector.py and run the script again:
python spam_detector.py

You should now see a sample spam email and a sample ham email printed below the class distribution. Notice the differences in tone, subject lines, and content.

Next, add some statistics that quantify the structural differences between spam and ham. These numbers will hint at what metadata features might be useful later.

  • Add the following code below the ham sample print statements you just added:
spam_msgs = df[df["Spam/Ham"] == "spam"]["Message"]
ham_msgs = df[df["Spam/Ham"] == "ham"]["Message"]
print(f"\nAvg spam message length: {spam_msgs.str.len().mean():.0f} chars")
print(f"Avg ham message length: {ham_msgs.str.len().mean():.0f} chars")
print(f"Emails with empty subject: {(df['Subject'] == '').sum()}")
print(f"Emails with empty message: {(df['Message'] == '').sum()}")

What do these statistics reveal?

Comparing average message lengths for spam and ham tells you whether spam emails tend to be shorter or longer. The empty subject and message counts reveal how many emails are missing data that a text-only model would rely on.

These structural differences are exactly the kind of signals that metadata features capture. A text-only model cannot see message length or empty fields as patterns.

  • Save spam_detector.py and run the script again:
python spam_detector.py

Below the sample emails, you should now see average message lengths for spam and ham, plus counts of emails with empty subjects and messages.

Seeing a KeyError or unexpected output?

Double-check that the column names in your code match the CSV exactly. The columns are Subject, Message, and Spam/Ham (case-sensitive, with a forward slash).

Help me fix my KeyError in the spam detector script

Create binary labels and split the data

Machine learning models work with numbers, not text labels. You need to convert the Spam/Ham column into binary values (1 for spam, 0 for ham) and then split the dataset so your model trains on one portion and gets tested on data it has never seen.

  • Add the following code at the bottom of spam_detector.py:
df["label"] = df["Spam/Ham"].map({"spam": 1, "ham": 0})

X_train, X_test, y_train, y_test = train_test_split(
    df, df["label"], test_size=0.2, random_state=42, stratify=df["label"]
)

print(f"\nTraining set size: {len(X_train)}")
print(f"Test set size: {len(X_test)}")

What does this code do?

  • .map({"spam": 1, "ham": 0}) converts the text labels into numbers. Every "spam" becomes 1 and every "ham" becomes 0.
  • train_test_split divides the data into 80% for training and 20% for testing. The stratify parameter ensures both sets keep the same spam/ham ratio as the original data.
  • random_state=42 fixes the random seed so you get the same split every time you run the script. This makes your results reproducible.
  • Save spam_detector.py.

Before you run the script, think about this: if the full dataset has 33,716 emails and you are using an 80/20 split, roughly how many emails do you expect in the training set?

  • Run the script to check your prediction:
python spam_detector.py

At the very bottom of the output, you should see the training set size (26,972) and test set size (6,744). That confirms your 80/20 split worked correctly.

Not seeing the training and test sizes?

Make sure the new code is at the very bottom of your file, after all the print statements from the previous chunks. Also check that the train_test_split import is present at the top of the file.

Help me fix my train/test split code

✔️ Awesome, I've got everything!

Great. Double check you have saved spam_detector.py before moving on.

ⓧ I'd like to double check the full code

Scroll up through your file and confirm these sections are present, in this order:

  • The import block (9 lines starting with import pandas as pd).
  • The dataset loading section: pd.read_csv, two fillna calls, and three print statements for shape and distribution.
  • The sample email section: one spam sample and one ham sample, each printing subject and message.
  • The statistics section: spam_msgs and ham_msgs variables plus four print statements for lengths and empty counts.
  • The label and split section: df["label"] mapping, train_test_split call, and two print statements for set sizes.

You have loaded, explored, and split 33,716 real emails. You have seen what spam and ham look like up close. Next up, you will build a text-only classifier to see how far raw email content alone can take you.

Build and Test a Text-Only Baseline Classifier

You have 33,716 Enron emails loaded, labelled, and split into training and test sets. The next question is straightforward: can a model learn to detect spam just by reading the words in each email?

In this step, you will build a text-only classifier and see exactly where it falls short. Seeing those failures is the point. They reveal the blind spots that the fusion approach in the next step is designed to fix.

In this step, get ready to:
  • Vectorize email text with TF-IDF and train a Random Forest classifier.
  • Evaluate the model's accuracy and identify specific misclassified emails.
  • Save baseline metrics for comparison with the fused model.
Vectorize email text, train a classifier, and evaluate it

A machine learning model cannot read raw text. It needs numbers. TF-IDF (Term Frequency-Inverse Document Frequency) converts each email into a vector of numbers where each number reflects how important a word is to that email relative to the full dataset.

  • In spam_detector.py, scroll to the bottom of your file (after the print(f"Test set size: {len(X_test)}") line) and add the following code:
# ============================================================
# STEP 3: Build and Test a Text-Only Baseline Classifier
# ============================================================

tfidf = TfidfVectorizer(max_features=5000, stop_words="english")
X_train_tfidf = tfidf.fit_transform(X_train["Message"])
X_test_tfidf = tfidf.transform(X_test["Message"])

rf_baseline = RandomForestClassifier(n_estimators=100, random_state=42)
rf_baseline.fit(X_train_tfidf, y_train)

y_pred_baseline = rf_baseline.predict(X_test_tfidf)

print("\n" + "=" * 60)
print("TEXT-ONLY BASELINE MODEL")
print("=" * 60)
baseline_accuracy = accuracy_score(y_test, y_pred_baseline)
print(f"Accuracy: {baseline_accuracy:.4f}")
print("\nClassification Report:")
print(classification_report(y_test, y_pred_baseline, target_names=["Ham", "Spam"]))

baseline_report = classification_report(
    y_test, y_pred_baseline, target_names=["Ham", "Spam"], output_dict=True
)
baseline_f1 = baseline_report["Spam"]["f1-score"]

What does this code do?

TfidfVectorizer converts each email's text into a vector of 5,000 numbers. Each number represents how important a particular word is to that email. The stop_words="english" setting filters out common words like "the" and "is" that don't help distinguish spam from ham.

RandomForestClassifier builds 100 decision trees that each vote on whether an email is spam or ham. The majority vote wins. random_state=42 locks the randomness so you get the same results every time you run the script.

The last block prints a classification report with precision, recall, and F1-score for each class. It also saves baseline_accuracy and baseline_f1 into variables so you can compare them against the fused model later.

Before you run this, what accuracy do you think a model that only reads the words in an email can achieve? Take a guess.

  • Save spam_detector.py.
  • Run the script in your terminal by running:
python spam_detector.py

After the Step 2 output, the script pauses for a few seconds while it trains the Random Forest on 26,000+ emails. You should then see a TEXT-ONLY BASELINE MODEL header followed by the accuracy score and a full classification report showing precision, recall, and F1-score for both Ham and Spam.

What is a classification report?

Precision tells you how many of the emails the model flagged as spam actually were spam. Recall tells you how many of the actual spam emails the model caught.

F1-score combines both into a single number. A model can have high accuracy but still miss dangerous spam if recall is low. That is why F1-score matters more than raw accuracy for spam detection.

Script taking a long time or crashing?

  • Training on 26,000+ emails is expected to take 10-30 seconds depending on your machine. If it takes over 2 minutes, try closing other applications to free up memory.
  • If you see a NameError for X_train or y_train, make sure your Step 2 code (the train_test_split call) is still in the file above this new code.
  • If you see a KeyError for Message, check that enron_spam_data.csv is in your spam-detector folder and that the CSV has a column named Message.

Still stuck? Ask for help.

Identify where the text-only model fails

The accuracy number looks impressive. But raw accuracy can hide critical failures. Some spam emails slip right past a text-only model because their words look perfectly normal.

  • Add the following code to the bottom of spam_detector.py to find and print the emails the model got wrong:
misclassified_mask = y_pred_baseline != y_test.values
misclassified = X_test[misclassified_mask]
print(f"\nMisclassified emails: {misclassified_mask.sum()} out of {len(y_test)}")
print("\n--- Examples of misclassified emails ---")
for i, (idx, row) in enumerate(misclassified.head(5).iterrows()):
    actual = "SPAM" if y_test[idx] == 1 else "HAM"
    predicted = "SPAM" if y_pred_baseline[list(X_test.index).index(idx)] == 1 else "HAM"
    print(f"\n[{i+1}] Actual: {actual} | Predicted: {predicted}")
    print(f"    Subject: {row['Subject'][:80]}")
    print(f"    Message: {row['Message'][:120]}")

What does this code do?

misclassified_mask is a boolean array that marks every email where the model's prediction disagrees with the actual label. The loop then prints the subject and first 120 characters of each misclassified email so you can inspect them yourself.

Pay attention to the patterns. Are the failures emails with very short bodies? Emails that sound professional but are actually spam? These are the blind spots a text-only model cannot see.

Before you run the script, do you think every misclassified email will have obvious spam words? Or will some look like completely normal emails?

  • Save spam_detector.py and run the script again:
python spam_detector.py

After the classification report, you should see the total number of misclassified emails followed by 5 specific examples. Each example shows the actual label, the predicted label, the subject line, and a preview of the message body.

Look at the misclassified emails carefully. Some spam emails have short or empty message bodies. Some use professional language that reads like a legitimate email. The text model has no way to detect structural spam signals like excessive uppercase, URLs, or suspicious punctuation patterns.

Not seeing the misclassified emails?

  • Make sure the new code is placed after the baseline_f1 line, not inside any other block.
  • Check that y_pred_baseline and y_test are defined earlier in the file. If you see a NameError, scroll up and verify the training code is present.
  • Your terminal output may be long. Scroll up past the Step 2 output and classification report to find the misclassified examples at the very bottom.

Need help debugging?

✔️ Awesome, I've got everything!

Great. Double check you have saved spam_detector.py with all of the Step 3 code.

ⓧ I'd like to double check the full code

Your spam_detector.py file should now contain all of the following sections in order. Scroll up through your file and confirm each one is present:

  • Import statements at the top (pandas, matplotlib, re, and all sklearn imports).
  • Step 2 code: loading the CSV, exploring the data, creating labels, and the train/test split.
  • Step 3 chunk 1: TF-IDF vectorization, Random Forest training, predictions, classification report printout, and baseline_accuracy / baseline_f1 saved.
  • Step 3 chunk 2: misclassified email detection and the loop printing 5 examples.

Your text-only baseline is trained, measured, and its blind spots are exposed. Those misclassified emails are not random failures. They are emails where the words alone do not tell the full story. Next up, you will engineer metadata features that capture the structural signals the text model missed and fuse them together into a single, stronger classifier.

Engineer Metadata Features and Fuse with Content

Your text-only model scored well overall, but the misclassified emails from the last step told a clear story. Some spam slipped through because the words alone looked legitimate.

In this step, you will engineer metadata features that capture structural patterns in emails. Things like subject length, uppercase ratio, and URL presence. Then you will fuse those features with the text features using scikit-learn's ColumnTransformer to build a single model that sees both what an email says and how it looks.

In this step, get ready to:
  • Create a feature engineering function that extracts seven metadata signals from each email.
  • Build a fused pipeline that combines TF-IDF text features with metadata features using a ColumnTransformer.
  • Compare the fused model's accuracy and F1-score against the text-only baseline, and re-check previously misclassified emails.
Write the feature engineering function

Spam emails often share structural fingerprints that have nothing to do with the words they use. An all-caps subject, a suspiciously long body stuffed with URLs, or a wall of exclamation marks are patterns your text-only model cannot see because TF-IDF only looks at word frequencies.

You will capture these patterns by writing a function that calculates seven metadata features from each email's raw text.

  • In spam_detector.py, scroll to the bottom of your file (after the misclassified emails loop) and add the following feature engineering function and its application:
# ============================================================
# STEP 4: Engineer Metadata Features and Fuse with Content
# ============================================================

def engineer_features(dataframe):
    data = dataframe.copy()
    data["subject_length"] = data["Subject"].str.len()
    data["message_length"] = data["Message"].str.len()
    data["uppercase_ratio"] = data["Message"].apply(
        lambda x: sum(1 for c in x if c.isupper()) / max(len(x), 1)
    )
    data["has_url"] = data["Message"].apply(
        lambda x: 1 if re.search(r"http|www", x, re.IGNORECASE) else 0
    )
    data["special_char_ratio"] = data["Message"].apply(
        lambda x: sum(1 for c in x if not c.isalnum() and not c.isspace()) / max(len(x), 1)
    )
    data["subject_word_count"] = data["Subject"].str.split().str.len().fillna(0).astype(int)
    data["exclamation_count"] = (data["Subject"] + " " + data["Message"]).str.count("!")
    return data

X_train_feat = engineer_features(X_train)
X_test_feat = engineer_features(X_test)

What does this code do?

The engineer_features() function takes a DataFrame and adds seven new columns, each capturing a different structural signal.

  • subject_length and message_length measure how long each part of the email is. Spam tends to be either very short (phishing links) or very long (keyword-stuffed).
  • uppercase_ratio calculates what fraction of characters are uppercase. "CLICK HERE NOW" scores high.
  • has_url checks for "http" or "www" in the message. Spam frequently includes links.
  • special_char_ratio and exclamation_count pick up punctuation-heavy formatting that spammers love.

The last two lines apply this function to both the training and test sets, creating X_train_feat and X_test_feat with the seven new metadata columns alongside the original data.

  • Save spam_detector.py.
  • Run the script to confirm the new function executes without errors by running:
python spam_detector.py

You should see all the Step 2 and Step 3 output you saw before. The script finishes without any new errors, which confirms the feature engineering ran successfully on all 33,716 emails.

Seeing a KeyError or AttributeError?

  • Check that the column names in your function match the DataFrame exactly. The dataset uses Subject and Message with capital first letters.
  • Make sure the import re line is still at the top of your file. The has_url feature uses re.search().
  • Still stuck? Help me debug my engineer_features function. I'm seeing this error when I run spam_detector.py.
Build the fusion pipeline with ColumnTransformer

Now you have two types of data for each email: the raw text in the Message column, and seven numeric metadata columns. The challenge is that text needs TF-IDF vectorization while metadata is already numeric.

What is a ColumnTransformer?

A ColumnTransformer lets you apply different transformations to different columns and then stitch the results together into a single feature matrix. Here, it applies TF-IDF to the text column and passes the numeric metadata columns straight through, untouched.

This is called feature-level fusion. The classifier sees text and metadata side by side in every training example, so it can learn how they interact.

  • Add the metadata feature list and the ColumnTransformer preprocessor below the X_test_feat line by copying this code:
metadata_features = [
    "subject_length",
    "message_length",
    "uppercase_ratio",
    "has_url",
    "special_char_ratio",
    "subject_word_count",
    "exclamation_count",
]

preprocessor = ColumnTransformer(
    transformers=[
        ("tfidf", TfidfVectorizer(max_features=5000, stop_words="english"), "Message"),
        ("metadata", "passthrough", metadata_features),
    ]
)

What does this code do?

  • metadata_features is a list of the seven column names you just engineered. This tells the ColumnTransformer which columns to pass through as-is.
  • The "tfidf" transformer applies TfidfVectorizer to the Message column, just like the baseline. The "metadata" transformer uses "passthrough" to keep the numeric features unchanged.
  • Now add the fused pipeline, train it, and print the fused model's evaluation below the preprocessor block:
fused_pipeline = Pipeline([
    ("preprocessor", preprocessor),
    ("classifier", RandomForestClassifier(n_estimators=100, random_state=42)),
])

fused_pipeline.fit(X_train_feat, y_train)

y_pred_fused = fused_pipeline.predict(X_test_feat)

print("\n" + "=" * 60)
print("FUSED MODEL (Content + Metadata)")
print("=" * 60)
fused_accuracy = accuracy_score(y_test, y_pred_fused)
print(f"Accuracy: {fused_accuracy:.4f}")
print("\nClassification Report:")
print(classification_report(y_test, y_pred_fused, target_names=["Ham", "Spam"]))

fused_report = classification_report(
    y_test, y_pred_fused, target_names=["Ham", "Spam"], output_dict=True
)
fused_f1 = fused_report["Spam"]["f1-score"]

What does this code do?

  • Pipeline chains the ColumnTransformer preprocessor with a Random Forest classifier into a single object. When you call .fit(), it transforms the data and trains the model in one step.
  • fused_report saves the classification report as a dictionary so you can extract the spam F1-score for comparison in the next chunk.
  • Save spam_detector.py and run the script:
python spam_detector.py

After the text-only baseline output you saw before, you should now see a new section titled FUSED MODEL (Content + Metadata) with its own accuracy and classification report. Training may take 10-20 seconds longer than before because the model is processing more features.

Seeing a ValueError about column shapes?

  • Make sure X_train_feat and X_test_feat are the DataFrames returned by engineer_features(), not the original X_train and X_test. The pipeline expects the seven metadata columns to exist.
  • Verify the strings in your metadata_features list match the column names your function creates exactly. A typo like "subject_len" instead of "subject_length" will cause a column mismatch.
  • Still stuck? Help me fix a ValueError in my ColumnTransformer pipeline.
Compare the fused model with the baseline

You have both models trained and evaluated. Now you will print a direct side-by-side comparison and revisit the specific emails the text-only model got wrong to see if fusion fixed them.

  • Add the comparison and re-check code below the fused_f1 line:
print("\n" + "=" * 60)
print("COMPARISON: Text-Only vs Fused")
print("=" * 60)
print(f"Accuracy:  {baseline_accuracy:.4f} -> {fused_accuracy:.4f} ({fused_accuracy - baseline_accuracy:+.4f})")
print(f"Spam F1:   {baseline_f1:.4f} -> {fused_f1:.4f} ({fused_f1 - baseline_f1:+.4f})")

fused_misclassified_mask = y_pred_fused != y_test.values
print(f"\nMisclassified: {misclassified_mask.sum()} (text-only) -> {fused_misclassified_mask.sum()} (fused)")

previously_wrong = X_test[misclassified_mask]
for idx in previously_wrong.head(5).index:
    test_indices = list(X_test.index)
    pos = test_indices.index(idx)
    old_pred = "SPAM" if y_pred_baseline[pos] == 1 else "HAM"
    new_pred = "SPAM" if y_pred_fused[pos] == 1 else "HAM"
    actual = "SPAM" if y_test[idx] == 1 else "HAM"
    if old_pred != new_pred:
        print(f"  Fixed: '{X_test.loc[idx, 'Subject'][:60]}' | Was: {old_pred} -> Now: {new_pred} (Actual: {actual})")

What does this code do?

  • The first two print lines show the accuracy and spam F1-score side by side, with a + or - showing the exact change.
  • fused_misclassified_mask counts the total misclassified emails for the fused model so you can compare the raw error count.
  • The loop revisits the five emails the text-only model got wrong. If the fused model now predicts them correctly, the script prints "Fixed" with the old and new predictions.
  • Save spam_detector.py.

Before you run this, think for a moment. Do you expect the fused model to have a higher or lower F1-score than the text-only baseline? By how much?

  • Run the script to see the full comparison:
python spam_detector.py

Scroll to the COMPARISON: Text-Only vs Fused section. You should see a positive improvement in both accuracy and spam F1-score. The total misclassified count drops, and some of the previously wrong emails now show as "Fixed".

Why does fusion work?

The text-only model only knows which words appear in an email. It misses structural clues like a message full of uppercase letters, a suspicious URL, or a subject line crammed with exclamation marks.

By fusing text features with metadata features, the classifier can catch spam that uses innocent-sounding words but has spammy structure. The "Fixed" emails in your output are proof of exactly this.

Not seeing any improvement?

  • Double check that your fused pipeline uses X_train_feat (with metadata columns), not X_train (without). If the metadata columns are missing, the "passthrough" transformer has nothing to pass.
  • Make sure random_state=42 is set on both the baseline and fused RandomForestClassifier. Different random states produce different results, making comparison unreliable.
  • Still stuck? Help me understand why my fused model is not outperforming the baseline.

Did any of the chunks in your full file get missed? Double check before moving on.

✔️ Awesome, I've got everything!

Great. Make sure you have saved spam_detector.py before continuing.

ⓧ I'd like to double check the full code

Your spam_detector.py is now too long to display in one block. Scroll through your file and confirm these sections appear in order:

  • Imports at the top: pandas, matplotlib, re, and all scikit-learn imports.
  • STEP 2: Load CSV, fill NaN, print shape, sample emails, stats, create labels, train/test split.
  • STEP 3: TF-IDF + Random Forest baseline, classification report, misclassified examples loop.
  • STEP 4: engineer_features() function, apply to train/test, metadata_features list, ColumnTransformer preprocessor, fused_pipeline + fit + predict, fused model report, comparison print, and re-check loop.

Your fused model is trained and proven to beat the baseline. Next up, you will visualize the improvement in a bar chart and build a prediction function that classifies any email you paste in.

Visualize the Improvement and Build a Prediction Function

Your fused model now outperforms the text-only baseline, and the printed comparison proves it with hard numbers. Numbers in a terminal are convincing for you, but they are hard to share or skim at a glance.

In this step, you will create a side-by-side bar chart that makes the improvement unmistakable in a single image. Then you will build a prediction function that turns your trained model into something you can actually use and demo on any email.

In this step, get ready to:
  • Create a bar chart comparing text-only vs fused model on accuracy, precision, recall, and F1-score.
  • Build a reusable prediction function that classifies any email you paste in.
  • Test predictions on sample emails and see confidence scores.
Create a comparison bar chart

A bar chart comparing both models on the same four metrics lets you see exactly where fusion helps. You will use matplotlib to build a grouped bar chart and save it as an image file.

  • In spam_detector.py, scroll to the very bottom of the file (after the Step 4 comparison output).
  • Add the following code to gather the four key metric scores from both models:
# ============================================================
# STEP 5: Visualize and Build Prediction Function
# ============================================================

# Collect the four key metrics from both models
metrics = ["Accuracy", "Precision", "Recall", "F1-Score"]
baseline_scores = [
    baseline_accuracy,
    baseline_report["Spam"]["precision"],
    baseline_report["Spam"]["recall"],
    baseline_report["Spam"]["f1-score"],
]
fused_scores = [
    fused_accuracy,
    fused_report["Spam"]["precision"],
    fused_report["Spam"]["recall"],
    fused_report["Spam"]["f1-score"],
]

x = range(len(metrics))
width = 0.35

What does this code do?

You are pulling precision, recall, and F1-score specifically for the "Spam" class from the classification report dictionaries you saved in earlier steps. The x and width variables control the bar positions so the two groups sit side by side.

  • Directly below, add the chart creation and save code:
# Create a grouped bar chart
fig, ax = plt.subplots(figsize=(10, 6))
bars1 = ax.bar([i - width / 2 for i in x], baseline_scores, width, label="Text-Only", color="#4a90d9")
bars2 = ax.bar([i + width / 2 for i in x], fused_scores, width, label="Fused (Content + Metadata)", color="#e74c3c")

# Configure axis labels, title, and scale
ax.set_ylabel("Score")
ax.set_title("Spam Detection: Text-Only vs Content-Metadata Fusion")
ax.set_xticks(list(x))
ax.set_xticklabels(metrics)
ax.set_ylim(0.85, 1.0)
ax.legend()

# Annotate each bar with its numeric score
for bar in bars1:
    ax.text(bar.get_x() + bar.get_width() / 2, bar.get_height() + 0.003,
            f"{bar.get_height():.3f}", ha="center", va="bottom", fontsize=9)
for bar in bars2:
    ax.text(bar.get_x() + bar.get_width() / 2, bar.get_height() + 0.003,
            f"{bar.get_height():.3f}", ha="center", va="bottom", fontsize=9)

plt.tight_layout()
plt.savefig("model_comparison.png", dpi=150)
print("\nComparison chart saved as model_comparison.png")
plt.show()

What does this code do?

  • plt.subplots() creates the chart canvas. Each ax.bar() call draws one group of bars, offset left or right so they sit side by side.
  • set_ylim(0.85, 1.0) zooms the y-axis into the 85-100% range so the differences between models are visually clear.
  • The two for bar loops add the exact numeric score as a label above each bar for precision.
  • plt.savefig() writes the chart to model_comparison.png so you can share it. plt.show() opens it in a window on your screen.
  • Save spam_detector.py.
  • Run the script by running this command in your terminal:
python spam_detector.py

After the earlier training output scrolls by, a chart window appears showing side-by-side bars for both models. The fused model's bars should be taller across all four metrics.

Close the chart window to continue

plt.show() pauses the script while the chart window is open. Close the chart window to let the script finish. You will also see Comparison chart saved as model_comparison.png printed in the terminal.

  • Confirm the image file was saved by running:
ls model_comparison.png

You should see model_comparison.png listed in the output.

Chart window didn't appear?

  • Some systems run Python without a GUI display backend. If you see a backend warning, the chart still saved to model_comparison.png. Open the file directly from your file explorer to view it.
  • Make sure your chart code comes after all the Step 4 code, not inside a function or indented block.

Why isn't my matplotlib chart showing up?

Build a prediction function

A trained model sitting in a script is useful for evaluation, but it becomes a real tool when you can feed it any email and get an instant verdict. The predict_email() function wraps the entire pipeline (feature engineering, prediction, probability scoring) into a single call.

  • In spam_detector.py, add the prediction function below the plt.show() line:
def predict_email(subject, message):
    # Build a single-row DataFrame matching the training format
    input_df = pd.DataFrame({"Subject": [subject], "Message": [message]})
    # Engineer the same metadata features the model was trained on
    input_df = engineer_features(input_df)
    # Get prediction and probability from the fused pipeline
    prediction = fused_pipeline.predict(input_df)[0]
    probabilities = fused_pipeline.predict_proba(input_df)[0]
    label = "SPAM" if prediction == 1 else "HAM"
    confidence = probabilities[prediction]
    return label, confidence

What does this function do?

  • It creates a one-row DataFrame from the subject and message you pass in, matching the format the model was trained on.
  • engineer_features() computes all seven metadata features (subject length, uppercase ratio, URL detection, etc.) exactly as it did during training.
  • predict_proba() returns the model's confidence for each class. The function picks the probability for whichever class it predicted.
Test predictions on sample emails

Time to see your model in action. You will classify three emails: one that is obviously spam, one that is clearly legitimate, and one that is ambiguous.

  • Add the test code below the predict_email function definition:
print("\n" + "=" * 60)
print("TESTING PREDICTION FUNCTION")
print("=" * 60)

# Test with clearly spam, clearly legitimate, and ambiguous emails
test_emails = [
    (
        "CONGRATULATIONS! You've won $1,000,000!!!",
        "Click HERE now to claim your PRIZE!!! Limited time offer, act NOW! www.claim-prize-now.com",
    ),
    (
        "Q3 Report",
        "Hi team, attached is the Q3 report for review. Please send your feedback by Friday. Thanks, Sarah",
    ),
    (
        "Important update",
        "Dear valued customer, your account requires immediate verification. Please update your details.",
    ),
]

for subject, message in test_emails:
    label, confidence = predict_email(subject, message)
    print(f"\nSubject: {subject}")
    print(f"Prediction: {label} (confidence: {confidence:.2%})")

What does this code do?

  • The first email is loaded with classic spam signals: all caps, exclamation marks, money claims, and a URL. This should be classified as SPAM with high confidence.
  • The second email is a routine workplace message. This should be classified as HAM with high confidence.
  • The third email sits in a gray area. It uses urgent language ("immediate verification") that is common in phishing, so the model's confidence may be lower.
  • Save spam_detector.py.

Before you run the script, think about the third email ("Dear valued customer, your account requires immediate verification"). Do you think the model will classify it as SPAM or HAM? What metadata signals might push it one way or the other?

  • Run the script to see predictions by running this command:
python spam_detector.py

After the chart window appears and you close it, the terminal prints each test email's subject, its prediction (SPAM or HAM), and the model's confidence as a percentage. The "CONGRATULATIONS" email should be classified as SPAM with high confidence.

Seeing a NameError for predict_email?

  • Make sure the def predict_email function definition appears before the test code that calls it. Python reads top to bottom, so the function must be defined first.
  • Check that the function is at the top level (no indentation), not accidentally nested inside another block.

Help me fix a NameError in my spam detector script.

✔️ Awesome, I've got everything!

Great. Double check you have saved spam_detector.py and that model_comparison.png exists in your spam-detector folder.

ⓧ I'd like to double check the full code

Your spam_detector.py file is long now. Scroll through it and confirm these sections appear in order:

  • Imports at the top (pandas, matplotlib, re, and all sklearn imports).
  • Step 2: Load and Explore (loading CSV, printing stats, train/test split).
  • Step 3: Text-Only Baseline (TF-IDF, Random Forest, classification report, misclassified examples).
  • Step 4: Feature Engineering and Fusion (engineer_features(), ColumnTransformer, fused pipeline, comparison print).
  • Step 5: Visualization (metrics lists, bar chart, plt.savefig(), plt.show()).
  • Step 5 continued: predict_email() function definition, then the test_emails list and the loop that prints predictions.

Your spam detector is complete. It loads real emails, trains two models, proves that fusing content with metadata outperforms text alone, and classifies any new email you throw at it. Next up: an optional Secret Mission to push your F1-score even higher by experimenting with different classifiers and hyperparameters.

Secret mission

Tune and Experiment for Maximum F1-Score

Your fused model already outperforms the text-only baseline. Can you push the F1-score even higher by swapping classifiers, tuning TF-IDF settings, and adding custom metadata features? Real-world ML work is all about iteration, and this mission puts you in that driver's seat.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs or cloud resources to worry about. Decide whether to keep your project files and libraries, or remove them from your machine.

Resources you used:

  • The spam-detector project folder on your Desktop (contains spam_detector.py, enron_spam_data.csv, and model_comparison.png).
  • Python packages installed via pip: scikit-learn, pandas, and matplotlib.

Keep everything running

No action needed. Choose this if you plan to keep experimenting with your spam detector or want to revisit the code later.

  • Your spam-detector folder stays on your Desktop with all your code and data.
  • The scikit-learn, pandas, and matplotlib packages remain installed in your Python environment.

Pause - I'll come back to this later

Close VS Code to free up memory. Your project files and installed packages will still be there when you return.

  • Close the VS Code window.
  • Your spam-detector folder and all installed packages remain untouched on your machine.
  • To resume later, open VS Code and reopen the spam-detector folder from your Desktop.

Delete - I don't want to use this again

Remove all project files and uninstall the Python packages to start fresh.

  • Close VS Code.
  • Delete the project folder by running this command in your terminal (macOS/Linux):
rm -rf ~/Desktop/spam-detector

On Windows, run this instead:

rmdir /s /q %USERPROFILE%\Desktop\spam-detector
  • Uninstall the Python packages by running:
pip uninstall scikit-learn pandas matplotlib -y
  • Confirm the folder is gone by checking your Desktop. The spam-detector folder should no longer appear.

Nice Work!

Nice Work!

You just built an adaptive spam detection system that learns from both what an email says and how it looks. You can now paste any email into your script and get an instant spam-or-ham prediction backed by a confidence score.

What you learned:

  • Built a text-only baseline classifier using TF-IDF and Random Forest on 33,716 real Enron emails, then pinpointed the specific emails it misclassified to understand where content-only detection falls short.
  • Engineered metadata features from email structure (subject length, uppercase ratio, URL detection, special character density) and fused them with text features using scikit-learn's ColumnTransformer to measurably improve detection accuracy over the baseline.
  • Visualized the improvement with a comparison bar chart and built a reusable predict_email() function that classifies any email you paste in with a confidence score.
  • Secret Mission: Experimented with different classifiers, TF-IDF hyperparameters, and custom metadata features to push your F1-score higher.

Ready to quiz yourself?