PhishScope AI: Explainable Scam Detector

Build a local dashboard that detects and explains SMS scam messages with NLP.

Introduction

30 Second Summary

A suspicious text can look ordinary. A harmless message can also contain words that sound urgent.

In this project, you will build PhishScope AI, a local dashboard that checks pasted messages for spam risk. You will train an explainable text classifier on public SMS examples.

What You'll Build

Your finished dashboard turns a pasted message into a visible risk label, a spam probability, and a table of the learned phrases that influenced the decision.

By the end of this project, you'll have:

  • A local risk dashboard that classifies any pasted message as spam or ham while displaying its spam probability.
  • A measured NLP pipeline that reports held-out accuracy plus spam-focused precision, recall, F1 score, and a confusion matrix.
  • A matched-signals table that shows which learned phrases raise or lower the predicted spam risk for each message.
  • Secret Mission: Harden the model against common leetspeak plus inserted punctuation so Fr33 pr1ze waiting. Cl1ck n0w. surfaces normalized scam vocabulary.

Do I need any experience before starting?

The project uses a Windows computer with Python 3.11 through 3.14 plus Visual Studio Code. A GitHub account, internet access for the first download, plus basic confidence reading Python code complete the prerequisites.

Before We Start

Before the hands-on work begins, take a moment to commit to the purpose behind PhishScope AI. You will define who it helps assess suspicious SMS messages and why each prediction needs matched evidence for safer decisions.

Prepare the Windows ML Workspace

Your detector now has a clear user and purpose. The next foundation is a local workspace that can run its machine learning dashboard consistently.

A reproducible environment prevents package drift. You will confirm a supported Python runtime before installing exact dependency versions inside an isolated virtual environment.

In this step, get ready to:
  • Open the phishscope-ai folder in Visual Studio Code.
  • Activate a supported Python runtime inside .venv.
  • Prove the pinned packages work with the Streamlit test page.
Confirm a supported Python runtime

Visual Studio Code keeps your project files beside an integrated PowerShell terminal. Keeping the folder on your Desktop also gives the rest of the project a predictable location.

  • Press the Windows key to open Windows Search.
  • Type Visual Studio Code.
  • Press Enter to launch Visual Studio Code.
  • Select File from the top menu.
  • Select Open Folder.
  • Select Desktop in the folder picker.
  • Click New folder.
  • Enter phishscope-ai as the folder name.

Why use one project folder?

The phishscope-ai folder keeps the virtual environment beside every file you create later. Visual Studio Code can also start each integrated terminal from this location.

  • Select the new phishscope-ai folder.
  • Click Select Folder.
  • Confirm that you trust the folder if Visual Studio Code displays a workspace trust prompt.

Your project now has a fixed home. You should see phishscope-ai at the top of the Explorer sidebar.

  • Select Terminal from the top menu.
  • Select New Terminal.
  • Choose PowerShell from the terminal profile selector if another shell opens.
  • List the Python runtimes available on your computer by running this command:
py list

What does this command check?

The Python Install Manager lists the runtimes available on your computer. This project supports versions 3.11, 3.12, 3.13, or 3.14.

Choose the tab that matches your runtime list.

✔️ I see a supported version

Your runtime list includes Python 3.11, 3.12, 3.13, or 3.14. Your computer is ready to create the project environment.

ⓧ I see an older version

The installed runtime falls outside the supported range for the pinned packages. Installing Python 3.14 gives the project a compatible runtime.

The runtime download can take a few minutes. A quiet terminal during this wait is normal.

  • Install Python 3.14 by running this command:
py install 3.14

What does this command do?

The Python Install Manager downloads the specified CPython runtime. The existing older runtime can remain on your computer.

  • Close the integrated terminal with the trash can button in its toolbar.
  • Create a new terminal from the Terminal menu.
  • Confirm that the supported runtime is now listed by running this command:
py list

What should the new list show?

The refreshed list should include Python 3.14. That version can run every pinned project dependency.

Still seeing only the older version?

Close every integrated terminal before creating a fresh PowerShell terminal. This lets Visual Studio Code detect the newly installed runtime.

If the supported runtime still does not appear, help me diagnose why the new Python runtime is missing.

ⓧ Command not found

Windows needs the official Python Install Manager before the runtime commands are available. Install it from the Microsoft Store first.

  • Press the Windows key to open Windows Search.
  • Type Microsoft Store.
  • Press Enter to launch the Microsoft Store.
  • Search for Python Install Manager.
  • Select the official Python Install Manager listing.
  • Click Install.

Why install the manager first?

The Python Install Manager provides the Windows runtime listing and installation commands used by this project. The Store finishes installing it before you return to Visual Studio Code.

  • Switch back to Visual Studio Code.
  • Close the existing integrated terminal with the trash can button.
  • Create a new PowerShell terminal from the Terminal menu.
  • Install Python 3.14 by running this command:
py install 3.14

What happens during installation?

The manager downloads the Python 3.14 runtime. This can take a few minutes on the first installation.

  • Confirm that Python 3.14 is available by running this command:
py list

What should the runtime list show?

The list should now include Python 3.14. This confirms the manager can find the supported runtime.

Still cannot run the manager?

Restart Visual Studio Code after the Microsoft Store installation finishes. A fresh application session can detect the newly installed manager.

If the command is still unavailable, help me troubleshoot the Python Install Manager on Windows.

Create and activate the virtual environment

A virtual environment gives this project its own package location. Activation makes python and pip use that isolated location.

  • Create the .venv environment inside phishscope-ai by running this command:
python -m venv .venv

What does this command create?

Python creates an isolated environment in the .venv folder. Packages installed after activation stay separate from your other Python projects.

  • Confirm that .venv appears in the Visual Studio Code Explorer sidebar.

Do not see the environment folder?

Check that the integrated terminal is inside the phishscope-ai folder. The terminal path should end with that folder name.

If the folder is still missing, help me troubleshoot virtual environment creation.

Activation tells PowerShell to use the environment for every Python command in this terminal. This keeps the package installation tied to PhishScope AI.

  • Activate the environment by running this command:
.venv\Scripts\Activate.ps1

What does activation change?

Activation redirects Python and package commands to the environment inside .venv. The terminal prompt identifies the active environment when the command succeeds.

Choose the tab that matches what PowerShell shows.

✔️ The environment is active

The PowerShell prompt now shows the environment name in parentheses. Your next package installation stays inside .venv.

ⓧ PowerShell blocks the script

The fallback changes the execution policy only for your Windows user account. This allows the environment activation script to run.

  • Enable the activation script for your user account by running this command:
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

What does this fallback change?

The current-user scope applies the execution policy to your account. PowerShell can then run the environment activation script.

  • Activate the environment again by running this command:
.venv\Scripts\Activate.ps1

What should change now?

The PowerShell prompt should identify the active environment in parentheses. Package commands now target .venv.

Activation still blocked?

Close the integrated terminal after applying the current-user policy. Create a fresh PowerShell terminal before trying the activation command again.

If PowerShell still blocks the script, help me troubleshoot the activation policy.

Install and test the pinned dependencies

The requirements.txt file records the exact package versions for this project. It pins pandas, scikit-learn, Streamlit, and ucimlrepo.

  • Click the New File button in the Visual Studio Code Explorer sidebar.
  • Enter requirements.txt as the file name.
  • Press Enter to create the file.
  • Populate requirements.txt by pasting this content:
pandas==3.0.6
scikit-learn==1.9.1
streamlit==1.65.0
ucimlrepo==0.0.7

What do these packages provide?

  • pandas 3.0.6 handles the dataset columns.
  • scikit-learn 1.9.1 provides the text features and classifier used later.
  • Streamlit 1.65.0 serves the local browser dashboard.
  • ucimlrepo 0.0.7 retrieves the public UCI dataset.
  • Press Ctrl+S to save requirements.txt.
  • Confirm that requirements.txt appears beside .venv in the Explorer sidebar.

File missing or named incorrectly?

Check that the file is inside phishscope-ai. Its full name must be requirements.txt.

If the file does not match, help me correct my requirements file.

✔️ Awesome, I've got everything!

Great. Double-check that requirements.txt is saved before installing the packages.

ⓧ I'd like to double check the full code

pandas==3.0.6
scikit-learn==1.9.1
streamlit==1.65.0
ucimlrepo==0.0.7

How should the file compare?

Your file should contain these four lines in this order. Each package name and version must match exactly.

The first dependency installation can take a few minutes while PowerShell downloads the packages. The terminal may print many progress lines during this process.

  • Install the pinned dependencies into the active environment by running this command:
pip install -r requirements.txt

What does this command install?

The installer reads every pinned entry from requirements.txt. It also resolves the supporting packages required by those four dependencies.

PowerShell returns to its prompt when the installation finishes. That is the dependency layer locked to the versions this project expects.

Package installation failed?

Confirm that the terminal prompt still identifies the active environment. Also check that the terminal path ends with phishscope-ai.

If the installation still fails, help me diagnose the dependency installation.

Pause for a prediction: decide whether the Streamlit example page will open from this environment.

  • Launch the Streamlit example from the active environment by running this command:
python -m streamlit hello

What does this final check prove?

Streamlit starts a local server from the active Python environment. It also opens its example app in your browser.

Your browser should open the Streamlit example page. You have now proved that the supported runtime and pinned packages can launch a local browser app.

Browser page did not open?

Check the PowerShell output for a local address. Paste that address into your browser if the page did not open automatically.

If the server does not start, help me troubleshoot the Streamlit example.

  • Press Ctrl+C in the integrated PowerShell terminal to stop the example server.

The PowerShell prompt returns while your environment stays active. Your Windows ML workspace is ready for the first PhishScope AI detector.

Build a Keyword Baseline

Your environment can now launch Streamlit apps in the browser. That working foundation gives you a fast way to turn detection logic into something you can test.

A security detector needs a baseline before you can measure an improvement. This version uses keyword matching to classify messages through a small set of literal risk signals.

In this step, get ready to:
  • Create a browser interface for pasted messages.
  • Classify messages with exact keyword matching.
  • Compare the detector's response to literal scam wording with its response to subtler wording.
Create the detector interface

The baseline needs a short list of risk signals. It also needs a text area where you can paste each message you want to inspect.

  • In the Explorer sidebar in Visual Studio Code, select the new-file control beside the phishscope-ai folder.
  • Type app.py in the file name field.
  • Press Enter to create the file.
  • Add the baseline interface to app.py by pasting this code:
import streamlit as st


RISKY_TERMS = ("free", "urgent", "winner", "prize", "cash")

st.title("PhishScope AI")
message = st.text_area("Message to analyze")

What does this code do?

  • The RISKY_TERMS tuple stores five literal signals that the detector can recognize.
  • The st.title() call displays the name of your detector.
  • The st.text_area() call creates the input where you paste a message.
  • Save app.py by pressing Ctrl+S.

The first launch may take a moment while Streamlit starts the local server.

  • Launch the temporary detector from the integrated PowerShell terminal by running this command:
python -m streamlit run app.py

What does this command do?

This command starts app.py as a local Streamlit app. It opens the interface in your browser.

Your first PhishScope AI screen is live. You will see the PhishScope AI heading above an empty Message to analyze text area.

App Does Not Open?

  • Confirm that the integrated terminal is inside the phishscope-ai folder.
  • Confirm that the terminal still shows the activated .venv environment.
  • Check that app.py is saved directly inside the phishscope-ai folder.
  • Help me troubleshoot why the app does not open.
Make the baseline respond

The detector checks a lowercase copy of the message against every term in RISKY_TERMS. The any() function reports whether at least one listed term occurs.

  • Switch back to the app.py tab.
  • Append the button and matching logic after message = st.text_area("Message to analyze") by pasting this code:
if st.button("Analyze message"):
    if any(term in message.lower() for term in RISKY_TERMS):
        st.warning("HIGH RISK")
    else:
        st.success("LOW RISK")

How Does the Decision Work?

  • The Analyze message button starts the classification check.
  • The generator compares each risky term with the lowercased message.
  • The st.warning() call displays HIGH RISK when a term matches.
  • The st.success() call displays LOW RISK when no term matches.
  • Save app.py.
  • Return to the browser page opened by the running Streamlit server.
  • Refresh the page.

You will now see an Analyze message button below the text area.

  • Enter URGENT! You have won a free prize. Reply now to claim it. in the Message to analyze text area.
  • Click Analyze message.

You will see HIGH RISK. The message contains several terms from RISKY_TERMS.

Do You See LOW RISK Instead?

  • Confirm that urgent, free, and prize appear in the RISKY_TERMS tuple.
  • Compare the indentation under if st.button("Analyze message"): with the reference below.
  • Help me debug the keyword result.

✔️ Awesome, I've got everything!

Your app.py file now matches the complete keyword baseline.

ⓧ I'd like to double check the full code

import streamlit as st


RISKY_TERMS = ("free", "urgent", "winner", "prize", "cash")

st.title("PhishScope AI")
message = st.text_area("Message to analyze")

if st.button("Analyze message"):
    if any(term in message.lower() for term in RISKY_TERMS):
        st.warning("HIGH RISK")
    else:
        st.success("LOW RISK")

How to Use This Reference

This is the complete temporary app.py file for this step. Compare its spelling and indentation with your saved file.

Test unseen scam wording

The first test proves that listed terms trigger the baseline. A stronger test uses suspicious wording that differs from the examples in the tuple.

Before you test the next message, do you expect the current rule to classify it as high risk or low risk?

  • Replace the text area content with Confirm your account details at the link below.
  • Click Analyze message.

You will see LOW RISK. This incorrect result is the intended shortfall of the keyword baseline.

Why Does the Rule Miss This Message?

The detector only recognizes the five literal strings inside RISKY_TERMS. The suspicious account language does not contain any of them.

Exact rules cannot generalize to scam wording they were never given. This visible failure defines the problem that a trained text classifier needs to solve.

You now have a working browser detector with a documented blind spot. Next, you will replace its fixed keyword list with a model that learns patterns from labeled messages.

Train and Measure the NLP Model

Your keyword detector exposed a generalization gap. The account-confirmation message received LOW RISK because its wording did not appear in RISKY_TERMS.

A supervised NLP model can learn patterns from labeled examples. A held-out evaluation then measures errors on messages excluded from training.

In this step, get ready to:
  • Load the labeled SMS dataset into a reproducible train/test split.
  • Train a TF-IDF pipeline with balanced logistic regression.
  • Measure held-out performance with classification metrics.
Load and split the labeled messages

The SMS Spam Collection contains labeled examples from the UCI Machine Learning Repository. A stratified train/test split preserves the class balance across the training set and test set.

  • In the Visual Studio Code Explorer sidebar from earlier, click the new-file control.
  • Name the new file model.py.

You should see model.py beside app.py and requirements.txt.

  • Add the dataset imports and loader to model.py by pasting this code:
from ucimlrepo import fetch_ucirepo
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report, confusion_matrix
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline


def load_dataset():
    dataset = fetch_ucirepo(id=228)
    messages = dataset.data.features.iloc[:, 0].astype(str)
    labels = dataset.data.targets.iloc[:, 0].astype(str)
    return messages, labels

What does this code do?

  • The UCI helper retrieves dataset ID 228.
  • The positional selectors read the first feature column and the first target column.
  • The scikit-learn imports provide splitting, feature extraction, model training, and evaluation.
  • Save model.py.
  • Confirm that load_dataset() appears below the six imports.

Seeing import warnings?

Check that the integrated terminal still shows the active .venv environment. Confirm that each import matches the snippet exactly.

If a package remains unavailable, ask for help checking the active environment.

A fixed random state gives each run the same held-out messages. This makes later comparisons reproducible.

  • Add the opening of train_model() below load_dataset() by pasting this code:
def train_model():
    messages, labels = load_dataset()
    x_train, x_test, y_train, y_test = train_test_split(
        messages,
        labels,
        test_size=0.2,
        random_state=42,
        stratify=labels,
    )

How does the split work?

  • The loader returns the message text and its class label.
  • The test_size=0.2 setting reserves one fifth of the dataset for testing.
  • The random_state=42 setting reproduces the same split.
  • The stratify=labels setting preserves the proportion of ham and spam messages.
  • Save model.py.
  • Confirm that the split creates x_train, x_test, y_train, and y_test.
Train the text classification pipeline

TF-IDF turns words and two-word phrases into weighted numeric features. Logistic regression learns how those features relate to the two message classes.

  • In train_model(), place the training pipeline directly below the split by pasting this code:
    pipeline = Pipeline(
        [
            (
                "tfidf",
                TfidfVectorizer(
                    ngram_range=(1, 2),
                    min_df=2,
                    sublinear_tf=True,
                ),
            ),
            (
                "classifier",
                LogisticRegression(
                    max_iter=1000,
                    class_weight="balanced",
                ),
            ),
        ]
    )
    pipeline.fit(x_train, y_train)

How does the pipeline learn?

  • The ngram_range=(1, 2) setting creates unigram and bigram features.
  • The min_df=2 setting removes terms found in only one message.
  • The sublinear_tf=True setting reduces the influence of repeated terms.
  • The class_weight="balanced" setting adjusts training for the smaller spam class.
  • Save model.py.
  • Confirm that pipeline.fit(x_train, y_train) remains inside train_model().

Why TF-IDF and Logistic Regression?

This combination trains quickly on sparse text features. Its learned coefficients can also support the matched-signal explanations added to the dashboard later.

The pipeline keeps the training process measurable. It also keeps each prediction inspectable.

Held-out predictions show how the fitted pipeline handles messages excluded from training. The next code calculates that evidence.

  • Add the held-out evaluation directly below pipeline.fit(x_train, y_train) by pasting this code:
    predictions = pipeline.predict(x_test)
    report = classification_report(
        y_test,
        predictions,
        output_dict=True,
        zero_division=0,
    )
    matrix = confusion_matrix(
        y_test,
        predictions,
        labels=["ham", "spam"],
    )

What does the evaluation calculate?

  • The fitted pipeline predicts a class for every held-out message.
  • The classification report calculates performance values for each class.
  • The confusion matrix counts actual classes against predicted classes.
  • The fixed label order keeps ham in the first row and spam in the second row.
  • Save model.py.
  • Confirm that report and matrix are created after predictions.

The dashboard needs a single bundle containing the trained pipeline and its measurements. The confusion counts also need readable row labels.

  • Complete train_model() by placing this return block below the confusion matrix code:
    return {
        "pipeline": pipeline,
        "dataset_size": len(messages),
        "accuracy": float(report["accuracy"]),
        "spam_precision": float(report["spam"]["precision"]),
        "spam_recall": float(report["spam"]["recall"]),
        "spam_f1": float(report["spam"]["f1-score"]),
        "confusion": [
            {
                "Actual class": "ham",
                "Predicted ham": int(matrix[0, 0]),
                "Predicted spam": int(matrix[0, 1]),
            },
            {
                "Actual class": "spam",
                "Predicted ham": int(matrix[1, 0]),
                "Predicted spam": int(matrix[1, 1]),
            },
        ],
    }

What is stored in the bundle?

The dictionary stores the trained pipeline and dataset size. It also stores the held-out metrics that the dashboard displays.

The confusion counts become two labeled rows. Their meaning stays clear when Streamlit renders them as a table.

  • Save model.py.
  • Confirm that the return block stays indented inside train_model().

A direct script entry point gives you a quick measurement check before the dashboard uses the model. It prints each headline result to the terminal.

  • Add the direct-run block at the bottom of model.py by pasting this code:
if __name__ == "__main__":
    model_bundle = train_model()
    print(f'Dataset size: {model_bundle["dataset_size"]}')
    print(f'Accuracy: {model_bundle["accuracy"]:.3f}')
    print(f'Spam precision: {model_bundle["spam_precision"]:.3f}')
    print(f'Spam recall: {model_bundle["spam_recall"]:.3f}')
    print(f'Spam F1: {model_bundle["spam_f1"]:.3f}')

Why add a direct-run block?

This block runs when you execute model.py directly. Importing the model functions elsewhere does not trigger these print statements.

  • Save model.py.

Before you run the model, which four performance measures do you expect beneath the dataset size?

  • Train the model and reveal its held-out measurements by running this command in the activated terminal:
python model.py

What should I see?

You should see Dataset size: 5574 followed by accuracy, spam precision, spam recall, and spam F1. Each metric appears to three decimal places.

The first download can take a moment because the dataset comes from UCI. The exact metric values can vary slightly across machines.

Training not completing?

Confirm that your internet connection is available for the first dataset download. Check that the terminal remains inside the phishscope-ai folder.

If Python reports a syntax or indentation problem, ask for help comparing train_model() with the intended structure.

Add reusable prediction and explanation helpers

Precision measures how often predicted spam is truly spam. Recall measures how much actual spam the model finds.

F1 score balances those two measures. A confusion matrix exposes the counts behind the summary metrics.

The model also needs reusable helpers for the dashboard. One helper returns the prediction while the other exposes matched feature weights.

  • Insert predict_message() above the direct-run block by pasting this code:
def predict_message(bundle, message):
    pipeline = bundle["pipeline"]
    probabilities = pipeline.predict_proba([message])[0]
    classes = list(pipeline.classes_)
    spam_probability = float(probabilities[classes.index("spam")])
    label = str(pipeline.predict([message])[0])
    return {
        "label": label,
        "spam_probability": spam_probability,
    }

How does prediction work?

The fitted pipeline returns probabilities in learned class order. The helper locates the spam class before selecting its probability.

The result contains a predicted label and a numeric spam probability. The dashboard uses both values in the next step.

  • Save model.py.
  • Confirm that predict_message() sits outside train_model().

A linear classifier assigns a learned coefficient to each feature. Matching those coefficients against the analyzed message creates an inspectable directional explanation.

  • Insert explain_message() below predict_message() by pasting this code:
def explain_message(bundle, message, limit=8):
    pipeline = bundle["pipeline"]
    vectorizer = pipeline.named_steps["tfidf"]
    classifier = pipeline.named_steps["classifier"]

    present_terms = set(vectorizer.build_analyzer()(message))
    feature_names = vectorizer.get_feature_names_out()
    weights = classifier.coef_[0]

    signals = []
    for feature, weight in zip(feature_names, weights):
        if feature in present_terms:
            numeric_weight = float(weight)
            signals.append(
                {
                    "Matched feature": str(feature),
                    "Direction": (
                        "Raises spam risk"
                        if numeric_weight > 0
                        else "Lowers spam risk"
                    ),
                    "Absolute weight": round(abs(numeric_weight), 3),
                }
            )

    signals.sort(key=lambda row: row["Absolute weight"], reverse=True)
    return signals[:limit]

How are matched signals selected?

  • The fitted analyzer extracts the message features.
  • The classifier coefficients provide the learned feature weights.
  • Positive weights point toward the spam class.
  • Sorting by absolute weight surfaces the strongest matched signals first.
  • Save model.py.

Before the final check, do you expect the two new helpers to change the printed training metrics?

  • Verify the completed model file by running this command:
python model.py

What should I see?

You should again see the dataset size plus held-out accuracy, spam precision, spam recall, and spam F1. The helpers do not change training because the direct-run block still calls the same train_model() function.

That is the measurement layer working. PhishScope AI now trains on public labeled data and reports how it performs on unseen messages.

Metrics missing after adding the helpers?

Check that both helper functions appear above the direct-run block. Confirm that each function starts at the left edge of model.py.

If the output still fails, ask for help checking the helper placement and indentation.

Use the tabs below to compare your saved file with the target code.

✔️ Awesome, I've got everything!

Your saved model.py now contains the training pipeline, held-out evaluation, prediction helper, and explanation helper.

ⓧ I'd like to double check the full code

from ucimlrepo import fetch_ucirepo
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report, confusion_matrix
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline


def load_dataset():
    dataset = fetch_ucirepo(id=228)
    messages = dataset.data.features.iloc[:, 0].astype(str)
    labels = dataset.data.targets.iloc[:, 0].astype(str)
    return messages, labels


def train_model():
    messages, labels = load_dataset()
    x_train, x_test, y_train, y_test = train_test_split(
        messages,
        labels,
        test_size=0.2,
        random_state=42,
        stratify=labels,
    )

    pipeline = Pipeline(
        [
            (
                "tfidf",
                TfidfVectorizer(
                    ngram_range=(1, 2),
                    min_df=2,
                    sublinear_tf=True,
                ),
            ),
            (
                "classifier",
                LogisticRegression(
                    max_iter=1000,
                    class_weight="balanced",
                ),
            ),
        ]
    )
    pipeline.fit(x_train, y_train)

    predictions = pipeline.predict(x_test)
    report = classification_report(
        y_test,
        predictions,
        output_dict=True,
        zero_division=0,
    )
    matrix = confusion_matrix(
        y_test,
        predictions,
        labels=["ham", "spam"],
    )

    return {
        "pipeline": pipeline,
        "dataset_size": len(messages),
        "accuracy": float(report["accuracy"]),
        "spam_precision": float(report["spam"]["precision"]),
        "spam_recall": float(report["spam"]["recall"]),
        "spam_f1": float(report["spam"]["f1-score"]),
        "confusion": [
            {
                "Actual class": "ham",
                "Predicted ham": int(matrix[0, 0]),
                "Predicted spam": int(matrix[0, 1]),
            },
            {
                "Actual class": "spam",
                "Predicted ham": int(matrix[1, 0]),
                "Predicted spam": int(matrix[1, 1]),
            },
        ],
    }


def predict_message(bundle, message):
    pipeline = bundle["pipeline"]
    probabilities = pipeline.predict_proba([message])[0]
    classes = list(pipeline.classes_)
    spam_probability = float(probabilities[classes.index("spam")])
    label = str(pipeline.predict([message])[0])
    return {
        "label": label,
        "spam_probability": spam_probability,
    }


def explain_message(bundle, message, limit=8):
    pipeline = bundle["pipeline"]
    vectorizer = pipeline.named_steps["tfidf"]
    classifier = pipeline.named_steps["classifier"]

    present_terms = set(vectorizer.build_analyzer()(message))
    feature_names = vectorizer.get_feature_names_out()
    weights = classifier.coef_[0]

    signals = []
    for feature, weight in zip(feature_names, weights):
        if feature in present_terms:
            numeric_weight = float(weight)
            signals.append(
                {
                    "Matched feature": str(feature),
                    "Direction": (
                        "Raises spam risk"
                        if numeric_weight > 0
                        else "Lowers spam risk"
                    ),
                    "Absolute weight": round(abs(numeric_weight), 3),
                }
            )

    signals.sort(key=lambda row: row["Absolute weight"], reverse=True)
    return signals[:limit]


if __name__ == "__main__":
    model_bundle = train_model()
    print(f'Dataset size: {model_bundle["dataset_size"]}')
    print(f'Accuracy: {model_bundle["accuracy"]:.3f}')
    print(f'Spam precision: {model_bundle["spam_precision"]:.3f}')
    print(f'Spam recall: {model_bundle["spam_recall"]:.3f}')
    print(f'Spam F1: {model_bundle["spam_f1"]:.3f}')

How to use this reference

Compare the imports, functions, values, and indentation with your saved model.py file. Keep the temporary app.py keyword detector unchanged for the next step.

Your measured model is ready. Next, you will connect it to the browser dashboard so anyone can classify a message and inspect its spam probability.

Connect the Model Dashboard

Your trained pipeline now turns the UCI SMS Spam Collection into held-out metrics. A browser user still cannot submit a message or see its confidence.

This step connects that model to a Streamlit dashboard. Caching keeps the trained model available across widget reruns.

In this step, get ready to:
  • Connect the dashboard to the trained model.
  • Cache the fitted model for widget reruns.
  • Display a label with its spam probability.
Connect and cache the model

Streamlit reruns your script when someone interacts with a widget. The cached resource keeps the trained model bundle ready for those reruns.

  • Return to app.py in the Explorer sidebar.
  • Replace all existing code by pasting this initialization code:
import streamlit as st

from model import explain_message, predict_message, train_model


st.set_page_config(page_title="PhishScope AI", layout="wide")


@st.cache_resource
def get_model_bundle():
    return train_model()


bundle = get_model_bundle()

st.title("PhishScope AI")
st.caption(
    "An explainable SMS spam detector trained locally on the UCI SMS Spam Collection."
)

What does this code do?

  • The model import gives the dashboard access to its three existing helpers.
  • The get_model_bundle() function trains the model for the cached resource.
  • The bundle variable stores the trained pipeline with its evaluation data.
  • The page configuration gives the dashboard a wide layout.
  • Save app.py.
  • Start the connected dashboard by running this command:
python -m streamlit run app.py

What does this command do?

This starts app.py as a local Streamlit server. It opens the dashboard in your browser.

The first launch may take a short while because train_model() retrieves the dataset before fitting the model.

That connection is live. You should see the PhishScope AI title with a caption describing the local SMS spam detector.

Dashboard not opening?

Confirm the integrated terminal still uses the activated .venv environment. Check that model.py remains beside app.py inside the phishscope-ai folder.

Compare the model import with the function names in model.py.

Help me diagnose why my Streamlit dashboard cannot import or train the model.

Add the message input

The dashboard needs a text widget that accepts an arbitrary message. A suspicious default message also gives you a repeatable test case.

  • Return to app.py from earlier.
  • Find the caption that starts with st.caption(.
  • Add the message input below the caption by pasting this code:
message = st.text_area(
    "Message to analyze",
    value="URGENT! You have won a free prize. Reply now to claim it.",
    height=150,
)

How does the input work?

  • The message variable stores the text currently inside the widget.
  • The default value gives the detector an obvious scam message to analyze.
  • The fixed height leaves enough room for longer messages.
  • Save app.py.
  • Refresh the dashboard in your browser.

You should see a large Message to analyze text area containing the default prize message.

Message box missing?

Check that the new block sits below the caption's closing parenthesis. Confirm that each argument in st.text_area ends with the same comma shown above.

Help me find why the message text area is missing from my Streamlit dashboard.

Display the prediction and probability

The final dashboard flow sends the pasted text to predict_message(). It turns the returned label with its probability into visible risk feedback.

  • Find the text area block that contains height=150,.
  • Add the analysis flow below the text area by pasting this code:
if st.button("Analyze message", type="primary", width="stretch"):
    if not message.strip():
        st.warning("Enter a message before running the detector.")
    else:
        result = predict_message(bundle, message)
        prediction_column, probability_column = st.columns(2)

        with prediction_column:
            st.metric("Prediction", result["label"].upper())
        with probability_column:
            st.metric("Spam probability", f'{result["spam_probability"]:.1%}')

        if result["label"] == "spam":
            st.warning("The model found spam-like language. Treat links and requests with caution.")
        else:
            st.success("The model found more ham-like language. This is not a security guarantee.")

How does the result flow work?

  • The blank-message check prevents an empty prediction request.
  • The result variable holds the predicted label with its spam probability.
  • The two columns place the prediction beside the probability.
  • The final condition selects risk feedback that matches the predicted class.

Before you test it, which message do you expect to receive the higher spam probability?

  • Save app.py.
  • Return to the running dashboard in your browser.
  • Replace the text area value with Can we meet for lunch tomorrow?.
  • Click Analyze message.
  • Replace the text area value with URGENT! You have won a free prize. Reply now to claim it..
  • Click Analyze message again.

The conversational message should show HAM with a success message. The prize message should show SPAM with a warning.

Both results should include a percentage in the Spam probability card. Your model now produces feedback that someone can operate from the browser.

Prediction cards not appearing?

Confirm the analysis block begins at the left edge of app.py. Check that every nested line keeps the indentation shown above.

Verify that the call uses predict_message(bundle, message) with the same identifiers defined earlier.

Help me debug why my Streamlit prediction cards or risk message do not appear.

✔️ Awesome, I've got everything!

Your app.py file matches the completed model dashboard for this step.

ⓧ I'd like to double check the full code

import streamlit as st

from model import explain_message, predict_message, train_model


st.set_page_config(page_title="PhishScope AI", layout="wide")


@st.cache_resource
def get_model_bundle():
    return train_model()


bundle = get_model_bundle()

st.title("PhishScope AI")
st.caption(
    "An explainable SMS spam detector trained locally on the UCI SMS Spam Collection."
)

message = st.text_area(
    "Message to analyze",
    value="URGENT! You have won a free prize. Reply now to claim it.",
    height=150,
)

if st.button("Analyze message", type="primary", width="stretch"):
    if not message.strip():
        st.warning("Enter a message before running the detector.")
    else:
        result = predict_message(bundle, message)
        prediction_column, probability_column = st.columns(2)

        with prediction_column:
            st.metric("Prediction", result["label"].upper())
        with probability_column:
            st.metric("Spam probability", f'{result["spam_probability"]:.1%}')

        if result["label"] == "spam":
            st.warning("The model found spam-like language. Treat links and requests with caution.")
        else:
            st.success("The model found more ham-like language. This is not a security guarantee.")

Your browser dashboard now turns model output into a risk assessment. Next, you will expose its learned signals with held-out performance evidence.

Explain and Document the Project

Your Streamlit dashboard already classifies a message. It also estimates the probability that the message is spam.

A probability alone is a black-box claim. Matched TF-IDF feature weights reveal the language behind a result. Held-out metrics from logistic regression show where the model succeeds or makes mistakes.

In this step, get ready to:
  • Display the matched model signals behind each prediction.
  • Present held-out metrics with a confusion matrix.
  • Document the architecture, local run process, and model limitations.
Show matched model signals

The existing explain_message() function finds message features that occur in the trained vocabulary. Their learned weights show which matched phrases influence the prediction.

  • Return to the app.py tab from earlier.
  • Locate the final st.success() line inside the button block.
  • Place the following explanation panel directly below that line:
        st.subheader("Matched model signals")
        signals = explain_message(bundle, message)
        if signals:
            st.dataframe(signals, hide_index=True)
        else:
            st.caption("No message terms survived the trained vocabulary filter, so there are no matched weights to display.")

What does this code do?

  • The signals variable stores the strongest trained features found in the submitted message.
  • The st.dataframe() call presents each feature with its risk direction and absolute weight.
  • The fallback caption explains why an unfamiliar message can produce an empty table.
  • Save app.py.

The first dashboard load can pause briefly while the cached model bundle is prepared. Keep the integrated terminal active while the browser uses the app.

Before you run this, picture the evidence you expect to appear beneath a prediction.

  • Start the dashboard from the integrated terminal by running this command:
python -m streamlit run app.py

What does this command do?

This starts app.py as a local dashboard. The running process keeps the browser connected to your Python code.

  • Keep the default message in the Message to analyze field.
  • Click Analyze message.

You should see a Matched model signals table beneath the prediction. Its rows show matched features with their direction and absolute weight.

That first explanation layer is working. Your dashboard can now show evidence from the model alongside its label.

Do not see the signals table?

Check that the new block uses the same indentation as the existing warning and success messages. Code placed outside the button branch cannot access result or the submitted message at the right time.

Confirm that you saved app.py before reloading the browser.

Help me fix the matched-signals panel in my Streamlit app.

Add held-out performance

An individual explanation shows why one message received its result. Held-out evaluation shows how the trained pipeline behaves across messages that were excluded from training.

  • Return to the bottom of app.py.
  • Place the following performance panel after the complete Analyze message button block:
st.subheader("Held-out test performance")
metric_columns = st.columns(5)
metric_values = (("Dataset messages", str(bundle["dataset_size"])), ("Accuracy", f'{bundle["accuracy"]:.1%}'), ("Spam precision", f'{bundle["spam_precision"]:.1%}'), ("Spam recall", f'{bundle["spam_recall"]:.1%}'), ("Spam F1", f'{bundle["spam_f1"]:.1%}'))
for column, (label, value) in zip(metric_columns, metric_values):
    with column:
        st.metric(label, value)
st.subheader("Confusion matrix")
st.dataframe(bundle["confusion"], hide_index=True)
st.caption("Educational model only. Do not use this prediction as the sole basis for a real security decision.")

What does this code do?

  • The five metric cards report the dataset size, accuracy, spam precision, spam recall, and spam F1 score.
  • The confusion table separates actual ham from actual spam. Its predicted columns expose false positives and false negatives.
  • The closing caption sets a clear boundary around this educational model.
  • Save app.py.
  • Refresh the running dashboard in your browser.

You should see five metric cards above a two-row confusion table. The rows represent the actual classes from the held-out test set.

Your evaluation is now visible beside the interactive detector. A reviewer can inspect the model through both individual evidence and held-out results.

How to read the evaluation

Spam precision measures how often predicted spam is actually spam. Spam recall measures how much of the real spam the model catches.

The confusion table turns those trade-offs into counts. A ham message in the predicted-spam column is a false positive.

Are the metrics missing?

Check that st.subheader("Held-out test performance") starts at the left edge of the file. Extra indentation can hide the evaluation inside the button block.

Confirm that each key in metric_values matches the keys returned by train_model().

Help me debug the held-out metrics in my Streamlit dashboard.

✔️ Awesome, I've got everything!

Your app.py file now combines predictions, matched signals, held-out metrics, and the confusion table.

The running dashboard gives you a complete visual checkpoint for the application code.

ⓧ I'd like to double check the full code

This reference shows the complete app.py file at this point in the project.

import streamlit as st

from model import explain_message, predict_message, train_model


st.set_page_config(page_title="PhishScope AI", layout="wide")


@st.cache_resource
def get_model_bundle():
    return train_model()


bundle = get_model_bundle()

st.title("PhishScope AI")
st.caption("An explainable SMS spam detector trained locally on the UCI SMS Spam Collection.")

message = st.text_area("Message to analyze", value="URGENT! You have won a free prize. Reply now to claim it.", height=150)

if st.button("Analyze message", type="primary", width="stretch"):
    if not message.strip():
        st.warning("Enter a message before running the detector.")
    else:
        result = predict_message(bundle, message)
        prediction_column, probability_column = st.columns(2)
        with prediction_column:
            st.metric("Prediction", result["label"].upper())
        with probability_column:
            st.metric("Spam probability", f'{result["spam_probability"]:.1%}')
        if result["label"] == "spam":
            st.warning("The model found spam-like language. Treat links and requests with caution.")
        else:
            st.success("The model found more ham-like language. This is not a security guarantee.")
        st.subheader("Matched model signals")
        signals = explain_message(bundle, message)
        if signals:
            st.dataframe(signals, hide_index=True)
        else:
            st.caption("No message terms survived the trained vocabulary filter, so there are no matched weights to display.")

st.subheader("Held-out test performance")
metric_columns = st.columns(5)
metric_values = (("Dataset messages", str(bundle["dataset_size"])), ("Accuracy", f'{bundle["accuracy"]:.1%}'), ("Spam precision", f'{bundle["spam_precision"]:.1%}'), ("Spam recall", f'{bundle["spam_recall"]:.1%}'), ("Spam F1", f'{bundle["spam_f1"]:.1%}'))
for column, (label, value) in zip(metric_columns, metric_values):
    with column:
        st.metric(label, value)
st.subheader("Confusion matrix")
st.dataframe(bundle["confusion"], hide_index=True)
st.caption("Educational model only. Do not use this prediction as the sole basis for a real security decision.")

What should this file contain?

This version preserves the cached model and prediction cards from earlier. It adds the explanation panel plus the evaluation panel required for the finished dashboard.

Document the project and verify the dashboard

Project documentation turns a working demo into a reproducible technical artifact. The repository needs to keep generated environment files out of source control while giving another developer enough context to run the detector.

  • Select the phishscope-ai folder in the Visual Studio Code file sidebar from earlier.
  • Use the new-file control to create .gitignore inside phishscope-ai.
  • Fill .gitignore with these ignore rules:
.venv/
__pycache__/
*.pyc

What do these rules protect?

The rules exclude the local virtual environment plus generated Python bytecode. These machine-specific files can be recreated from requirements.txt.

  • Save .gitignore.
  • Confirm that .gitignore appears beside app.py in the file sidebar.

Is the ignore file missing?

Check that the filename starts with a period. Confirm that the editor did not add a second file extension.

Help me check the .gitignore file in my PhishScope AI folder.

A README.md gives the project a clear entry point. Its opening sections explain the product plus the machine-learning evidence it demonstrates.

  • Use the new-file control in the same file sidebar to create README.md.
  • Add the project overview and demonstration list below:
# PhishScope AI

PhishScope AI is a local, explainable SMS spam detector built with Python, scikit-learn, and Streamlit.

## What it demonstrates

- Loading a public labeled dataset from the UCI Machine Learning Repository
- Building unigram and bigram TF-IDF features
- Training balanced logistic regression
- Measuring accuracy, spam precision, recall, F1 score, and confusion counts
- Showing matched feature weights for an individual message
- Discussing false positives, dataset age, and adversarial wording

What does this section communicate?

The opening identifies the finished product. The demonstration list summarizes the data pipeline, evaluation evidence, explainability layer, and security limitations.

  • Save README.md.
  • Confirm that the editor shows the What it demonstrates heading with six bullet points.

Does the Markdown look uneven?

Check that each heading starts with the same number of hash characters shown in the reference. Keep one blank line around each heading.

Help me fix the opening Markdown in my PhishScope AI README.

The rest of the README makes the detector reproducible. It also records the limitations that should shape every demonstration of the model.

  • Place your cursor after the final bullet in README.md.
  • Add the run guide, data source, and limitations below:

## Run locally

1. Create and activate a Python 3.11 to 3.14 virtual environment.
2. Install the pinned packages:

   ```text
   pip install -r requirements.txt
   ```

3. Start the dashboard:

   ```text
   python -m streamlit run app.py
   ```

The first run needs internet access so `ucimlrepo` can retrieve dataset ID `228`.

## Data source

SMS Spam Collection, UCI Machine Learning Repository: https://archive.ics.uci.edu/dataset/228/sms%2Bspam%2Bcollection

## Limitations

This educational model was trained on historical SMS data. It can misclassify messages, its displayed probability is not a guarantee of danger or safety, and attackers can deliberately change wording to evade text features. Do not use it as the sole basis for a real security decision.

Why include these details?

The run guide lets another developer reproduce the dashboard with the pinned dependencies. The data-source section makes the training corpus traceable.

The limitations section covers false classifications, historical data, adversarial wording, and the non-production boundary. Honest limits make the project easier to defend in a technical discussion.

  • Save README.md.
  • Confirm that the file contains the Run locally, Data source, and Limitations headings.

Is a code fence broken?

Check that each command sits between a matching pair of three-backtick fences. Confirm that the indentation beneath each numbered item matches the reference.

Help me repair the command blocks in my PhishScope AI README.

✔️ Awesome, I've got everything!

Your project now includes source-control exclusions plus a reproducible README. The documentation describes what the model demonstrates and where its predictions can fail.

ⓧ I'd like to double check the full code

This reference shows the complete .gitignore file.

.venv/
__pycache__/
*.pyc

What should this file contain?

The complete file has one rule for the virtual environment plus two rules for generated Python bytecode.

This reference shows the complete README.md file.

# PhishScope AI

PhishScope AI is a local, explainable SMS spam detector built with Python, scikit-learn, and Streamlit.

## What it demonstrates

- Loading a public labeled dataset from the UCI Machine Learning Repository
- Building unigram and bigram TF-IDF features
- Training balanced logistic regression
- Measuring accuracy, spam precision, recall, F1 score, and confusion counts
- Showing matched feature weights for an individual message
- Discussing false positives, dataset age, and adversarial wording

## Run locally

1. Create and activate a Python 3.11 to 3.14 virtual environment.
2. Install the pinned packages:

   ```text
   pip install -r requirements.txt
   ```

3. Start the dashboard:

   ```text
   python -m streamlit run app.py
   ```

The first run needs internet access so `ucimlrepo` can retrieve dataset ID `228`.

## Data source

SMS Spam Collection, UCI Machine Learning Repository: https://archive.ics.uci.edu/dataset/228/sms%2Bspam%2Bcollection

## Limitations

This educational model was trained on historical SMS data. It can misclassify messages, its displayed probability is not a guarantee of danger or safety, and attackers can deliberately change wording to evade text features. Do not use it as the sole basis for a real security decision.

What should this file contain?

The complete README covers the architecture, dataset, local commands, and limitations. It gives you the talking points needed for a reproducible portfolio demonstration.

A one-minute demonstration should connect the product to its evidence. Keep each point focused on something visible in the dashboard or documented in the repository.

  • Frame the problem as identifying suspicious language in pasted SMS messages.
  • Name TF-IDF as the feature method used by the model.
  • Identify balanced logistic regression as the classifier.
  • Show the matched signals as evidence for one prediction.
  • Use the confusion table to discuss a false positive.
  • Close with the educational non-production disclaimer.

Before you refresh, picture the complete evidence you expect to see around the prediction.

  • Refresh the running dashboard in your browser.
  • Paste URGENT! You have won a free prize. Reply now to claim it. into the Message to analyze field.
  • Click Analyze message.

You should see a predicted label plus a spam probability. You should also see matched model signals, five held-out metrics, and a two-row confusion table.

That is the finished PhishScope AI dashboard. You can now demonstrate its result, defend its evidence, and explain its limitations.

Secret mission

Harden the Model Against Obfuscated Text

Attackers can hide familiar scam words behind digit swaps or inserted punctuation. You will normalize those patterns before TF-IDF feature extraction. You will then prove that the dashboard recovers matched vocabulary from an obfuscated message.

Clean Up Your Resources

Clean Up Your Resources

Choose whether to keep your local dashboard running, pause it for later, or remove the project entirely. PhishScope AI runs on your computer, so leaving its files in place creates no ongoing service charges.

Resources you used:

  • The local phishscope-ai folder with all project files.
  • The .venv environment with its pinned Python packages inside phishscope-ai.
  • The local Streamlit server process that serves the browser dashboard.

Keep everything running

No action needed. Choose this if you are still testing the hardened detector or refining its explanations.

  • Leave the Streamlit server running while you continue testing the dashboard.
  • Keep the phishscope-ai folder on your computer so every project file remains available.
  • Use the existing .venv environment when you continue working with the pinned packages.

Pause - I'll come back to this later

Stop the local server to free its port. Your project files remain ready for your next session.

  • Switch back to the integrated terminal in Visual Studio Code that is serving the dashboard.
  • Press Ctrl+C to stop the Streamlit server.
  • Close the PhishScope AI browser tab.
  • Close Visual Studio Code when you are finished.

Your phishscope-ai folder remains intact. Activate .venv before restarting the dashboard when you return.

Delete - I don't want to use this again

This option removes the local server process. It also removes the complete project folder.

Save What You Need

Permanent deletion removes every artifact inside phishscope-ai.

  • Copy any code or documentation you want to keep to another folder before continuing.
  • Switch back to the integrated terminal that is serving the dashboard.
  • Press Ctrl+C to stop the Streamlit server.
  • Right-click the phishscope-ai folder at the top of the Visual Studio Code Explorer sidebar.
  • Use the folder context menu to reveal its location in Windows File Explorer.
  • Close Visual Studio Code.
  • Select the phishscope-ai folder in Windows File Explorer.
  • Press Shift+Delete to permanently delete the folder.
  • Confirm the permanent deletion when Windows asks.

Cleanup complete. You should no longer see phishscope-ai in its parent folder.

Nice Work!

Nice Work!

You did it! You built PhishScope AI, an explainable SMS spam detector that turns pasted messages into predictions you can inspect.

You've learned how to:

  • Build an interactive Streamlit dashboard for pasted messages. See the predicted class in the browser. Check the spam probability beside it.
  • Train a reproducible classifier with TF-IDF features powered by balanced logistic regression. Measure held-out performance with accuracy, spam precision, recall, F1 score, plus a confusion matrix.
  • Use matched model signals to inspect the learned weights behind each prediction. Demonstrate why exact keyword rules fail on unseen wording. Document false positives plus model limitations in README.md.
  • Secret Mission: Harden the classifier against common leetspeak through input normalization. Strip inserted punctuation before TF-IDF analysis. Demonstrate recovered scam vocabulary in obfuscated input.

Ready to quiz yourself?