Build an AI Resume Detector

Train an NLP classifier to detect AI-generated resumes with a Streamlit app.

Introduction

30 Second Summary

AI can now write a resume in seconds that reads better than most human-written ones. Recruiters are drowning in polished applications and have no way to tell which ones are genuine.

In this project, you will build an AI Resume Detector that analyzes any pasted resume and tells you whether a human or an AI wrote it. You will train your own text classifier, wrap it in a web app, and see exactly which words gave the AI away.

What You'll Build

You paste a resume into your app, hit Analyze, and within a second see a confidence score plus a ranked breakdown of the telltale words (like "leveraged", "spearheaded", "innovative") that flagged it as AI-generated.

By the end of this project, you'll have:

  • A working TF-IDF + Logistic Regression classifier that you trained on your own laptop CPU in under 2 seconds, scoring any resume as "Likely Human" or "Likely AI-Generated" with a confidence percentage.
  • A Streamlit web app where you paste any resume and instantly see a verdict plus the top flagged words that pushed the classification, ranked by influence.
  • A Generate Test Resume button that calls Claude to produce a fresh AI resume on the spot, so you can watch your detector react in real time.
  • Secret Mission: Add stylometric features like sentence length variance, buzzword density, and type-token ratio to catch structural AI fingerprints that word frequency alone misses.

Are there any prerequisites?

You need basic Python familiarity (variables, functions, running scripts). No prior NLP or machine learning experience is required.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters.

Set Up Your Project Environment

Your AI resume detector needs three core ingredients: a machine learning library to build the classifier, a web framework to create the interface, and an AI SDK to generate training data.

In this step, you will create an isolated Python environment and install all three packages. You will also configure your API key for Claude, the AI model that will generate fake resumes for your training dataset.

In this step, get ready to:
  • Create a project folder with an isolated Python environment.
  • Install the machine learning, web app, and AI packages.
  • Configure your Claude API key for generating training data.
Create a project folder and virtual environment

A virtual environment keeps your project's packages separate from the rest of your system. This prevents version conflicts between different Python projects on your machine.

  • Open Cursor by pressing Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Cursor and press Enter to open it.
  • Open the integrated terminal by pressing Ctrl+` (backtick key, below Escape).
  • Check your Python version by running:
python3 --version

✔️ I see version 3.11 or higher

You're good to go. Continue to the next instruction below.

ⓧ I see an older version

This project requires Python 3.11+ because scikit-learn 1.9 depends on it. On macOS, the system python3 is often an older version even after you install a newer one.

  • Install Python 3.11+ from python.org/downloads.
  • After installing, run python3.11 --version or python3.13 --version to confirm the new version is available.
  • Use that specific binary (e.g. python3.13) in place of python3 for the rest of this project.

ⓧ Command not found

Python is not installed on your machine yet.

  • Download and install Python 3.11+ from python.org/downloads.
  • On Windows, check the box that says Add Python to PATH during installation.
  • Close and reopen your terminal after installing, then run python3 --version again.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir ai-resume-detector
cd ai-resume-detector
  • Create a virtual environment and activate it by running:

macOS

python3 -m venv .venv
source .venv/bin/activate

Windows

python -m venv .venv
.venv\Scripts\activate

You should see (.venv) appear at the beginning of your terminal prompt. This confirms you are inside the virtual environment.

Don't see (.venv) in your prompt?

Make sure you ran the activate command (not just the venv creation command). On Windows, if you see an execution policy error, run Set-ExecutionPolicy -Scope CurrentUser RemoteSigned in PowerShell first, then try activation again.

help me activate my Python virtual environment

  • Install the three packages your project needs by running:
pip install scikit-learn streamlit anthropic

What are these packages?

  • scikit-learn (scikit-learn) provides machine learning tools including TF-IDF vectorization and logistic regression.
  • streamlit (Streamlit) turns Python scripts into interactive web apps with a single command.
  • anthropic is the official Python SDK for the Claude API.

You should see pip downloading and installing packages, ending with a success message.

Seeing install errors?

If pip cannot find packages, confirm your virtual environment is active (you should see (.venv) in your prompt). If you see permission errors, make sure you activated the environment before running pip install.

help me fix pip install errors in my virtual environment

Set up your Claude API key

The Anthropic Python SDK reads an environment variable called ANTHROPIC_API_KEY automatically when you create a client. You need to create a key and set this variable so your script can call the Claude API.

  • Open your browser and go to console.anthropic.com.
  • Sign up for a new account or log in if you already have one.
  • Navigate to the API Keys section in the sidebar.
  • Click Create Key to generate a new API key.
  • Copy the key immediately. You will not be able to see it again after closing the dialog.

How much does Claude cost?

This project uses claude-haiku-4-5, priced at $1 per million input tokens and $5 per million output tokens. Generating 40 resume bullet points costs well under $0.01, easily covered by free trial credits.

  • Back in Cursor's terminal, set the environment variable by running:

macOS

export ANTHROPIC_API_KEY=your-api-key-here

Replace your-api-key-here with the key you just copied.

Windows

In Command Prompt (CMD):

set ANTHROPIC_API_KEY=your-api-key-here

Or in PowerShell:

$env:ANTHROPIC_API_KEY="your-api-key-here"

Replace your-api-key-here with the key you just copied.

Keep your key safe

Never paste your API key directly into code files or share it publicly. Setting it as an environment variable keeps it out of your source code.

Verify everything works

Before moving on, confirm that all three packages import correctly and that your API key is accessible.

  • Check that scikit-learn installed correctly by running:
python -c "import sklearn; print(sklearn.__version__)"

You should see a version number like 1.9.0 printed.

  • Check that anthropic installed correctly by running:
python -c "import anthropic; print(anthropic.__version__)"

You should see a version number printed.

  • Before you run the final check, what do you think streamlit --version will print if everything is set up correctly?
  • Verify Streamlit by running:
streamlit --version

You should see Streamlit, version 1.60.0 printed to the terminal.

Seeing 'command not found' for streamlit?

This usually means your virtual environment is not active. Check that (.venv) appears in your terminal prompt. If not, rerun the activation command from the earlier substep.

help me fix streamlit command not found in my virtual environment

Your environment is locked and loaded. Next up, you will generate a training dataset by writing real resume samples and having Claude produce AI-generated ones.

Generate Your Training Dataset

Your environment is ready to go. Now you need data to teach your classifier what AI-generated text actually looks like.

A classifier learns by example. You will hardcode real human resume bullet points (messy, specific, full of personality) and then ask Claude to mass-produce polished AI-generated ones. This creates the exact real-vs-fake split a recruiter faces every day.

In this step, get ready to:
  • Write 40 human-style resume bullet points as your baseline training data.
  • Call the Claude API to generate 40 AI-written resume bullet points.
  • Save both sets into a labeled JSON dataset ready for the classifier.
Create the human samples dataset

Human-written resumes have a distinctive style. They mention specific people, use informal language, include rough numbers, and reference real tools by name. Your classifier needs to see these patterns to recognize them later.

  • Create a new file called generate_dataset.py in your ai-resume-detector folder.
  • Add the imports and the first 20 human resume samples by pasting this code:
import anthropic
import json
import os

# Human-written resume bullet points - messy, specific, personal
human_samples = [
    "Managed daily operations for a team of 3 at our downtown retail location, handling scheduling conflicts and customer complaints",
    "Helped redesign the company website which led to about 15% more visitors over 2 months",
    "Worked with the marketing dept to plan our annual conference, ended up saving roughly $4k on venue costs",
    "Trained 5 new hires on our POS system and inventory management procedures",
    "Created weekly sales reports in Excel for my manager, started adding charts which she really liked",
    "Fixed a recurring billing error that had been annoying customers for months - turned out to be a timezone issue in our database",
    "Ran social media accounts (mainly Instagram and Twitter) posting 3-4 times per week",
    "Answered 40+ customer support tickets daily via Zendesk, maintained 94% satisfaction rating",
    "Organized the office move to our new building on 5th street, coordinated with movers and IT",
    "Built a simple Python script to automate our monthly report generation, saved about 3 hours per week",
    "Assisted the VP of Sales with preparing pitch decks and client proposals",
    "Handled accounts payable for 50+ vendors, processed invoices within 48 hours usually",
    "Volunteered to lead the diversity committee after nobody else stepped up, organized 4 events",
    "Debugged a memory leak in our Node.js app that had been causing crashes every few days",
    "Wrote documentation for our internal API that other teams actually started using",
    "Managed our Google Ads campaigns with a monthly budget around $12k, kept CPA under $25",
    "Set up automated email sequences in Mailchimp for new customer onboarding",
    "Coordinated with 3 overseas contractors for our mobile app development project",
    "Reduced average response time from 24hrs to about 6hrs by reorganizing the support queue",
    "Presented quarterly results to the leadership team (usually about 12 people in the room)",

What do these samples look like?

Notice how human resumes mention specific people ("my manager"), use casual language ("which she really liked"), give rough numbers ("about 15%"), and reference named tools ("Zendesk", "Mailchimp"). These fingerprints are what your classifier will learn to recognize.

  • Directly below the last line, add the remaining 20 human samples and close the list:
    "Took over the blog when our content person left, wrote 2 posts per month for 6 months",
    "Migrated our data from the old CRM to Salesforce, which took about 3 weeks total",
    "Helped interview candidates for two junior developer positions last spring",
    "Created an employee onboarding checklist that HR still uses today",
    "Negotiated a 10% discount with our main supplier after building relationship over 2 years",
    "Stayed late multiple times to fix production issues during our busiest season",
    "Proposed and implemented a new filing system for the legal department's contracts",
    "Managed inventory for 200+ SKUs across two warehouse locations",
    "Developed training materials for our new CRM workflow, recorded 5 tutorial videos",
    "Caught a $15k accounting discrepancy during month-end close that would have gone unnoticed",
    "Served as point of contact for our biggest client ($500k/year account) for 18 months",
    "Streamlined the hiring paperwork process by creating fillable PDF forms",
    "Set up weekly standup meetings for our remote team of 8 across 3 time zones",
    "Built dashboards in Tableau for the sales team to track their pipeline",
    "Processed payroll for 45 employees biweekly using ADP, never missed a deadline",
    "Handled returns and exchanges at the front desk, averaging 20-30 per day during holidays",
    "Wrote grant applications for our nonprofit, secured $75k in funding over 2 years",
    "Managed the transition from Slack to Microsoft Teams for a 60-person company",
    "Updated our employee handbook to reflect new remote work policies post-COVID",
    "Led a small team that built an internal tool for tracking customer feedback in Google Sheets",
]

Why 40 samples?

40 per class is intentionally small for learning purposes. A production detector would need thousands. The small size means your accuracy will vary between runs (typically 75-90%), which itself teaches you about data quantity and model reliability.

  • Save generate_dataset.py.
  • Confirm the human samples loaded correctly by running this command:
python -c "from generate_dataset import human_samples; print(len(human_samples), 'human samples loaded')"

You should see 40 human samples loaded printed in your terminal.

Seeing a SyntaxError?

Make sure the last item in the first code block ends with a comma and the second block's closing ] sits on its own line. A missing comma between entries is the most common cause.

Also check that your virtual environment is still active. You should see (.venv) at the start of your terminal prompt.

help me fix a SyntaxError in my generate_dataset.py file

Add Claude API calls to generate AI samples

Now for the other side of the coin. AI-generated resumes sound polished and professional, but they follow predictable patterns. You will use Claude to generate 40 bullet points that represent the "too perfect" style your classifier needs to learn.

  • Below the closing ] of human_samples, add the generate_ai_samples function:
# Generate AI resume bullet points using Claude
def generate_ai_samples(num_samples=40):
    client = anthropic.Anthropic()
    
    ai_samples = []
    
    prompts = [
        "Write 10 professional resume bullet points for a marketing manager. Each should be one sentence, action-verb focused, with metrics.",
        "Write 10 professional resume bullet points for a software engineer. Each should be one sentence, action-verb focused, with metrics.",
        "Write 10 professional resume bullet points for a project manager. Each should be one sentence, action-verb focused, with metrics.",
        "Write 10 professional resume bullet points for a data analyst. Each should be one sentence, action-verb focused, with metrics.",
    ]
    
    for prompt in prompts:
        message = client.messages.create(
            model="claude-haiku-4-5",
            max_tokens=1024,
            messages=[
                {
                    "role": "user",
                    "content": prompt + " Return ONLY the bullet points, one per line, no numbering or extra text.",
                }
            ],
        )
        
        response_text = message.content[0].text
        bullets = [line.strip().lstrip("\u2022-* ") for line in response_text.strip().split("\n") if line.strip()]
        ai_samples.extend(bullets[:10])
    
    return ai_samples[:num_samples]

What does this function do?

The function sends 4 prompts to Claude, each requesting 10 resume bullet points for a different role (marketing manager, software engineer, project manager, data analyst). This gives variety across job types while keeping the AI writing style consistent.

The anthropic.Anthropic() client automatically reads your ANTHROPIC_API_KEY environment variable. Each response is split into individual bullet points and cleaned of formatting characters like bullet dots or dashes.

Save the dataset to a JSON file

The final piece combines both sample sets, labels them (0 for human, 1 for AI), and writes everything to a JSON file your classifier can load directly.

  • Below the generate_ai_samples function, add the main function and entry point:
def main():
    print("Generating AI resume samples using Claude...")
    ai_samples = generate_ai_samples(40)
    
    # Build the dataset
    dataset = []
    for sample in human_samples:
        dataset.append({"text": sample, "label": 0})  # 0 = human
    for sample in ai_samples:
        dataset.append({"text": sample, "label": 1})  # 1 = AI
    
    # Save to JSON
    with open("resume_dataset.json", "w") as f:
        json.dump(dataset, f, indent=2)
    
    human_count = sum(1 for d in dataset if d["label"] == 0)
    ai_count = sum(1 for d in dataset if d["label"] == 1)
    print(f"Dataset created: {human_count} human samples, {ai_count} AI samples")
    print(f"Saved to resume_dataset.json")


if __name__ == "__main__":
    main()

What does this code do?

The main function calls generate_ai_samples(40) to get Claude's output, then builds a list of dictionaries where each entry has a text field (the bullet point) and a label field (0 for human, 1 for AI).

The if __name__ == "__main__" guard ensures main() only runs when you execute the script directly, not when you import from it.

  • Save generate_dataset.py.

Before you run this, what do you think will happen? The script calls the Claude API 4 times to generate resume bullets. Think about what the output will look like and how long it might take.

  • Run the dataset generator:
python generate_dataset.py

You should see output like this (it takes a few seconds while Claude generates the AI samples):

Generating AI resume samples using Claude...
Dataset created: 40 human samples, 40 AI samples
Saved to resume_dataset.json
  • Confirm the file was created by running:
ls

You should see resume_dataset.json alongside generate_dataset.py in the output.

Seeing an AuthenticationError?

This means your ANTHROPIC_API_KEY environment variable is not set or is incorrect. Re-run the export command from Step 1 in the same terminal session you are using now.

If you opened a new terminal window since setting the key, you need to set it again in this window.

help me fix an AuthenticationError when calling the Anthropic API

✔️ Awesome, I've got everything!

Great. Double check you have saved generate_dataset.py and that resume_dataset.json exists in your folder.

ⓧ I'd like to double check the full code

Your generate_dataset.py file should contain these sections in order. Scroll up and confirm each is present:

  • Three imports: anthropic, json, os.
  • The human_samples list with exactly 40 string entries.
  • The generate_ai_samples function with 4 prompts and the API call loop.
  • The main function that builds the dataset and saves to JSON.
  • The if __name__ == "__main__" entry point at the bottom.

Your dataset is ready. You have 80 labeled examples that capture the real difference between how humans and AI write resumes. Next up, you will feed this data into a TF-IDF classifier that learns to spot those differences automatically.

Train the TF-IDF Classifier

You have 80 labeled resume samples sitting in resume_dataset.json. Now you need to teach a machine to spot the difference between human and AI writing.

TF-IDF converts raw text into numbers by scoring how important each word is to a document relative to all documents. Logistic Regression then finds the decision boundary between AI text and human text. You will see accuracy metrics printed immediately after training.

In this step, get ready to:
  • Load the dataset and split it into training and test sets.
  • Build a TF-IDF + Logistic Regression pipeline using scikit-learn.
  • Evaluate accuracy and save the trained model.
Load the dataset and split into train/test

Before a model can learn anything, it needs data split into two groups. The training set teaches the model, and the test set checks whether it actually learned or just memorized.

  • Create a new file called train_model.py in your ai-resume-detector folder.
  • Add the imports, data loading, and train/test split by pasting this code:
import json
import joblib
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from sklearn.pipeline import Pipeline


def main():
    # Load dataset
    with open("resume_dataset.json", "r") as f:
        dataset = json.load(f)

    texts = [item["text"] for item in dataset]
    labels = [item["label"] for item in dataset]

    # Split into train and test sets
    X_train, X_test, y_train, y_test = train_test_split(
        texts, labels, test_size=0.2, random_state=42, stratify=labels
    )

What does this code do?

  • The imports pull in everything from scikit-learn that you need: the vectorizer, classifier, splitting utility, and evaluation metrics.
  • train_test_split holds back 20% of the data (16 samples) for testing. stratify=labels ensures the split keeps the same human/AI ratio in both sets.
  • random_state=42 makes the split reproducible so your results will be consistent across runs.
  • Save train_model.py.
Build the TF-IDF + Logistic Regression pipeline

A scikit-learn Pipeline chains the vectorizer and classifier into a single object. You fit it once, and it handles the text-to-numbers conversion and classification in one call.

  • In train_model.py, add the following code directly below the train_test_split call (still inside the main() function):
    # Build TF-IDF + Logistic Regression pipeline
    pipeline = Pipeline([
        ("tfidf", TfidfVectorizer(
            ngram_range=(1, 2),
            max_features=5000,
            stop_words="english",
        )),
        ("classifier", LogisticRegression(max_iter=1000, random_state=42)),
    ])

    # Train the model
    pipeline.fit(X_train, y_train)

    # Evaluate on test set
    y_pred = pipeline.predict(X_test)
    print("Classification Report:")
    print("=" * 50)
    print(classification_report(y_test, y_pred, target_names=["Human", "AI-Generated"]))

    # Save the trained pipeline
    joblib.dump(pipeline, "model.joblib")
    print("Model saved to model.joblib")


if __name__ == "__main__":
    main()

What does this code do?

  • TfidfVectorizer with ngram_range=(1, 2) captures single words AND two-word phrases. This lets the model detect patterns like "leveraged innovative" that AI resumes favor.
  • max_features=5000 limits the vocabulary to the 5000 most informative terms, preventing noise from rare words.
  • pipeline.fit(X_train, y_train) trains both the vectorizer and classifier in one step.
  • classification_report prints precision, recall, and F1-score for each class so you can see exactly where the model succeeds or struggles.
  • joblib.dump saves the entire trained pipeline to model.joblib so your Streamlit app can load it later without retraining.
  • Save train_model.py.

Seeing an indentation error?

All the code from the pipeline onward must be indented inside the main() function (4 spaces). Make sure the if __name__ block at the bottom is NOT indented (it sits at the top level of the file).

help me fix an indentation error in my train_model.py file

Evaluate and save the model

The script is ready. When you run it, scikit-learn will train on 64 samples and test on 16, printing a full classification report.

Before you run this, what accuracy do you expect from a model trained on only 80 samples?

  • Train the model and see the results by running:
python train_model.py

You should see a classification report with precision, recall, and F1-score for both "Human" and "AI-Generated" classes. Expect overall accuracy somewhere between 75-90% on this small dataset.

What do these metrics mean?

  • Precision answers: of all the resumes the model flagged as AI, what fraction actually were AI?
  • Recall answers: of all the actual AI resumes, what fraction did the model catch?
  • F1-score is the balance between precision and recall. Higher is better.

With only 80 training samples, accuracy will vary. A real production system would use thousands of examples. The small dataset here is intentional so training completes in under 2 seconds on your CPU.

  • Confirm the model file was saved by running:
ls

You should see model.joblib listed alongside train_model.py, generate_dataset.py, and resume_dataset.json.

Seeing a FileNotFoundError?

This means resume_dataset.json is not in the same folder as train_model.py. Make sure you ran generate_dataset.py successfully in the previous step and that your terminal is in the ai-resume-detector folder.

help me fix a FileNotFoundError in train_model.py

✔️ Awesome, I've got everything!

Great. Double check you have saved train_model.py and that model.joblib appeared in your folder after running.

ⓧ I'd like to double check the full code

Here is the complete train_model.py file. Scroll up and confirm your code has both the load/split section and the pipeline/evaluate section, in this order:

  • Imports at the top (json, joblib, sklearn modules).
  • main() function with dataset loading and train_test_split.
  • Pipeline construction, fit, predict, classification_report, and joblib.dump.
  • if __name__ == "__main__": main() at the bottom, not indented.

Your classifier can now tell the difference between human and AI writing. Next up, you will wrap it in a web interface so anyone can paste a resume and get an instant verdict.

Build the Streamlit Web App

Your model can classify resume text, but right now it only outputs a 0 or 1 in the terminal. A recruiter needs to paste a resume and see a clear verdict instantly.

In this step, you will wrap your trained classifier in a Streamlit web app. The app gives users a text area, an Analyze button, and a color-coded result with a confidence percentage.

In this step, get ready to:
  • Create a Streamlit interface with a text input area and action buttons.
  • Add a "Generate AI Resume" button that calls the Claude API to produce test data on the fly.
  • Wire up the Analyze button to run predictions and display a styled verdict.
Create the app interface

Streamlit apps are single Python files that render UI elements in the order you write them. You will start by setting up the page, loading your trained model, and rendering the input area.

  • Create a new file called app.py in your ai-resume-detector folder and add the following code:
import streamlit as st
import joblib
import numpy as np
import anthropic
import os

# Page configuration
st.set_page_config(page_title="AI Resume Detector", page_icon="\ud83d\udd0d", layout="centered")

# Load the trained model
@st.cache_resource
def load_model():
    return joblib.load("model.joblib")

pipeline = load_model()

# App header
st.title("AI Resume Detector")
st.markdown("Paste a resume below to check if it was likely written by a human or generated by AI.")

# Text input area
resume_text = st.text_area(
    "Paste resume text here:",
    height=200,
    placeholder="Paste resume bullet points or full resume text here...",
)

What does this code do?

  • st.set_page_config sets the browser tab title and layout before any other Streamlit calls.
  • @st.cache_resource ensures model.joblib loads only once, even when the app re-runs on each interaction.
  • st.text_area renders a large input box where the user will paste resume text.
  • Save app.py.
  • Launch the app by running this command in your terminal:
streamlit run app.py

Your browser opens automatically to localhost:8501. You should see the title "AI Resume Detector" and an empty text area.

Don't see the app in your browser?

  • Make sure your virtual environment is still activated (you should see (.venv) in your terminal prompt).
  • Check that model.joblib exists in the same folder as app.py. If not, re-run python train_model.py first.
  • If the browser did not open, navigate to http://localhost:8501 manually.

help me debug why my Streamlit app won't start

Add buttons and the generate feature

The app needs two buttons: one to analyze pasted text, and one to generate a fresh AI resume on the spot. The generate button demonstrates the problem your detector solves: AI can produce unlimited convincing resumes instantly.

  • In app.py, add the following code below the resume_text = st.text_area(...) block:
# Generate test resume button
col1, col2 = st.columns(2)

with col1:
    analyze_button = st.button("Analyze Resume", type="primary")

with col2:
    generate_button = st.button("Generate AI Resume (Test)")

# Handle generate button
if generate_button:
    api_key = os.environ.get("ANTHROPIC_API_KEY")
    if not api_key:
        st.error("ANTHROPIC_API_KEY environment variable not set. Please set it to generate test resumes.")
    else:
        with st.spinner("Generating AI resume..."):
            client = anthropic.Anthropic()
            message = client.messages.create(
                model="claude-haiku-4-5",
                max_tokens=512,
                messages=[
                    {
                        "role": "user",
                        "content": "Write 5 professional resume bullet points for a senior software engineer. Each should be one sentence, action-verb focused, with impressive metrics. Return ONLY the bullet points, one per line.",
                    }
                ],
            )
            generated_text = message.content[0].text
        st.session_state["generated_resume"] = generated_text
        st.rerun()

What does this code do?

  • st.columns(2) places two buttons side by side in a clean layout.
  • The generate handler calls Claude's claude-haiku-4-5 model to produce 5 fake resume bullet points on demand.
  • st.session_state stores the generated text across Streamlit reruns, and st.rerun() triggers a fresh render cycle that picks it up.
  • Now add the auto-fill logic directly below the generate handler:
# Auto-fill if we just generated a resume
if "generated_resume" in st.session_state:
    resume_text = st.session_state.pop("generated_resume")
    st.text_area(
        "Paste resume text here:",
        value=resume_text,
        height=200,
        key="filled_area",
    )

Why a separate text area?

Streamlit re-runs the entire script on every interaction. This block checks if a generated resume is waiting in session state, pops it out, and renders a pre-filled text area so the user sees their AI resume immediately.

  • Save app.py. The running Streamlit app auto-refreshes in your browser.
  • Click the Generate AI Resume (Test) button.
  • You should see a spinner, then 5 AI-generated bullet points fill the text area.

Generate button shows an error?

  • If you see "ANTHROPIC_API_KEY environment variable not set", make sure you set the key in the same terminal session that is running Streamlit. Stop the app with Ctrl+C, set the key again, then re-run streamlit run app.py.
  • If you see an authentication error from the API, double-check your key at console.anthropic.com.

help me fix my Claude API key for Streamlit

Add the resume analysis logic

The generate button works, but clicking Analyze does nothing yet. This final piece runs the TF-IDF pipeline on the pasted text and displays a color-coded verdict with a confidence bar.

  • In app.py, add the following code at the very end of the file (below the auto-fill block):
# Handle analyze button
if analyze_button and resume_text.strip():
    # Get prediction probability
    proba = pipeline.predict_proba([resume_text])[0]
    ai_probability = proba[1]
    human_probability = proba[0]

    # Display verdict
    st.markdown("---")

    if ai_probability > 0.5:
        st.error(f"**Verdict: Likely AI-Generated** ({ai_probability:.0%} confidence)")
    else:
        st.success(f"**Verdict: Likely Human-Written** ({human_probability:.0%} confidence)")

    # Show probability bar
    st.metric("AI Probability", f"{ai_probability:.1%}")
    st.progress(ai_probability)

elif analyze_button:
    st.warning("Please paste some resume text to analyze.")

What does this code do?

  • predict_proba() returns a probability for each class (human at index 0, AI at index 1) instead of just a hard 0/1.
  • st.error() renders a red box for AI-likely resumes. st.success() renders a green box for human-likely ones.
  • st.metric shows the AI probability as a headline number. st.progress renders it as a visual bar so the confidence is immediately scannable.
  • Save app.py.

Before you test, what do you think will happen if you paste one of the human-written bullet points from your dataset? Will the model confidently say "Human" or will it struggle?

  • Copy one of the human samples from generate_dataset.py (for example: "Fixed a recurring billing error that had been annoying customers for months - turned out to be a timezone issue in our database").
  • Paste it into the text area in your browser and click Analyze Resume.

You should see a green "Likely Human-Written" verdict with a confidence percentage and a progress bar showing the AI probability.

  • Now click Generate AI Resume (Test) to get a fresh AI resume, then click Analyze Resume.
  • This time you should see a red "Likely AI-Generated" verdict with a higher AI probability.

Clicking Analyze does nothing?

  • Make sure the if analyze_button block is at the very bottom of your file, after the auto-fill code.
  • Check that you have resume_text.strip() in the condition. If the text area is empty, Streamlit shows the warning instead.
  • Confirm app.py is saved. Streamlit watches the file for changes but only picks them up on save.

help me debug why Analyze Resume isn't showing results

✔️ Awesome, I've got everything!

Great. Double check you have saved app.py and the app is running at localhost:8501.

ⓧ I'd like to double check the full code

Your app.py should contain these sections in order. Scroll up and confirm each block is present:

  • Imports (streamlit, joblib, numpy, anthropic, os).
  • Page configuration with st.set_page_config(...).
  • Model loading with @st.cache_resource and load_model().
  • App header (st.title and st.markdown).
  • Text input area (st.text_area).
  • Two-column button layout (st.columns(2)) with Analyze and Generate buttons.
  • Generate button handler (if generate_button) with Claude API call and st.rerun().
  • Auto-fill block (if "generated_resume" in st.session_state).
  • Analyze handler (if analyze_button and resume_text.strip()) with verdict, metric, and progress bar.
  • The elif analyze_button fallback showing a warning.

Your classifier now has a proper interface. Anyone can paste a resume and get an instant AI probability score. Next up, you will add word-level explainability so users can see exactly which words flagged the resume.

Add Word-Level Explainability

Your app gives a confidence score, but a recruiter reading "78% AI-Generated" has no idea why. They need to see which specific words triggered the classification.

The TF-IDF + Logistic Regression pipeline makes this possible. Each word gets a coefficient that pushes the prediction toward AI or Human. By multiplying each word's TF-IDF score by its coefficient, you get a per-word influence score you can rank and display.

In this step, get ready to:
  • Extract per-word contributions from the trained model's coefficients.
  • Display the top flagged words alongside the verdict in the Streamlit app.
  • Test explainability by generating an AI resume and inspecting the flagged words.
Extract feature importance from the trained model

Your pipeline stores two things that combine to explain any prediction: the TF-IDF vocabulary (which words the model knows) and the Logistic Regression coefficients (how much each word pushes toward AI or Human).

  • In app.py, find the line that says pipeline = load_model().
  • Add the following function below it (leave a blank line between):
def get_top_words(text, pipeline, top_n=10):
    """Extract the top words contributing to the prediction."""
    vectorizer = pipeline.named_steps["tfidf"]
    classifier = pipeline.named_steps["classifier"]

    # Transform the input text
    tfidf_vector = vectorizer.transform([text])

    # Get feature names and coefficients
    feature_names = vectorizer.get_feature_names_out()
    coefficients = classifier.coef_[0]

    # Get the TF-IDF values for this specific text
    tfidf_array = tfidf_vector.toarray()[0]

    # Calculate per-word contribution (TF-IDF value * coefficient)
    contributions = tfidf_array * coefficients

    # Get top AI-leaning words (positive contributions)
    ai_indices = np.argsort(contributions)[-top_n:][::-1]
    ai_words = [(feature_names[i], contributions[i]) for i in ai_indices if contributions[i] > 0]

    # Get top Human-leaning words (negative contributions)
    human_indices = np.argsort(contributions)[:top_n]
    human_words = [(feature_names[i], contributions[i]) for i in human_indices if contributions[i] < 0]

    return ai_words, human_words

What does this code do?

This function reverse-engineers the classifier's decision for a single piece of text.

  • pipeline.named_steps["tfidf"] and pipeline.named_steps["classifier"] reach inside the pipeline to grab each component directly.
  • get_feature_names_out() returns the full vocabulary the TF-IDF vectorizer learned during training.
  • coef_[0] holds one weight per word. Positive weights push toward AI. Negative weights push toward Human.
  • Multiplying the TF-IDF value by the coefficient gives each word's actual contribution for this specific text. A word might have a high coefficient but low influence if it barely appears in the input.
  • Save app.py.

The function is defined but not called yet. The next chunk wires it into the Analyze button.

Display the top flagged words in the app

Now you need to call get_top_words() after the verdict renders, and display the results as a ranked list.

  • In app.py, scroll down to the # Handle analyze button section.
  • Find the line that says st.progress(ai_probability).
  • Add the following code directly below it:
    # Get and display top flagged words
    ai_words, human_words = get_top_words(resume_text, pipeline)

    st.markdown("---")
    st.subheader("Top Flagged Words")

    if ai_words:
        st.markdown("**Words pushing toward AI-Generated:**")
        for word, score in ai_words[:5]:
            st.markdown(f"- `{word}` (influence: {score:.3f})")

    if human_words:
        st.markdown("**Words pushing toward Human-Written:**")
        for word, score in human_words[:5]:
            st.markdown(f"- `{word}` (influence: {abs(score):.3f})")

What does this code do?

  • It calls get_top_words() with the pasted resume text and the trained pipeline.
  • ai_words[:5] shows the top 5 words whose TF-IDF presence pushes the prediction toward AI-Generated.
  • human_words[:5] shows the top 5 words pushing toward Human-Written.
  • abs(score) is used for human words because their coefficients are negative. The absolute value makes the display consistent.
  • Save app.py.

Your Streamlit app auto-reloads on save. Time to test it.

Test the explainability with a generated resume

The Generate AI Resume (Test) button you built in the previous step is the perfect way to produce a fresh AI resume and immediately inspect why the model flags it.

  • Switch back to your browser at localhost:8501.
  • Click Generate AI Resume (Test) to fill the text area with a fresh Claude-generated resume.
  • Click Analyze Resume.

Before you look at the result, what words do you predict will appear in the "Top Flagged Words" section for an AI-generated resume?

Below the verdict and confidence bar, you should now see a "Top Flagged Words" section. It lists words pushing toward AI-Generated (like "leveraged", "innovative", or "spearheaded") and words pushing toward Human-Written, each with an influence score.

Why these words?

AI-generated resumes lean heavily on polished buzzwords. Human resumes are messier and more specific ("our downtown branch", "about 3 weeks"). The model learned this contrast from your 80-sample dataset.

🙋‍♀️ Not seeing the Top Flagged Words section?

  • Make sure you saved app.py after adding both code chunks. Streamlit only reloads on save.
  • Check that the display code is indented inside the if analyze_button and resume_text.strip(): block. It should be at the same indent level as st.progress(ai_probability).
  • If you see an error about get_top_words not being defined, make sure the function is placed above the # App header line, not at the bottom of the file.

Help me debug my Top Flagged Words section not appearing.

  • Try pasting a human-written resume too (copy one of the bullet points from generate_dataset.py) and compare the flagged words.

✔️ Awesome, I've got everything!

Great. Make sure you have saved app.py with both the get_top_words() function and the display code.

ⓧ I'd like to double check the full code

Your app.py should contain these sections in order. Scroll up and confirm each is present:

  • Imports (streamlit, joblib, numpy, anthropic, os).
  • Page config and load_model() with @st.cache_resource.
  • The get_top_words() function (new in this step).
  • App header, text area, and two-column buttons.
  • Generate button handler.
  • Auto-fill logic for generated resumes.
  • Analyze button handler with verdict, metric, progress bar, AND the new Top Flagged Words section.

Your detector now shows its reasoning. A recruiter can see exactly which words triggered the classification. Next up, you will add a gap analysis feature that tells users how to rewrite those flagged sections to sound more authentic.

Add Resume Gap Analysis

Your detector flags which words gave the AI away and shows a confidence score. But a recruiter staring at a list of flagged words still has to guess what to do about them.

In this step, you will add a rules-based gap analysis feature. When the user clicks a button, your app maps each flagged AI word to a specific writing gap (like "uses generic buzzwords instead of named tools") and provides a concrete rewriting suggestion.

In this step, get ready to:
  • Build a rules dictionary that maps AI-flagged words to specific writing gaps and rewriting advice.
  • Add a "Get Rewriting Suggestions" button that only appears after analysis.
  • Display actionable gap cards showing what's wrong and how to fix it.
Create the gap analysis rules dictionary

Each AI-flagged word maps to a writing pattern that makes resumes sound machine-generated. Your dictionary captures the gap (what's wrong) and a concrete fix (what to write instead).

  • In app.py, find the line pipeline = load_model().
  • Add the first half of the GAP_RULES dictionary above the get_top_words() function:
# Rules-based gap analysis: maps AI-flagged words to writing advice
GAP_RULES = {
    "leveraged": {
        "gap": "Uses a generic power verb instead of a specific action.",
        "fix": "Replace with the actual action: 'used', 'built', 'configured'. Name the tool.",
    },
    "spearheaded": {
        "gap": "Overly dramatic verb that hides what you actually did.",
        "fix": "Say what you led: 'Led a 3-person team to ship...' with a specific outcome.",
    },
    "innovative": {
        "gap": "Adjective with no evidence. What was innovative about it?",
        "fix": "Replace with the specific thing that was new: 'first at our company to...'",
    },
    "orchestrated": {
        "gap": "Vague coordination verb. Orchestrated what, exactly?",
        "fix": "Name the pieces: 'Coordinated 3 vendors and 2 internal teams to deliver...'",
    },
    "streamlined": {
        "gap": "Claims efficiency without showing the before/after.",
        "fix": "Add numbers: 'Cut report generation from 3 hours to 20 minutes by...'",
    },
    "facilitated": {
        "gap": "Passive role description. Did you lead or just attend?",
        "fix": "Be specific: 'Ran weekly standups for 8 people' or 'Led client demos.'",
    },
  • Directly below the last entry, add the remaining rules and close the dictionary:
    "utilized": {
        "gap": "Unnecessarily formal. Nobody says 'utilized' in conversation.",
        "fix": "Just say 'used'. Then name the tool: 'Used Tableau to build dashboards.'",
    },
    "comprehensive": {
        "gap": "Filler adjective that adds no information.",
        "fix": "Remove it or specify scope: '45-page audit covering all 6 departments.'",
    },
    "strategic": {
        "gap": "Claims high-level thinking without evidence.",
        "fix": "Show the strategy: 'Shifted ad spend from Facebook to TikTok after...'",
    },
    "impactful": {
        "gap": "Self-congratulatory adjective. Let the numbers show impact.",
        "fix": "Replace with a metric: 'Increased sign-ups by 30% in Q2.'",
    },
    "scalable": {
        "gap": "Buzzword without context. Scalable to what?",
        "fix": "Name the scale: 'Handled 10x traffic spike during Black Friday.'",
    },
    "robust": {
        "gap": "Vague quality claim with no supporting detail.",
        "fix": "Describe what makes it reliable: '99.9% uptime over 6 months.'",
    },
}

What makes these rules effective?

Each rule targets a specific writing anti-pattern that AI text generators default to. The "gap" explains why the word sounds fake. The "fix" gives a concrete rewriting template with examples the user can adapt to their own experience.

Human resumes are specific, messy, and grounded in named tools, real people, and rough numbers. The rules push writers back toward that authentic style.

  • Save app.py.
Add the rewriting suggestions button and display

The suggestions should only appear when the user asks for them. You will add a button after the Top Flagged Words section that, when clicked, looks up each flagged word in your rules dictionary and renders a card with the gap and fix.

  • In app.py, scroll to the bottom of the if analyze_button and resume_text.strip(): block.
  • Add the following code after the Top Flagged Words display (below the last st.markdown line in the human_words loop):
    # Gap analysis button
    st.markdown("---")
    if st.button("Get Rewriting Suggestions"):
        st.subheader("Resume Gap Analysis")
        st.markdown(
            "Here's what makes this text sound AI-generated "
            "and how to fix it:"
        )

        found_gaps = False
        for word, score in ai_words[:5]:
            # Check single word and bigram parts
            word_lower = word.lower()
            matched_rule = None
            if word_lower in GAP_RULES:
                matched_rule = GAP_RULES[word_lower]
            else:
                for part in word_lower.split():
                    if part in GAP_RULES:
                        matched_rule = GAP_RULES[part]
                        break

            if matched_rule:
                found_gaps = True
                st.markdown(f"**`{word}`**")
                st.markdown(f"Gap: {matched_rule['gap']}")
                st.markdown(f"Fix: {matched_rule['fix']}")
                st.markdown("")
  • Add the fallback message directly below the loop:
        if not found_gaps:
            st.info(
                "No specific rewriting rules matched the flagged "
                "words. Try adding more rules to GAP_RULES for "
                "the words shown above."
            )

What does this code do?

  • st.button("Get Rewriting Suggestions") renders a button that only fires its inner block when clicked. The suggestions stay hidden until the user asks.
  • The loop checks each of the top 5 AI-flagged words against GAP_RULES. For bigrams (two-word phrases), it checks each part separately.
  • Each matched rule renders three lines: the flagged word in bold code, the gap explanation, and the concrete fix.
  • Save app.py.

Button shows but nothing appears when clicked?

  • Make sure the if st.button("Get Rewriting Suggestions"): block is inside the if analyze_button and resume_text.strip(): block (indented 4 spaces).
  • Check that GAP_RULES is defined above the analyze handler. Python reads top to bottom, so the dictionary must exist before it's referenced.
  • If you see suggestions for the generated resume but not for your own text, that's expected. The rules only match specific AI buzzwords.

Help me debug my Get Rewriting Suggestions button.

Test the gap analysis with a generated resume
  • Switch to your browser at localhost:8501.
  • Click Generate AI Resume (Test) to create a fresh AI resume.
  • Click Analyze Resume.
  • Scroll past the verdict and flagged words, then click Get Rewriting Suggestions.

You should see a "Resume Gap Analysis" section with cards showing the flagged word, what's wrong with it, and how to rewrite it.

  • Now try pasting a human-written resume bullet from generate_dataset.py and click through the same flow.

For a human-written sample, you should see fewer (or zero) matched rules, since human text avoids the buzzwords the rules target.

Extend the rules dictionary

The 12 rules cover common AI buzzwords, but you can add more. After testing, look at the flagged words that don't have rules yet and add entries for them. The more rules you add, the more useful the gap analysis becomes.

✔️ Awesome, I've got everything!

Great. Double check you have saved app.py and the gap analysis shows when you click Get Rewriting Suggestions after analyzing an AI resume.

ⓧ I'd like to double check the full code

Your app.py should now contain these sections in order:

  • Imports at the top.
  • Page config and load_model() with cache.
  • The GAP_RULES dictionary (new in this step).
  • The get_top_words() function.
  • App header, text area, buttons, generate handler, auto-fill.
  • Analyze handler with verdict, progress bar, Top Flagged Words, AND the new Get Rewriting Suggestions button with gap display logic.

Your detector now goes beyond flagging problems. It tells users exactly what's wrong and how to fix it. In the Secret Mission, you will add stylometric features that catch structural AI fingerprints that word frequency alone misses.

Secret mission

Add Stylometric Feature Analysis

Your TF-IDF classifier catches word-frequency patterns, but AI text has structural fingerprints too. Sentence lengths cluster tightly, buzzword density spikes, and vocabulary diversity drops. Can you add these stylometric signals to strengthen your detector?

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely on your local machine. The only ongoing cost is the Claude API, which charges per token only when you actively click the Generate AI Resume (Test) button.

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • ai-resume-detector project folder on your Desktop (contains .venv, Python scripts, trained model, and dataset).
  • ANTHROPIC_API_KEY environment variable set in your terminal session.
  • Claude API key stored on platform.claude.com (usage charged per token when called).

Keep everything running

Choose this if you plan to keep experimenting with the detector or generating new test resumes.

  • No action needed. Your model and app live locally with no ongoing cost.
  • API charges only occur when you click Generate AI Resume (Test) in the Streamlit app. Each click costs a fraction of a cent.
  • The ANTHROPIC_API_KEY environment variable only persists in your current terminal session. It disappears when you close the terminal.

Pause - I'll come back to this later

Shut down the running app and clear your API key from the session to prevent accidental usage.

  • Switch to the terminal where Streamlit is running and press Ctrl+C to stop the server.
  • Clear the API key from your current session by running the appropriate command for your shell.

macOS

unset ANTHROPIC_API_KEY

Windows

set ANTHROPIC_API_KEY=
  • Your project files stay on disk. When you return, reactivate the virtual environment and set the API key again.

Delete - I don't want to use this again

Remove all project files and revoke your API key so nothing remains.

  • Switch to the terminal where Streamlit is running and press Ctrl+C to stop the server.
  • Deactivate the virtual environment by running:
deactivate
  • Navigate to your Desktop and delete the entire project folder.

macOS

cd ~/Desktop
rm -rf ai-resume-detector

Windows

cd ~\Desktop
rmdir /s /q ai-resume-detector
  • Revoke your Claude API key so it can no longer be used.
  • Go to console.anthropic.com/settings/keys.
  • Find the API key you created for this project and click the trash icon to delete it.

That's a Wrap!

That's a Wrap!

Nice work. You just built a working AI-content detection system from scratch, trained it on real and AI-generated resume data, and wrapped it in a web app that anyone can use.

You've learned how to:

  • Train a TF-IDF + Logistic Regression pipeline that classifies text as human-written or AI-generated, and evaluate it with precision, recall, and F1-score.
  • Build a Streamlit web app that loads a trained model, runs predictions on pasted text, and displays a confidence score with a color-coded verdict.
  • Add rules-based gap analysis that maps flagged AI words to specific writing problems and gives concrete rewriting suggestions.
  • Secret Mission: Add stylometric features (sentence length variance, buzzword density, type-token ratio) to catch structural AI fingerprints that word frequency alone misses.

Ready to quiz yourself?