Movie Review Sentiment Analyzer

Classify movie reviews as positive or negative using pretrained transformer models.

Introduction

30 Second Summary

Every time you scroll through movie reviews trying to decide what to watch, you're doing sentiment analysis in your head. You're reading text and deciding: positive or negative. Now imagine doing that automatically across thousands of reviews in seconds.

In this project, you will build a sentiment analysis pipeline that classifies movie reviews as positive or negative using pretrained transformer models. You will evaluate performance on real data and compare two models head-to-head.

What You'll Build

You will type any movie review into your script and instantly see it classified as positive or negative with a confidence percentage, then run that same model against hundreds of real IMDB reviews to measure exactly how accurate it is.

By the end of this project, you'll have:

A working sentiment analysis pipeline that takes any text and returns a prediction like "POSITIVE 98.3%" or "NEGATIVE 94.1%" in real time.
A quantitative evaluation showing your model's accuracy, F1, precision, and recall scores computed automatically across hundreds of real IMDB movie reviews.
A side-by-side model comparison proving that a domain-specific model outperforms a general one on movie review data.
Secret Mission: An interactive command-line tool that lets you type reviews and see live predictions from both models side-by-side.

Are there any prerequisites?

This project assumes you have basic Python programming knowledge. You should already have Python and VS Code installed on your machine. No prior machine learning experience is needed.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters.

Set Up Your ML Environment

To build your sentiment analyzer, you need specialized machine learning libraries that handle downloading pretrained models, loading datasets, and computing evaluation metrics.

In this step, you'll create an isolated project environment and install the core Hugging Face libraries. A virtual environment keeps this project's packages separate from your system Python, preventing version conflicts with other projects on your machine.

In this step, get ready to:

Create a project folder with an isolated Python virtual environment.
Install the Hugging Face Transformers, Datasets, and Evaluate libraries.
Verify the setup by running a real sentiment prediction.

Create your project folder and virtual environment

You need a dedicated folder for this project and a virtual environment to install libraries without affecting your system Python.

Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
Type Terminal (macOS) or PowerShell (Windows) and press Enter to open it.
Move to your Desktop by running this command:

cd ~/Desktop

Create your project folder and move into it by running these commands:

mkdir sentiment-analyzer
cd sentiment-analyzer

Create a virtual environment by running this command:

python -m venv venv

Confirm the virtual environment was created by running:

ls

You should see a venv folder listed in the output.

Don't see the venv folder?

Make sure you ran the command from inside the sentiment-analyzer folder. Run pwd (macOS) or cd (Windows) to check your current location.

If you see an error about python not being found, try python3 -m venv venv instead.

Help me figure out why my venv wasn't created.

Now activate the virtual environment so any packages you install go into this isolated space.

macOS

Activate the virtual environment by running:

source venv/bin/activate

Windows

Activate the virtual environment by running:

.\venv\Scripts\Activate.ps1

Your terminal prompt now shows (venv) at the beginning. This confirms you're working inside the virtual environment.

Not seeing (venv) in your prompt?

On Windows PowerShell, you may see an execution policy error. Run Set-ExecutionPolicy RemoteSigned -Scope CurrentUser and then try the activate command again.

On macOS, double-check you typed source before the path. Without it, the shell won't execute the activation script.

Help me activate my virtual environment.

Install the required libraries

With your virtual environment active, you can now install the machine learning libraries. These are the building blocks of your sentiment analysis pipeline.

Install all four libraries by running this command:

pip install "transformers[torch]" datasets evaluate scikit-learn

What does this install?

transformers[torch] installs Hugging Face Transformers along with PyTorch as the deep learning backend.
datasets provides one-line access to thousands of NLP datasets, including IMDB movie reviews.
evaluate computes standard ML metrics like accuracy and F1 score.
scikit-learn provides the math behind precision, recall, and other metric calculations.

The installation downloads several hundred megabytes of dependencies. Wait until you see the terminal return to your prompt with no errors.

Seeing installation errors?

Make sure (venv) appears in your terminal prompt. If it doesn't, you're installing into your system Python. Go back and activate the virtual environment first.

If you see a permissions error, do not use sudo. Instead, confirm you activated the venv correctly.

Help me fix my pip install errors.

Verify the installation

Time to confirm everything works by running a real sentiment prediction. This single command loads a pretrained model and classifies a sentence.

Before you run the next command, predict: what do you think the model will output for the phrase "I love this!"? Will it be positive or negative, and how confident?
Test your installation by running this command:

python -c "from transformers import pipeline; print(pipeline('sentiment-analysis')('I love this!'))"

What does this command do?

This one-liner imports the pipeline function from Transformers, creates a sentiment analysis pipeline (which downloads a pretrained DistilBERT model), and classifies the text "I love this!" as positive or negative.

The first time you run this, it downloads the model files. This may take 30-60 seconds depending on your internet speed.

You should see output like this:

[{'label': 'POSITIVE', 'score': 0.9998}]

The model classified "I love this!" as POSITIVE with a confidence score very close to 1. Your ML environment is fully working.

Seeing an error instead of a prediction?

If you see ModuleNotFoundError: No module named 'transformers', your venv is not activated. Check that (venv) appears in your prompt and try again.

If you see a network error, the command is trying to download the model. Check your internet connection and try again.

Help me get my verification command working.

Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
Type Visual Studio Code and press Enter to open it.
Click File in the top menu bar.
Click Open Folder.
Navigate to your Desktop, select the sentiment-analyzer folder, and click Open.

Why open VS Code now?

In the next step, you'll create a Python script to build your sentiment analysis pipeline. Having VS Code open in the project folder gives you a code editor and an integrated terminal in one window.

Your ML environment is ready and you just ran your first real sentiment prediction. Next up, you'll build a Python script that classifies movie reviews and start exploring how the model handles different types of text.

Run Your First Sentiment Predictions

Your ML environment is ready with all the libraries installed. Now it's time to see the model actually work.

Before worrying about datasets or evaluation metrics, you need to build intuition for what a sentiment analysis pipeline does. In this step, you'll load a pretrained DistilBERT model and feed it movie reviews to classify as positive or negative.

In this step, get ready to:

Load a pretrained sentiment analysis model.
Run predictions on example movie reviews.
See confidence scores for each prediction.

Create sentiment.py and load the pipeline

Hugging Face Transformers provides a pipeline function that wraps model loading, tokenization, and inference into a single call. You'll use the sentiment-analysis task with a DistilBERT model that was fine-tuned on movie review sentiment.

In VS Code, create a new file called sentiment.py in your sentiment-analyzer folder.
Add the following code to load the sentiment analysis pipeline:

from transformers import pipeline

# Step 2: Load the default sentiment analysis pipeline
print("Loading models...")
classifier_sst2 = pipeline("sentiment-analysis", model="distilbert/distilbert-base-uncased-finetuned-sst-2-english")
print("Models loaded!\n")

What does this code do?

pipeline("sentiment-analysis", model="...") downloads a pretrained model and wraps it in a ready-to-use classifier.
The model distilbert/distilbert-base-uncased-finetuned-sst-2-english is a DistilBERT model fine-tuned specifically on sentiment data.
The print statements give you feedback while the model loads (which can take a moment the first time).

Save sentiment.py.
Run the script in your terminal to confirm the model loads:

python sentiment.py

You should see Loading models... followed by Models loaded! printed in your terminal.

First run takes longer

The first time you run this, Transformers downloads the model weights (~250 MB). This is a one-time download. Future runs load the cached model instantly.

Seeing an error on import?

Make sure your virtual environment is activated. You should see (venv) at the start of your terminal prompt.
If you see ModuleNotFoundError: No module named 'transformers', re-run the install command from Step 1.

Why isn't my model loading?

Run predictions on test sentences

Now that the model loads, you can feed it text and get back a sentiment prediction. Each prediction returns a label (POSITIVE or NEGATIVE) and a confidence score between 0 and 1.

In sentiment.py, add the following code below the existing lines:

# Step 2: Quick test predictions
test_sentences = [
    "This movie was absolutely fantastic!",
    "What a terrible waste of time.",
    "The movie had great acting but a terrible plot.",
]

print("=== Quick Test Predictions (SST-2 Model) ===")
for sentence in test_sentences:
    result = classifier_sst2(sentence)[0]
    print(f"  '{sentence[:50]}...' -> {result['label']} ({result['score']:.4f})")
print()

What does this code do?

test_sentences is a list of three reviews: one clearly positive, one clearly negative, and one mixed.
The for loop passes each sentence through the classifier and grabs the first result.
The f-string prints the text, the predicted label, and the confidence score formatted to 4 decimal places.

Save sentiment.py.

Before you run this, what do you think the model will predict for the third sentence ("The movie had great acting but a terrible plot")? Will it say POSITIVE or NEGATIVE, and how confident do you think it will be?

Run the script again to see predictions:

python sentiment.py

You should see output like this for each sentence: a label (POSITIVE or NEGATIVE) followed by a confidence score in parentheses. The first sentence gets POSITIVE with high confidence, and the second gets NEGATIVE with high confidence.

Not seeing predictions printed?

Check that the new code is below the pipeline loading code, not inside or above it.
Make sure you saved the file before running python sentiment.py again.

My predictions aren't printing correctly.

✔️ Awesome, I've got everything!

Great. Double check you've saved sentiment.py and move on to the next section.

ⓧ I'd like to double check the full code

from transformers import pipeline

# Step 2: Load the default sentiment analysis pipeline
print("Loading models...")
classifier_sst2 = pipeline("sentiment-analysis", model="distilbert/distilbert-base-uncased-finetuned-sst-2-english")
print("Models loaded!\n")

# Step 2: Quick test predictions
test_sentences = [
    "This movie was absolutely fantastic!",
    "What a terrible waste of time.",
    "The movie had great acting but a terrible plot.",
]

print("=== Quick Test Predictions (SST-2 Model) ===")
for sentence in test_sentences:
    result = classifier_sst2(sentence)[0]
    print(f"  '{sentence[:50]}...' -> {result['label']} ({result['score']:.4f})")
print()

Observe the output format

Look at your terminal output. Each prediction is a dictionary with two fields: label (either POSITIVE or NEGATIVE) and score (a confidence value between 0 and 1).

What about the ambiguous sentence?

The third sentence ("great acting but terrible plot") is interesting because it contains both positive and negative language. Notice the confidence score. It's likely lower than the other two predictions.

A score of 0.99 means the model is very sure. A score closer to 0.5 means the model is essentially guessing. This is your first glimpse into where models struggle: mixed-sentiment text.

You've now seen a pretrained NLP model classify text in real time. Next, you'll test it against thousands of real movie reviews to find out how accurate it truly is.

Load Real Data and Hit a Problem

Your model nailed three hand-picked examples in the previous step. But three samples tell you nothing about real-world performance.

To discover whether this model actually works well on movie reviews, you need to test it on real data. In this step, you'll load 25,000 actual IMDB movie reviews and try running your model against a sample.

In this step, get ready to:

Load the IMDB movie review dataset from Hugging Face.
Explore the dataset structure and contents.
Run the model on a sample and count its accuracy by hand.

Load and explore the IMDB dataset

The Hugging Face Datasets library gives you one-line access to thousands of public datasets. The IMDB dataset contains 25,000 real movie reviews labeled as positive or negative.

In sentiment.py, find the existing import at the top of the file:

from transformers import pipeline

Add this import directly below it:

from datasets import load_dataset

Now add this code at the bottom of your file, after the print() line:

# Step 3: Load the IMDB dataset
print("Loading IMDB dataset...")
dataset = load_dataset("stanfordnlp/imdb")
print(f"Dataset loaded: {dataset}")
print(f"Test split: {len(dataset['test'])} examples")
print(f"Fields: {dataset['test'].column_names}")
print(f"\nExample review (first 200 chars): {dataset['test'][0]['text'][:200]}...")
print(f"Label: {dataset['test'][0]['label']} (0=negative, 1=positive)\n")

What does this code do?

load_dataset("stanfordnlp/imdb") downloads the full IMDB movie review dataset from Hugging Face Hub. The first time you run this, it downloads about 84 MB and caches it locally.
The print statements show you the dataset's structure: how many examples it has, what fields each example contains, and what an actual review looks like.
Each example has a text field (the review) and a label field (0 for negative, 1 for positive).

Save sentiment.py.
Run your script by running this command:

python sentiment.py

After the model loads and prints the test predictions from before, you should see new output showing the dataset structure. Look for the test split with 25,000 examples and the fields text and label. You'll also see the first 200 characters of a real movie review.

Why does it say stanfordnlp/imdb?

The IMDB dataset lives under the stanfordnlp namespace on Hugging Face Hub. Using just "imdb" without the namespace may cause errors. Always use the full path stanfordnlp/imdb.

Seeing a download error?

Make sure you have an internet connection. The dataset is downloaded from Hugging Face Hub the first time you run the script.
If the download times out, try running the script again. The dataset is hosted on an external server that occasionally has connection issues.
help me debug a dataset download error

Run the pipeline on a small sample

You've loaded 25,000 reviews. Now the question is: how well does your model actually perform on this real data? A quick way to find out is to run predictions on a sample and count how many it gets right.

Add this code at the bottom of sentiment.py:

# Rough accuracy check on first 100 reviews
print("Running model on 100 test reviews...")
correct = 0
for i in range(100):
    text = dataset["test"][i]["text"]
    true_label = dataset["test"][i]["label"]
    result = classifier_sst2(text, truncation=True, max_length=512)[0]
    predicted = 1 if result["label"] == "POSITIVE" else 0
    if predicted == true_label:
        correct += 1
    if (i + 1) % 25 == 0:
        print(f"  Processed {i + 1}/100...")

print(f"\nRough accuracy: {correct}/100 ({correct/100:.1%})")

What does this code do?

The loop grabs each review's text and true label from the test split.
It runs the classifier_sst2 pipeline on each review. The truncation=True, max_length=512 parameters handle reviews that are longer than the model can process at once.
The model outputs POSITIVE or NEGATIVE labels, so you map those to 1 and 0 to compare against the dataset's integer labels.
Progress prints every 25 reviews so you know it's working. Running 100 reviews one-by-one on CPU takes 1-2 minutes.

Save sentiment.py.

Before you run this, what accuracy do you think the model will get? Remember, it was trained on short sentences (SST-2) but you're testing it on long movie reviews. Will it be perfect?

Run your script:

python sentiment.py

After 1-2 minutes of processing, you'll see a rough accuracy somewhere around 80-90 out of 100. Not terrible, but not perfect either.

What does this result tell you?

The model gets some reviews wrong. But counting 100 predictions by hand raises two problems.

It doesn't scale. What about the other 24,900 reviews?
A raw count misses important patterns. A model that always predicts POSITIVE would look 50% accurate on balanced data, but it's actually useless for detecting negative reviews.
You need proper evaluation metrics that reveal these patterns. That's what the next step introduces.

Script seems stuck or too slow?

Running 100 predictions on CPU takes 1-2 minutes. Each review is processed individually, which is slow. The progress counter prints every 25 reviews so you can confirm it's working.
If you see no output after 3+ minutes, press Ctrl+C to stop the script and try again. Make sure no other heavy processes are running.
help me figure out why my sentiment analysis script is running slowly

✔️ Awesome, I've got everything!

Great. Double check you've saved sentiment.py and that your terminal shows the dataset structure plus the rough accuracy count.

ⓧ I'd like to double check the full code

Here is the persistent code for sentiment.py up to this point. Your file will also have the 100-review loop at the bottom, which gets replaced with proper metrics in the next step.

from transformers import pipeline
from datasets import load_dataset

# Step 2: Load the default sentiment analysis pipeline
print("Loading models...")
classifier_sst2 = pipeline("sentiment-analysis", model="distilbert/distilbert-base-uncased-finetuned-sst-2-english")
print("Models loaded!\n")

# Step 2: Quick test predictions
test_sentences = [
    "This movie was absolutely fantastic!",
    "What a terrible waste of time.",
    "The movie had great acting but a terrible plot.",
]

print("=== Quick Test Predictions (SST-2 Model) ===")
for sentence in test_sentences:
    result = classifier_sst2(sentence)[0]
    print(f"  '{sentence[:50]}...' -> {result['label']} ({result['score']:.4f})")
print()

# Step 3: Load the IMDB dataset
print("Loading IMDB dataset...")
dataset = load_dataset("stanfordnlp/imdb")
print(f"Dataset loaded: {dataset}")
print(f"Test split: {len(dataset['test'])} examples")
print(f"Fields: {dataset['test'].column_names}")
print(f"\nExample review (first 200 chars): {dataset['test'][0]['text'][:200]}...")
print(f"Label: {dataset['test'][0]['label']} (0=negative, 1=positive)\n")

You've seen real data and discovered a gap. Counting by hand doesn't scale and misses important patterns. Next up, you'll evaluate with proper machine learning metrics that reveal exactly where and how the model fails.

Evaluate with Real Metrics

You loaded real review data and counted correct predictions by hand in the previous step. That rough count told you the model isn't perfect, but it didn't tell you how it fails.

A model that always predicts POSITIVE would score 50% accuracy on a balanced dataset, but you'd never catch that pattern without proper metrics. In this step, you'll use the Hugging Face Evaluate library to compute accuracy, F1, precision, and recall on 200 real IMDB reviews.

In this step, get ready to:

Set up the evaluate library to compute multiple metrics at once.
Run the SST-2 model on 200 real test reviews and collect predictions.
See accuracy, F1, precision, and recall printed in a clean format.

Set up the evaluation framework

The evaluate library provides standardized ML metrics that work with Hugging Face models. You already installed it in Step 1. Now you'll import it and prepare the data for evaluation.

In sentiment.py, find this line at the top of your file:

from datasets import load_dataset

Add this new import directly below it, so the line now reads:

import evaluate

Now scroll to the very bottom of sentiment.py (after the dataset print statements) and add this code:

# Step 4: Evaluate with real metrics
NUM_SAMPLES = 200
print(f"=== Evaluating on {NUM_SAMPLES} test reviews ===\n")

metric = evaluate.combine(["accuracy", "f1", "precision", "recall"])

test_texts = [dataset["test"][i]["text"] for i in range(NUM_SAMPLES)]
true_labels = [dataset["test"][i]["label"] for i in range(NUM_SAMPLES)]

What does this code do?

NUM_SAMPLES = 200 limits evaluation to 200 reviews. Running all 25,000 would take over an hour on CPU.
evaluate.combine() bundles four metrics into one object so you can compute them all in a single call.
test_texts and true_labels extract the review text and correct labels from the first 200 test examples.

Save sentiment.py and run it to confirm the new section header appears:

python sentiment.py

At the end of the output (after the dataset info), you should see the line === Evaluating on 200 test reviews === printed.

Seeing a ModuleNotFoundError for evaluate?

Make sure your virtual environment is activated. You should see (venv) in your terminal prompt. If not, activate it again and rerun the script.

If it's activated but the error persists, reinstall with:

pip install evaluate scikit-learn

Help me fix this ModuleNotFoundError for evaluate in my sentiment analysis project.

Run the model and compute metrics

Now you have the data and metrics ready. Time to run all 200 reviews through the SST-2 model and see how it actually performs on real movie review data.

Add this code at the bottom of sentiment.py, directly after the true_labels line:

# Evaluate SST-2 model
print("Running SST-2 model...")
sst2_predictions = classifier_sst2(test_texts, truncation=True, max_length=512)
sst2_pred_labels = [1 if p["label"] == "POSITIVE" else 0 for p in sst2_predictions]

sst2_results = metric.compute(references=true_labels, predictions=sst2_pred_labels)
print(f"  Accuracy:  {sst2_results['accuracy']:.4f}")
print(f"  F1:        {sst2_results['f1']:.4f}")
print(f"  Precision: {sst2_results['precision']:.4f}")
print(f"  Recall:    {sst2_results['recall']:.4f}")
print()

What does this code do?

classifier_sst2(test_texts, truncation=True, max_length=512) runs all 200 reviews through the model at once. truncation=True handles reviews longer than the model's 512-token limit by cutting them short.
The list comprehension maps the model's text labels (POSITIVE/NEGATIVE) to numbers (1/0) so they can be compared with the dataset's integer labels.
metric.compute() calculates all four metrics by comparing your predicted labels against the true labels.

Save sentiment.py.

Before you run this, think about what accuracy you expect. The SST-2 model was trained on short sentences and reports 91.3% accuracy on that data. You're testing it on long movie reviews. Will it do just as well, or will the domain mismatch hurt it?

Run the script:

python sentiment.py

This will take 1-3 minutes as it processes 200 reviews on your CPU. At the end of the output, you'll see four metrics printed for the SST-2 model: accuracy, F1, precision, and recall.

What do these metrics tell you?

Accuracy is the percentage of reviews the model classified correctly overall.
Precision tells you: of all the reviews the model labeled POSITIVE, how many actually were positive?
Recall tells you: of all the actually-positive reviews, how many did the model catch?
F1 is the harmonic mean of precision and recall. It balances both into a single number.

If precision is much higher than recall, the model is conservative (misses real positives). If recall is higher, it's trigger-happy (labels too many things as positive).

Script seems stuck or taking too long?

Processing 200 reviews on CPU is expected to take 1-3 minutes. If you see "Running SST-2 model..." printed, it's working. Just wait for it to finish.

If it's been more than 5 minutes, press Ctrl+C to stop, reduce NUM_SAMPLES to 50, save, and rerun.

Help me troubleshoot why my sentiment analysis evaluation is running slowly.

✔️ Awesome, I've got everything!

Great. Double check you've saved your file, then move on.

ⓧ I'd like to double check the full code

Your sentiment.py should have these sections in order. Scroll through your file and confirm each one is present:

Imports at the top: transformers, datasets, and evaluate.
Model loading and quick test predictions (from Step 2).
IMDB dataset loading and exploration (from Step 3).
Evaluation setup: NUM_SAMPLES, metric, test_texts, and true_labels (from this step, substep 1).
SST-2 model evaluation: sst2_predictions, sst2_pred_labels, sst2_results, and four metric print statements (from this step, substep 2).

Why might the scores be lower than 91.3%?

The SST-2 model was fine-tuned on short movie review sentences (one or two lines). IMDB reviews are full paragraphs. This domain mismatch means the model encounters patterns it wasn't trained on.

Look at your precision vs. recall. If one is noticeably lower, that tells you whether the model is missing positives or creating false positives on long-form text.

You now have real numbers showing how this general-purpose model handles movie reviews. In the next step, you'll load a model trained specifically on IMDB data and see if training on the right domain makes a measurable difference.

Compare a Different Model

You now have hard numbers for one model. But a single score in isolation does not tell you much.

The real question is: can a different model do better? In this step, you will load a second DistilBERT model that was specifically trained on IMDB reviews, run the exact same evaluation, and compare results head-to-head. This is the critical ML skill of model selection.

In this step, get ready to:

Load a second model fine-tuned on IMDB movie reviews.
Run the same evaluation on both models for a fair comparison.
Display a side-by-side comparison table showing which model wins on each metric.

Load a second model trained on IMDB data

The SST-2 model was trained on short sentences from movie reviews. The model you are about to load, lvwerra/distilbert-imdb, is the same DistilBERT architecture but fine-tuned directly on IMDB data. Same architecture, different training data.

In sentiment.py, find this line near the top of the file:

classifier_sst2 = pipeline("sentiment-analysis", model="distilbert/distilbert-base-uncased-finetuned-sst-2-english")

Add this line directly below it to load the IMDB-specific model:

classifier_imdb = pipeline("sentiment-analysis", model="lvwerra/distilbert-imdb")

Why this specific model?

Both models use the same DistilBERT architecture and the same number of parameters. The only difference is the training data. This isolates a single variable (training domain) so the comparison is fair.

Save sentiment.py.
Run the script to confirm both models load successfully by running:

python sentiment.py

You should see "Loading models..." followed by "Models loaded!" with no errors. The IMDB model will download on first run (about 250MB).

Model download failing?

Make sure you have a working internet connection. The model downloads from the Hugging Face Hub.

Check that you typed the model name exactly as shown: lvwerra/distilbert-imdb. A typo in the model ID causes an HTTP 404 error.

Help me debug a model loading error in Hugging Face Transformers.

Evaluate the IMDB model on the same test data

For a fair comparison, both models must be evaluated on the exact same 200 test reviews. You already have test_texts and true_labels ready from the previous step.

In sentiment.py, find the blank line after the SST-2 results are printed (after the print() at the end of the SST-2 evaluation section).
Add the following code below it to evaluate the IMDB model:

# Step 5: Evaluate IMDB model
print("Running IMDB model...")
imdb_predictions = classifier_imdb(test_texts, truncation=True, max_length=512)
imdb_pred_labels = [1 if p["label"] == "POSITIVE" else 0 for p in imdb_predictions]

metric_imdb = evaluate.combine(["accuracy", "f1", "precision", "recall"])
imdb_results = metric_imdb.compute(references=true_labels, predictions=imdb_pred_labels)
print(f"  Accuracy:  {imdb_results['accuracy']:.4f}")
print(f"  F1:        {imdb_results['f1']:.4f}")
print(f"  Precision: {imdb_results['precision']:.4f}")
print(f"  Recall:    {imdb_results['recall']:.4f}")
print()

What does this code do?

classifier_imdb(test_texts, truncation=True, max_length=512) runs the IMDB model on the same 200 reviews. The truncation=True parameter cuts reviews that exceed 512 tokens (the model's maximum input length).
metric_imdb = evaluate.combine(...) creates a fresh metric object. You need a new one because the previous metric was already consumed by the SST-2 evaluation.
The label mapping is identical: POSITIVE becomes 1, NEGATIVE becomes 0.

Save sentiment.py and run it:

python sentiment.py

After the SST-2 results, you should now see "Running IMDB model..." followed by four metric scores for the IMDB model.

Only seeing the SST-2 results?

Make sure your new code is placed after the SST-2 evaluation block, not inside it. Look for the final print() that ends the SST-2 section and add your code below that line.

Help me figure out why my IMDB model evaluation code is not running.

Display the comparison table

Raw numbers for each model are useful, but a side-by-side table makes the comparison instant. You will print a formatted table that declares a winner for each metric.

Add the following code at the bottom of sentiment.py, after the IMDB results print block:

# Step 5: Comparison table
print("=== Model Comparison ===")
print(f"{'Metric':<12} {'SST-2 Model':<15} {'IMDB Model':<15} {'Winner'}")
print("-" * 55)
for metric_name in ["accuracy", "f1", "precision", "recall"]:
    sst2_val = sst2_results[metric_name]
    imdb_val = imdb_results[metric_name]
    winner = "IMDB" if imdb_val > sst2_val else "SST-2"
    print(f"{metric_name:<12} {sst2_val:<15.4f} {imdb_val:<15.4f} {winner}")
print()
print("Key insight: The IMDB-trained model likely performs better because")
print("it was fine-tuned on the same type of data (long movie reviews).")
print("The SST-2 model was trained on short sentences - a domain mismatch.")

What does this code do?

The f"{...:<12}" syntax left-aligns each column to a fixed width, creating a neat table.
The loop iterates over each metric name and looks it up in both result dictionaries.
The winner variable compares the two scores and picks whichever model scored higher on that metric.

Save sentiment.py.

Before you run this, what do you predict? Will the IMDB-trained model beat the SST-2 model on movie reviews, or will the general-purpose model hold its own?

Run the full script:

python sentiment.py

At the bottom of the output, you should see a formatted comparison table with columns for Metric, SST-2 Model, IMDB Model, and Winner. The IMDB model should win on most (or all) metrics because it was trained on the same type of long movie reviews it is being tested on.

Why does the IMDB model win?

This demonstrates a core ML concept called domain adaptation. The SST-2 model was trained on short movie-review sentences. IMDB reviews are full paragraphs with complex reasoning, sarcasm, and nuance.

A model trained on data that matches the deployment domain almost always outperforms a general-purpose model. This is why in real ML projects, you always evaluate multiple models before choosing one.

Seeing a KeyError or missing values?

Make sure the variable names sst2_results and imdb_results match exactly what you assigned earlier. Both should be dictionaries returned by metric.compute().

Help me debug a KeyError in my comparison table code.

✔️ Awesome, I've got everything!

Great. Double check that you have saved sentiment.py and your comparison table prints correctly at the bottom of the output.

ⓧ I'd like to double check the full code

Your sentiment.py should contain these sections in order. Scroll through your file and confirm each is present:

Imports (pipeline, load_dataset, evaluate).
Loading both models (classifier_sst2 and classifier_imdb).
Quick test predictions with 3 sentences.
IMDB dataset loading and exploration.
SST-2 model evaluation (200 samples, printing accuracy/F1/precision/recall).
IMDB model evaluation (same 200 samples, same metrics).
Comparison table with a Winner column and the key insight print statements.

You now have quantitative proof that training data matters more than architecture alone. Next up, you will take these models and build something you can actually use day-to-day: an interactive classifier that predicts sentiment on anything you type.

Secret mission

Build an Interactive Classifier

Your evaluation script runs once and exits. What if you could type any sentence and see both models' predictions instantly, side-by-side? In this secret mission, you'll build a command-line tool that classifies any text in real-time using both models.

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your resources on disk, pause them to come back later, or delete them entirely.

Resources you used:

The sentiment-analyzer project folder on your Desktop (includes virtual environment and sentiment.py).
Hugging Face model cache stored at ~/.cache/huggingface/ (approximately 500MB of downloaded model files).

Keep everything running

No action needed. Choose this if you want to keep experimenting with sentiment analysis or try different models.

Your virtual environment and cached models stay on your machine (approximately 500MB total).
You can reactivate the environment any time by navigating to the sentiment-analyzer folder and running the activate command.

Pause - I'll come back to this later

Deactivate your virtual environment to free up your terminal session. Your files and cached models remain on disk for when you return.

Deactivate the virtual environment by running:

deactivate

Your project folder and Hugging Face cache remain intact. When you come back, navigate to the folder and reactivate the virtual environment.

Delete - I don't want to use this again

Remove all project files and cached models to free up disk space.

Deactivate your virtual environment first by running:

deactivate

Delete the project folder by running:

macOS

cd ~/Desktop
rm -rf sentiment-analyzer

Windows

cd ~/Desktop
rmdir /s /q sentiment-analyzer

Delete the Hugging Face model cache to free up the remaining disk space:

macOS

rm -rf ~/.cache/huggingface/

Windows

rmdir /s /q %USERPROFILE%\.cache\huggingface

Nice Work!

Nice work! You just built a complete sentiment analysis system that classifies movie reviews using pretrained transformer models, evaluates their performance on real data, and compares them head-to-head.

You've learned how to:

Built a sentiment analysis pipeline using Hugging Face Transformers that classifies any text as positive or negative with a confidence score.
Evaluated model performance on 200 real IMDB reviews using proper ML metrics: accuracy, F1, precision, and recall.
Compared a general-purpose model against a domain-specific model and proved that training data matters more than architecture alone.
Secret Mission: Built an interactive command-line tool that classifies any text you type with both models side-by-side.

Ready to quiz yourself?