Build a Hybrid Movie Recommender

Build a recommendation engine combining content and collaborative filtering.

Introduction

30 Second Summary

Every time Netflix suggests a show you end up binge-watching, or Spotify queues a song you have never heard but instantly love, a recommendation engine is working behind the scenes. These systems combine what you like with what people similar to you enjoy to surface surprisingly good picks.

In this project, you will build a movie recommendation engine in Python that combines two real-world techniques: genre-based similarity and user-behaviour matching. You will experience firsthand why neither technique works alone, then solve the gap with a hybrid approach.

What You'll Build

You will type a movie title into your terminal and get back five movies you would probably love, powered by the same architectural pattern that Netflix and Spotify use in production.

By the end of this project, you'll have:

  • A content-based recommender where you type a movie title and instantly see five similar movies based on TF-IDF genre vectors and cosine similarity.
  • A collaborative filtering recommender where you enter a user ID and get cross-genre discoveries based on what users with similar taste enjoyed.
  • A hybrid engine that blends both signals with weighted scoring and gracefully handles the cold-start problem when a brand-new user has no rating history.
  • Secret Mission: Add precision@k evaluation to measure which approach actually produces the best recommendations.

Are there any prerequisites?

You should be comfortable with basic Python (variables, functions, loops) and have a basic familiarity with pandas DataFrames. No machine learning experience is required.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters. This step is your moment to commit to the goal before any hands-on work begins.

Set Up Your Environment

Every recommendation engine needs two things: real data to learn from and a clean environment to build in.

In this step, you'll create an isolated Python environment, install the data science libraries your engine needs, and download the MovieLens dataset containing thousands of real movie ratings.

In this step, get ready to:
  • Create a project folder and isolated virtual environment.
  • Install pandas and scikit-learn.
  • Download and load the MovieLens dataset.
Create project folder and virtual environment
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir movie-recommender
cd movie-recommender
  • Confirm the folder was created by running:
ls

You should see an empty directory. That's your blank canvas.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code and press Enter to open it.
  • Click File in the top menu bar.
  • Click Open Folder.
  • Select the movie-recommender folder on your Desktop, then click Open.

You should see an empty folder open in the VS Code Explorer sidebar.

  • Open a terminal inside VS Code by pressing Ctrl+` (backtick key, same on both macOS and Windows).
  • Create a virtual environment by running this command:
python -m venv venv

What is a virtual environment?

A virtual environment is an isolated Python installation inside your project folder.

It keeps this project's library versions separate from any other Python project on your machine. No version conflicts, no surprises.

  • Activate the virtual environment by running:

macOS

source venv/bin/activate

Windows

venv\Scripts\activate

You should see (venv) appear at the beginning of your terminal prompt. This confirms the virtual environment is active.

Don't see (venv) in your prompt?

On Windows, you may need to allow script execution first. Run Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned in PowerShell, then try activating again.

On macOS, make sure you typed source before the path. Without it, the command won't work.

help me activate my Python virtual environment

  • Install the two libraries your recommendation engine needs by running:
pip install pandas==3.0.4 scikit-learn==1.9.0

You should see output ending with Successfully installed followed by the package names.

Why these two libraries?

pandas handles loading and manipulating the movie rating data. scikit-learn provides the machine learning tools you'll use to compute similarity between movies and find users with matching taste.

  • Verify both packages installed correctly by running:
python -c "import pandas; import sklearn; print(pandas.__version__, sklearn.__version__)"

You should see 3.0.4 1.9.0 printed in the terminal.

Seeing an import error?

Make sure (venv) is visible in your prompt. If it disappeared, re-activate the virtual environment before running the import check.

If pip failed earlier, check that you have an internet connection and try running the install command again.

help me fix my pandas or scikit-learn import error

Download and explore the MovieLens dataset

Your recommendation engine needs real movie data to learn from. The MovieLens dataset from GroupLens Research contains 100,836 ratings across 9,742 movies, collected from real users.

  • Open your browser and go to the MovieLens download page:

https://grouplens.org/datasets/movielens/latest/

  • Find the Small section on the page.
  • Click the ml-latest-small.zip link to download the file (about 1 MB).

The zip file saves to your Downloads folder.

  • Back in your VS Code terminal, create a data folder and extract the dataset into it by running:

macOS

mkdir data
unzip ~/Downloads/ml-latest-small.zip -d data/

Windows

mkdir data
Expand-Archive -Path ~\Downloads\ml-latest-small.zip -DestinationPath data\
  • Confirm the files extracted correctly by running:
ls data/ml-latest-small/

You should see movies.csv, ratings.csv, tags.csv, and links.csv listed.

Don't see the CSV files?

If you see a nested ml-latest-small folder inside another ml-latest-small folder, move the inner contents up one level so the path is data/ml-latest-small/movies.csv.

If the file did not download to ~/Downloads, check your browser's download location and adjust the path in the command.

help me find or extract the MovieLens dataset

Now you'll create the Python file that loads this data.

  • Create a new file called recommender.py in the root of your movie-recommender folder by clicking the new file icon in the VS Code Explorer sidebar.
  • Add the following code to load and inspect the dataset:
import pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity
from sklearn.neighbors import NearestNeighbors


# --- Data Loading ---

movies = pd.read_csv("data/ml-latest-small/movies.csv")
ratings = pd.read_csv("data/ml-latest-small/ratings.csv")

print(f"Movies: {movies.shape[0]} entries")
print(f"Ratings: {ratings.shape[0]} entries")
print()

What does this code do?

The first four lines import all the machine learning libraries you'll use throughout this project. You're importing them upfront so the file stays organized.

pd.read_csv() loads each CSV file into a DataFrame. A DataFrame is a table of rows and columns you can filter, sort, and transform with pandas.

The print statements show how many movies and ratings loaded, confirming the data is accessible.

Verify the full setup

Everything is in place. Before you run the script, think about what you expect to see based on the dataset size mentioned earlier.

  • Save recommender.py by pressing Cmd+S (macOS) or Ctrl+S (Windows).
  • Run the script:
python recommender.py

You should see output like this:

Movies: 9742 entries
Ratings: 100836 entries

What do these numbers mean?

The movies DataFrame has 9,742 rows. Each row is a movie with a movieId, title, and genres column.

The ratings DataFrame has 100,836 rows. Each row is a single rating from a user, with userId, movieId, rating, and timestamp columns.

Seeing a FileNotFoundError?

This means the CSV files aren't at the path data/ml-latest-small/movies.csv relative to where you're running the script.

Check that your terminal is in the movie-recommender folder (run pwd to confirm). Then check that ls data/ml-latest-small/ shows the CSV files.

help me fix a FileNotFoundError when loading my MovieLens data

✔️ Awesome, I've got everything!

Great. Double check you've saved recommender.py and your terminal shows both entry counts.

ⓧ I'd like to double check the full code

Here is the complete recommender.py at this point:

import pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity
from sklearn.neighbors import NearestNeighbors


# --- Data Loading ---

movies = pd.read_csv("data/ml-latest-small/movies.csv")
ratings = pd.read_csv("data/ml-latest-small/ratings.csv")

print(f"Movies: {movies.shape[0]} entries")
print(f"Ratings: {ratings.shape[0]} entries")
print()

Your environment is ready and your data is loaded. Next up, you'll build your first recommender that suggests similar movies based on genre.

Build Content-Based Recommendations

Your data is loaded and ready. Now it is time to build your first recommender and see actual movie suggestions appear in your terminal.

Content-based filtering is the simplest approach to recommendations. You describe each movie by its genres, measure how similar two movies are, and return the closest matches. The result is immediate and satisfying. Type a movie title, get five similar movies back.

In this step, get ready to:
  • Convert movie genres into numerical feature vectors using TF-IDF.
  • Build a recommendation function that finds the most similar movies by genre.
  • Test with real movies and observe the limitation of this approach.
Build genre-based feature vectors with TF-IDF

The genres column in movies.csv uses pipe-separated values like Adventure|Animation|Children|Comedy|Fantasy. To compare movies mathematically, you need to convert these text genres into numerical vectors.

What is TF-IDF?

TF-IDF (Term Frequency-Inverse Document Frequency) turns text into numbers. It assigns higher weights to genres that are distinctive for a movie and lower weights to genres that appear everywhere. This gives you a numerical fingerprint for each movie's genre profile.

  • In recommender.py, find the line print() at the bottom of your Data Loading section.
  • Add the following code below it to prepare genre features and compute similarity:


# --- Content-Based Filtering ---

# Prepare genre features: replace pipe separator with spaces for TF-IDF
movies["genres_clean"] = movies["genres"].str.replace("|", " ", regex=False)

# Build TF-IDF matrix from genres
tfidf = TfidfVectorizer(stop_words="english")
tfidf_matrix = tfidf.fit_transform(movies["genres_clean"])

# Compute cosine similarity between all movies
cosine_sim = cosine_similarity(tfidf_matrix, tfidf_matrix)

# Create a mapping of movie title to index
title_to_idx = pd.Series(movies.index, index=movies["title"]).drop_duplicates()

What does this code do?

  • movies["genres_clean"] replaces the pipe separators with spaces so TfidfVectorizer treats each genre as a separate word ("Adventure Animation Children" instead of "Adventure|Animation|Children").
  • tfidf_matrix holds the numerical representation of every movie's genre profile.
  • cosine_sim is a matrix where each cell tells you how similar two movies are (1.0 = identical genres, 0.0 = nothing in common).
  • title_to_idx lets you look up a movie's position in the matrix by its title string.

What is cosine similarity?

Cosine similarity measures the angle between two vectors. Two movies pointing in the same direction in "genre space" get a score close to 1.0, regardless of how many genres they have. It cares about the pattern of genres, not the count.

  • Save recommender.py.
  • Run the script to confirm the computation succeeds by running:
python recommender.py

You should see the same movie and ratings count output as before, with no errors. This confirms the TF-IDF vectorization and cosine similarity computation ran successfully.

Seeing a NameError or ImportError?

  • Check that your import lines at the top of the file include from sklearn.feature_extraction.text import TfidfVectorizer and from sklearn.metrics.pairwise import cosine_similarity. These were added in Step 1.
  • Make sure your virtual environment is activated. You should see (venv) at the start of your terminal prompt.

Why is my TF-IDF code throwing an error?

Write the recommendation function

You have a similarity matrix that knows how close every pair of movies is. Now you need a function that takes a movie title, looks up its row in that matrix, and returns the top matches.

  • Add the following function below the title_to_idx line you just wrote:


def recommend_content_based(title, n=5):
    """Recommend movies similar to the given title based on genre similarity."""
    if title not in title_to_idx:
        print(f"Movie '{title}' not found in dataset.")
        return []

    idx = title_to_idx[title]
    sim_scores = list(enumerate(cosine_sim[idx]))
    sim_scores = sorted(sim_scores, key=lambda x: x[1], reverse=True)
    sim_scores = sim_scores[1 : n + 1]  # Exclude the movie itself

    movie_indices = [i[0] for i in sim_scores]
    results = movies["title"].iloc[movie_indices].tolist()
    return results

What does this function do?

  • It first checks if the movie title exists in the dataset. If not, it prints a message and returns an empty list.
  • It looks up the movie's index, then grabs that row from the cosine similarity matrix. Each value in that row is a similarity score to another movie.
  • It sorts all movies by similarity score (highest first), skips position 0 (the movie itself, which always has score 1.0), and returns the top N titles.
  • Save recommender.py.
Test with multiple movies and observe the limitation

The function is ready. Now you will call it with two different movies and inspect what comes back.

  • Add the following test code at the bottom of recommender.py:


# Test content-based
print("=== Content-Based Recommendations ===")
print(f"\nMovies similar to 'Toy Story (1995)':")
for movie in recommend_content_based("Toy Story (1995)"):
    print(f"  - {movie}")

print(f"\nMovies similar to 'Matrix, The (1999)':")
for movie in recommend_content_based("Matrix, The (1999)"):
    print(f"  - {movie}")

print("\n[Notice: All recommendations share the same genres as the input!]")
print()

What does this test code do?

It calls your recommendation function twice with very different movies. "Toy Story (1995)" is a children/animation/comedy film. "Matrix, The (1999)" is an action/sci-fi/thriller. By comparing both sets of results, you will see a clear pattern emerge.

  • Save recommender.py.

Before you run the script, what do you expect the recommendations for "Toy Story" to look like? Will they span different genres, or stick to one kind of movie?

  • Run the script to see your first recommendations:
python recommender.py

You should see 5 recommended movies printed for each title. The Toy Story recommendations are all children/animation/comedy films. The Matrix recommendations are all action/sci-fi/thriller films.

Why is this a limitation?

Every recommendation shares the same genres as the input movie. The system can never surprise you with a cross-genre gem that you would actually love. A user who rated "The Matrix" highly might also enjoy "Inception" (a sci-fi/thriller), but they would never discover a drama or documentary that similar users adored.

This is the fundamental ceiling of content-based filtering. It only knows about item features, not about people. In the next step, you will build a recommender that learns from actual user behaviour.

Not seeing any recommendations?

  • Check the movie titles match exactly, including the year in parentheses. The dataset uses Matrix, The (1999) not The Matrix (1999).
  • If you see "Movie not found in dataset", the title string does not match a row in movies.csv. Double-check for extra spaces or missing commas.
  • If you see a NameError on cosine_sim or title_to_idx, make sure the function definition appears AFTER those variables are created.

Why are my content-based recommendations empty or erroring?

✔️ Awesome, I've got everything!

Great. Double check you have saved recommender.py and that your output shows recommendations for both movies.

ⓧ I'd like to double check the full code

Your recommender.py should contain these sections in order. Scroll up and confirm each one is present:

  • Imports at the top (pandas, TfidfVectorizer, cosine_similarity, NearestNeighbors).
  • Data Loading section: reads movies.csv and ratings.csv, prints shapes.
  • Content-Based Filtering section: creates genres_clean, builds tfidf_matrix, computes cosine_sim, creates title_to_idx.
  • The recommend_content_based(title, n=5) function.
  • Test code that prints recommendations for Toy Story and The Matrix, plus the genre limitation notice.

Your content-based recommender works, but it is trapped inside genre bubbles. In the next step, you will build a completely different approach that learns from actual user behaviour and can recommend movies across genre boundaries.

Build Collaborative Filtering

Your content-based recommender works, but there is a clear limitation. Every suggestion shares the same genres as the input movie. You can never discover a cross-genre gem this way.

Collaborative filtering solves this by finding users with similar taste and recommending what they enjoyed, regardless of genre. Instead of asking "what movies look like this one?" it asks "what did people like me enjoy?"

In this step, get ready to:
  • Build a user-item rating matrix from the MovieLens ratings data.
  • Find similar users and generate cross-genre recommendations.
  • Hit the cold-start wall and understand why hybrid approaches exist.
Build the user-item rating matrix

Collaborative filtering needs to compare users by their rating behaviour. To do this, you need a matrix where each row is a user, each column is a movie, and each cell is the rating that user gave that movie.

  • In recommender.py, scroll to the bottom of the file (below the content-based test output).
  • Add this code to create the rating matrix and fit a nearest-neighbours model:
# --- Collaborative Filtering ---

# Build user-item rating matrix
user_item_matrix = ratings.pivot_table(
    index="userId", columns="movieId", values="rating"
).fillna(0)

# Fit NearestNeighbors to find similar users
nn_model = NearestNeighbors(metric="cosine", algorithm="brute", n_neighbors=20)
nn_model.fit(user_item_matrix.values)

What does this code do?

pivot_table reshapes the ratings data so each user gets a row and each movie gets a column. The cell values are ratings. Missing values (movies the user never rated) are filled with 0.

NearestNeighbors with metric="cosine" measures how similar two users' rating patterns are. The model indexes all users so it can quickly find the closest matches for any given user.

  • Save recommender.py.
  • Run the script to confirm the new code processes without errors:
python recommender.py

You should see the same content-based recommendations as before, followed by no errors. The pivot table silently processed all 100,836 ratings into a user-movie matrix.

Seeing a MemoryError?

The pivot table for this dataset is around 600 users by 9,742 movies. If your machine is very low on memory, close other applications and try again. The matrix fits comfortably in under 100MB of RAM.

Help me debug a MemoryError when creating my user-item pivot table.

Find similar users and generate recommendations

Now you need a function that takes a user ID, finds the most similar users in the matrix, and recommends movies those similar users loved but the target user has not seen yet.

  • Below the nn_model.fit() line, add this function:
def recommend_collaborative(user_id, n=5):
    """Recommend movies based on what similar users enjoyed."""
    if user_id not in user_item_matrix.index:
        print(f"User {user_id} not found. Cannot generate collaborative recommendations.")
        return []

    user_idx = user_item_matrix.index.get_loc(user_id)
    user_vector = user_item_matrix.iloc[user_idx].values.reshape(1, -1)

    distances, indices = nn_model.kneighbors(user_vector, n_neighbors=11)

    # Get similar users (skip first one, which is the user themselves)
    similar_user_indices = indices.flatten()[1:]

    # Aggregate ratings from similar users
    similar_users_ratings = user_item_matrix.iloc[similar_user_indices]
    mean_ratings = similar_users_ratings.mean(axis=0)

    # Filter out movies the user has already rated
    user_rated = user_item_matrix.iloc[user_idx]
    already_rated = user_rated[user_rated > 0].index
    candidates = mean_ratings.drop(already_rated, errors="ignore")

    # Get top N recommendations
    top_movie_ids = candidates.nlargest(n).index.tolist()
    recommended_titles = movies[movies["movieId"].isin(top_movie_ids)]["title"].tolist()
    return recommended_titles

What does this function do?

  • First it checks whether the user exists in the matrix. If not, it prints an error and returns empty.
  • kneighbors finds the 11 nearest users (10 neighbours plus the user themselves). The first result is always the user, so it gets skipped.
  • It averages ratings across those 10 similar users, removes movies the target user already rated, and returns the top N highest-scoring titles.
  • Save recommender.py. You will test this function in the next substep.
Test and hit the cold-start wall

Time to see collaborative filtering in action. You will test it with a real user first, then push it to its breaking point.

  • Below the recommend_collaborative function, add this test code for User 1:
# Test collaborative filtering
print("=== Collaborative Filtering Recommendations ===")
print(f"\nRecommendations for User 1:")
for movie in recommend_collaborative(1):
    print(f"  - {movie}")
  • Save recommender.py and run it:
python recommender.py

Below the content-based output, you should now see 5 collaborative recommendations for User 1. Notice that these movies span different genres. The system found users who rate similarly to User 1 and surfaced what those users enjoyed.

Not seeing collaborative output?

  • Make sure the test code is OUTSIDE the function body (not indented under def recommend_collaborative).
  • Check that you saved the file before running it.
  • If you see User 1 not found, confirm that your user_item_matrix pivot table is defined above the function.

Help me debug why my collaborative filtering recommendations are not printing.

Now the real test. What do you think will happen if you ask for recommendations for a user who has never rated a single movie?

  • Add this code below the User 1 test to simulate a brand-new user:
# Test cold-start problem
print(f"\nRecommendations for User 9999 (new user, no ratings):")
recommend_collaborative(9999)
print()
  • Save recommender.py and run it:
python recommender.py

At the bottom of the output, you will see this message:

User 9999 not found. Cannot generate collaborative recommendations.

The system cannot find similar users because User 9999 has no rating history to compare against. No ratings means no vector, which means no neighbours.

What is the cold-start problem?

This is the cold-start problem. A brand-new user with zero interaction history is invisible to collaborative filtering. The system literally has nothing to work with.

Real products like Netflix and Spotify face this with every new signup. It is one of the primary reasons hybrid recommendation systems exist.

✔️ Awesome, I've got everything!

Great. Double check you have saved recommender.py and your script prints both collaborative recommendations for User 1 and the cold-start error for User 9999.

ⓧ I'd like to double check the full code

Your recommender.py should now contain these sections in order. Scroll through your file and confirm each is present:

  • Imports (pandas, TfidfVectorizer, cosine_similarity, NearestNeighbors).
  • Data loading (movies and ratings CSVs with shape prints).
  • Content-based filtering (genres_clean, tfidf_matrix, cosine_sim, title_to_idx, recommend_content_based function, test prints for Toy Story and The Matrix).
  • Collaborative filtering (user_item_matrix pivot table, nn_model fitting, recommend_collaborative function, test print for User 1, cold-start test for User 9999).

You now have two recommenders. One finds movies with matching genres. The other finds movies that similar users loved. But each has a gap. Content-based cannot surprise you, and collaborative cannot help new users. Next up, you will combine them into a hybrid engine that handles both.

Build the Hybrid Engine

In the previous step, you hit a wall. Collaborative filtering found cross-genre gems for existing users, but it completely crashed for anyone new.

Neither approach works alone. Content-based filtering can never surprise you with something outside your genres, and collaborative filtering cannot handle new users. The hybrid engine combines both with weighted scoring and a fallback for cold-start scenarios. This is what Netflix and Spotify actually use.

In this step, get ready to:
  • Combine content-based and collaborative scores with alpha weighting.
  • Add cold-start fallback logic that gracefully handles new users.
  • Compare all three approaches side-by-side for the same user.
Combine both recommenders with weighted scoring

The hybrid engine needs two helper functions before it can combine signals. One retrieves a user's top-rated movies (so content-based filtering knows what to use as seeds). The other provides a popularity-based fallback for brand-new users with no history.

  • In recommender.py, scroll to the bottom of the file (below the collaborative filtering test code).
  • Add the hybrid engine section comment and both helper functions by copying this code:
# --- Hybrid Engine ---


def get_user_top_movies(user_id, n=3):
    """Get the top-rated movies for a given user."""
    user_ratings = ratings[ratings["userId"] == user_id]
    top_rated = user_ratings.nlargest(n, "rating")
    top_movie_ids = top_rated["movieId"].tolist()
    top_titles = movies[movies["movieId"].isin(top_movie_ids)]["title"].tolist()
    return top_titles


def recommend_popularity(n=5, min_ratings=50):
    """Fallback: recommend popular movies with high average ratings."""
    movie_stats = ratings.groupby("movieId").agg(
        avg_rating=("rating", "mean"), num_ratings=("rating", "count")
    )
    popular = movie_stats[movie_stats["num_ratings"] >= min_ratings]
    top_popular = popular.nlargest(n, "avg_rating")
    top_movie_ids = top_popular.index.tolist()
    recommended_titles = movies[movies["movieId"].isin(top_movie_ids)]["title"].tolist()
    return recommended_titles

What do these helpers do?

  • get_user_top_movies looks up a user's highest-rated movies from the ratings data. The hybrid engine uses these as seeds for content-based recommendations.
  • recommend_popularity is the ultimate fallback. It finds movies with the highest average rating that have at least 50 ratings (to filter out obscure films with one perfect score).

Now for the main hybrid function. It starts by checking for cold-start conditions (new users with few or no ratings) and falls back gracefully before reaching the full combining logic.

  • Below the recommend_popularity function, add the first part of recommend_hybrid with its cold-start fallback branches:
def recommend_hybrid(user_id, n=5, alpha=0.6):
    """
    Hybrid recommendation engine.
    - alpha controls the weight of collaborative vs content-based (0.6 = 60% collaborative)
    - Falls back to content-based for users with few ratings
    - Falls back to popularity for users with no ratings
    """
    user_ratings = ratings[ratings["userId"] == user_id]
    num_user_ratings = len(user_ratings)

    # Cold-start: no ratings at all
    if num_user_ratings == 0:
        print(f"  [Fallback: popularity-based for new user {user_id}]")
        return recommend_popularity(n)

    # Cold-start: very few ratings, use content-based from their rated movies
    if num_user_ratings < 5:
        print(f"  [Fallback: content-based for user {user_id} ({num_user_ratings} ratings)]")
        top_movies = get_user_top_movies(user_id, n=2)
        all_recs = []
        for movie in top_movies:
            all_recs.extend(recommend_content_based(movie, n=n))
        # Remove duplicates and movies already rated
        rated_titles = movies[
            movies["movieId"].isin(user_ratings["movieId"])
        ]["title"].tolist()
        unique_recs = []
        for rec in all_recs:
            if rec not in unique_recs and rec not in rated_titles:
                unique_recs.append(rec)
        return unique_recs[:n]

What is the cold-start problem?

The cold-start problem occurs when a system has no data about a new user. Collaborative filtering relies on comparing user behaviour, so a user with zero or very few ratings has nothing to compare against.

The fallback logic handles two levels. Users with zero ratings get popularity-based recommendations (universally liked movies). Users with fewer than 5 ratings get content-based recommendations seeded from whatever they have rated.

  • Continue the recommend_hybrid function by adding the full hybrid combining logic directly below the cold-start branches (still inside the function, at the same indentation level):
    # Full hybrid: combine collaborative and content-based
    collab_recs = recommend_collaborative(user_id, n=n * 2)
    top_movies = get_user_top_movies(user_id, n=3)
    content_recs = []
    for movie in top_movies:
        content_recs.extend(recommend_content_based(movie, n=n))

    # Score and combine
    scores = {}
    for i, movie in enumerate(collab_recs):
        scores[movie] = scores.get(movie, 0) + alpha * (1 - i / len(collab_recs))
    for i, movie in enumerate(content_recs):
        scores[movie] = scores.get(movie, 0) + (1 - alpha) * (1 - i / len(content_recs))

    # Remove already-rated movies
    rated_titles = movies[
        movies["movieId"].isin(user_ratings["movieId"])
    ]["title"].tolist()
    for title in rated_titles:
        scores.pop(title, None)

    # Sort by score and return top N
    sorted_recs = sorted(scores.items(), key=lambda x: x[1], reverse=True)
    return [title for title, score in sorted_recs[:n]]

How does alpha weighting work?

The alpha=0.6 parameter gives 60% weight to collaborative scores and 40% to content-based scores. This favours actual user behaviour (what similar people enjoyed) while still incorporating genre similarity.

Each recommendation gets a score based on its rank position. Higher-ranked movies from either source score higher. Movies that appear in BOTH lists get combined scores, which is how the hybrid surfaces movies that are both genre-relevant and popular among similar users.

  • Below the recommend_hybrid function, add this test code to run the hybrid engine for User 1:
# Test hybrid engine
print("=== Hybrid Recommendations ===")
print(f"\nHybrid recommendations for User 1 (existing user):")
for movie in recommend_hybrid(1):
    print(f"  - {movie}")
  • Save recommender.py.
  • Run the script to verify the hybrid engine works for an existing user by running:
python recommender.py
  • Scroll to the end of the output. You should see === Hybrid Recommendations === followed by 5 movie recommendations for User 1.

Seeing a NameError or IndentationError?

  • Make sure the full hybrid logic (the # Full hybrid: section) is indented inside the function body, at the same level as the cold-start if blocks.
  • Check that get_user_top_movies and recommend_popularity are defined BEFORE recommend_hybrid since the hybrid function calls them.
  • Confirm you have no blank lines between the two halves of the function that would break Python's indentation parsing.

Why is my hybrid engine throwing an error?

Add cold-start fallback logic

You saw in the previous step that User 9999 crashed the collaborative engine. The cold-start branches you wrote handle this, but you have not tested them yet. Let's simulate a brand-new user with just one rating and prove the fallback works.

  • Below the hybrid test for User 1, add this code to simulate a cold-start user and test the fallback:
# Simulate cold-start user with 1 rating
cold_start_rating = pd.DataFrame(
    {"userId": [9999], "movieId": [2571], "rating": [5.0], "timestamp": [0]}
)
ratings = pd.concat([ratings, cold_start_rating], ignore_index=True)

print(f"\nHybrid recommendations for User 9999 (cold-start, 1 rating for 'Matrix, The (1999)'):")
for movie in recommend_hybrid(9999):
    print(f"  - {movie}")

print()

What does this simulation do?

You create a single fake rating for User 9999 (who previously had zero ratings). Movie ID 2571 is "Matrix, The (1999)". With only 1 rating, that is fewer than the threshold of 5, so the hybrid engine will detect this and fall back to content-based recommendations derived from The Matrix.

Instead of crashing like collaborative filtering did, the system gracefully degrades and still produces useful recommendations.

  • Save recommender.py and run the script again:
python recommender.py
  • At the end of the output, you should see the fallback message [Fallback: content-based for user 9999 (1 ratings)] followed by 5 movie recommendations that share genres with The Matrix.

Still seeing User 9999 crash?

  • Make sure the cold_start_rating and pd.concat lines come BEFORE the call to recommend_hybrid(9999). The rating must exist in the DataFrame before the function tries to look it up.
  • Verify the if num_user_ratings < 5: branch is present in your recommend_hybrid function. Without it, the function tries to run full collaborative filtering on a user with only 1 rating.

Why is my cold-start fallback not working for User 9999?

Compare all three approaches side-by-side

The real payoff is seeing how hybrid blends genre-similar movies with collaborative discoveries. Printing all three approaches for the same user makes the differences concrete.

  • At the bottom of recommender.py, add this side-by-side comparison code:
# --- Side-by-Side Comparison ---

print("=== Side-by-Side Comparison for User 1 ===")
print("\nContent-Based (from their top-rated movie):")
top_movie = get_user_top_movies(1, n=1)
if top_movie:
    for movie in recommend_content_based(top_movie[0]):
        print(f"  - {movie}")

print("\nCollaborative:")
for movie in recommend_collaborative(1):
    print(f"  - {movie}")

print("\nHybrid:")
for movie in recommend_hybrid(1):
    print(f"  - {movie}")

What does this comparison reveal?

  • Content-based shows movies from the same genres as User 1's favourite. These are safe but predictable.
  • Collaborative shows what users with similar taste enjoyed. These might cross genre boundaries.
  • Hybrid blends both signals. Movies that appear in both lists score highest, giving you the best of both worlds.
  • Save recommender.py.

Before you run this, what do you expect the hybrid list to look like compared to the other two? Will it match one more closely, or contain movies from both?

  • Run the complete script end-to-end:
python recommender.py
  • At the very end of the output, you should see === Side-by-Side Comparison for User 1 === with three labelled lists. The hybrid list draws from both sources, with some overlap and some unique entries from each.

Seeing empty output for the comparison?

  • Confirm that get_user_top_movies(1, n=1) returns a non-empty list. If User 1 has no ratings (unlikely but possible if data loading failed earlier), you would see no content-based results.
  • Make sure the comparison code sits AFTER the cold-start simulation block. The ratings DataFrame must already be loaded for the functions to work.

Why is my side-by-side comparison output empty or missing?

✔️ Awesome, I've got everything!

Great. Double check you have saved recommender.py and that your output shows all three sections: hybrid for User 1, hybrid fallback for User 9999, and the side-by-side comparison.

ⓧ I'd like to double check the full code

Your recommender.py is now quite long. Scroll through and confirm you have these sections in order:

  • # --- Data Loading --- (imports, read CSVs, print shapes).
  • # --- Content-Based Filtering --- (TF-IDF, cosine_sim, recommend_content_based, test prints).
  • # --- Collaborative Filtering --- (pivot table, NearestNeighbors, recommend_collaborative, test prints including User 9999 failure).
  • # --- Hybrid Engine --- (get_user_top_movies, recommend_popularity, recommend_hybrid, hybrid tests, cold-start simulation).
  • # --- Side-by-Side Comparison --- (content-based, collaborative, and hybrid printed for User 1).

You now have a working hybrid recommendation engine that demonstrates the architecture behind Netflix and Spotify. It combines content features for similarity, user behaviour for personalization, and intelligent fallbacks for robustness. Next up, the Secret Mission challenges you to measure which approach actually wins with a precision@K evaluation.

Secret mission

Add Precision@K Evaluation

You have three recommenders, but which one actually performs best? Build a precision@K evaluation pipeline to find out with real data.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • The movie-recommender project folder on your Desktop (contains your Python scripts, virtual environment, and MovieLens dataset).
  • The venv virtual environment with pandas and scikit-learn installed.
  • The downloaded MovieLens dataset inside data/ml-latest-small/.

Keep everything running

No action needed. Choose this if you are still actively building or want to experiment with different alpha weights, new datasets, or additional recommendation techniques.

  • Your movie-recommender folder stays on your Desktop with all files intact.
  • To resume work later, open the folder in VS Code, activate the virtual environment, and run your scripts.

Pause - I'll come back to this later

Deactivate the virtual environment to free up your terminal session. Your files stay on disk and are ready to pick back up anytime.

  • In your terminal, deactivate the virtual environment by running:
deactivate
  • When you want to resume, navigate back to your project folder and reactivate the virtual environment.

Delete - I don't want to use this again

Remove all project files, the virtual environment, and the downloaded dataset. This frees up disk space and leaves no trace of the project.

  • Deactivate the virtual environment first (if it is still active):
deactivate
  • Navigate to your Desktop and delete the entire project folder by running:
cd ~/Desktop
rm -rf movie-recommender
  • On Windows, use this command instead:
cd ~\Desktop
rmdir /s /q movie-recommender
  • Confirm the folder is gone by running ls on your Desktop. You should no longer see movie-recommender in the output.

That's a Wrap!

That's a Wrap!

Nice work! You just built a hybrid movie recommendation engine that combines content-based filtering, collaborative filtering, and cold-start fallback logic into a single intelligent system.

You've learned how to:

  • Build a content-based recommender using TF-IDF vectorization and cosine similarity on movie genres.
  • Build a collaborative filtering recommender using NearestNeighbors to find users with similar taste and surface cross-genre discoveries.
  • Experience the cold-start problem firsthand and solve it with a hybrid architecture that combines weighted scoring with graceful fallback logic.
  • Secret Mission: evaluate all three approaches with precision@5 to prove the hybrid outperforms either individual approach.

Ready to quiz yourself?