Fashion Product Classifier with Scikit-learn

Build an image classifier that predicts clothing categories using PCA and pipelines.

Introduction

30 Second Summary

When you browse an online store, every product photo is automatically tagged with its category. Behind that simple label is a machine learning model that learned to tell a sneaker from a sandal by studying thousands of product images.

In this project, you will build a fashion product classifier that predicts clothing categories from pixel data. You'll compress images to make training faster, then compare multiple ML algorithms side-by-side to find the best performer.

What You'll Build

You'll feed a grayscale product image into your trained pipeline and instantly get back the predicted category, whether it's a T-shirt, Sneaker, Bag, or one of seven other clothing types.

By the end of this project, you'll have:

  • A working classification pipeline that predicts 10 clothing categories from 28x28 grayscale product images trained on the Fashion-MNIST benchmark dataset.
  • Visual proof of PCA compression showing original vs reconstructed images, and a clear picture of the speed vs accuracy trade-off.
  • A side-by-side model comparison of three ML algorithms using cross-validation scores, plus a confusion matrix for the winner.
  • Secret Mission: Use GridSearchCV to automatically find the best combination of compression level and model settings.

Are there any prerequisites?

This project assumes you're comfortable with Python fundamentals and basic object-oriented programming. Familiarity with NumPy, Pandas, and Matplotlib will help you follow the data handling and visualization steps.

You'll work entirely in Google Colab, which comes with Scikit-learn and all other libraries pre-installed. No local setup required.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters. This quick checkpoint helps you set a clear goal before any hands-on coding begins.

Set Up and Load the Data

Before you can train a model to recognize clothing items, you need data to learn from. Fashion-MNIST is a benchmark dataset containing 70,000 grayscale images of real fashion products across 10 categories.

In this step, you'll load the entire dataset into a Google Colab notebook, split it into smaller training and test subsets for speed, and normalize the pixel values so they're ready for machine learning.

In this step, get ready to:
  • Verify scikit-learn is ready in your Colab environment.
  • Load 70,000 fashion product images from Fashion-MNIST.
  • Split and normalize the data for training.
Create a new Colab notebook and check scikit-learn

Google Colab comes with scikit-learn pre-installed. A quick version check confirms everything is ready before you write any real code.

  • In Google Colab, click File in the top menu bar.
  • Click New notebook.
  • In the first code cell, verify scikit-learn is installed by running this code:
import sklearn
print(sklearn.__version__)

You should see a version number printed below the cell, such as 1.6.1.

Not seeing a version number?

If you see an error, try restarting the runtime. Click Runtime in the menu, then Restart runtime, and run the cell again.

Scikit-learn is always pre-installed in Colab, so a restart usually fixes import issues. Why can't I import sklearn in Colab?

Import libraries and load Fashion-MNIST

Now you'll set up the main code cell for this project. This cell imports every library you'll use throughout the project, downloads the dataset, and prepares it for training.

  • Click the + Code button to create a new code cell below your version check.
  • Paste the following imports into the new cell:
# Cell 1: Setup and Load Data
import numpy as np
import matplotlib.pyplot as plt
import time

from sklearn.datasets import fetch_openml
from sklearn.model_selection import train_test_split, cross_val_score, GridSearchCV
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.pipeline import make_pipeline
from sklearn.metrics import accuracy_score, classification_report, ConfusionMatrixDisplay

What are all these imports for?

  • numpy handles numerical arrays. matplotlib draws charts and images. time measures training speed.
  • fetch_openml downloads the Fashion-MNIST dataset directly from OpenML.
  • The sklearn modules cover everything from splitting data to training models to measuring accuracy.
  • You're importing them all now so every future cell can use them without re-importing.
  • Now add the following code below the imports in the same cell to load and prepare the data:
# Load Fashion-MNIST from OpenML
X, y = fetch_openml(data_id=40996, return_X_y=True, as_frame=False)
y = y.astype(int)

# Class names for the 10 fashion categories
class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',
               'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']

# Split into train/test subsets for speed
X_train, X_test, y_train, y_test = train_test_split(
    X, y, train_size=10000, test_size=2000, random_state=42, stratify=y
)

# Normalize pixel values to 0-1 range
X_train = X_train / 255.0
X_test = X_test / 255.0

print(f"Training set: {X_train.shape}")
print(f"Test set: {X_test.shape}")
print(f"Number of classes: {len(np.unique(y_train))}")

What does this code do?

  • fetch_openml(data_id=40996) downloads all 70,000 Fashion-MNIST images as a flat array of 784 pixel values each (28x28 images flattened into rows).
  • class_names maps the numeric labels (0-9) to human-readable product categories like T-shirt/top or Sneaker.
  • train_test_split takes a 10,000-image training subset and a 2,000-image test subset. Using the full 70,000 would make later steps too slow. The stratify=y argument keeps the class distribution balanced in both subsets.
  • Dividing by 255.0 scales raw pixel values (0-255) down to a 0-1 range, which helps ML algorithms converge faster.
Confirm the data loaded correctly

Before you run the cell, think about what you expect to see. You asked for 10,000 training samples and 2,000 test samples, each with 784 pixel features.

  • Click the play button (or press Shift+Enter) to run the cell.

What should I see?

The download may take 10-30 seconds the first time. Once it finishes, you should see three lines of output:

Training set: (10000, 784)
Test set: (2000, 784)
Number of classes: 10

This confirms 10,000 training images, 2,000 test images, each with 784 pixel features, spread across 10 product categories.

Seeing a download error or timeout?

The fetch_openml call downloads data from the internet. If you see a connection error, try running the cell again. Colab's network can occasionally hiccup on the first attempt.

If the cell seems stuck for more than 60 seconds, click Runtime then Restart runtime, and re-run both cells from the top. Help me fix my Fashion-MNIST download error in Colab.

✔️ Awesome, I've got everything!

Great. Double check you've saved your notebook (Ctrl+S or Cmd+S) and move on.

ⓧ I'd like to double check the full code

Your Cell 1 should contain these two sections in order:

  • The import block (lines 1-14): numpy, matplotlib, time, and all the sklearn modules.
  • The load-and-prepare block (lines 16-33): fetch_openml call, class_names list, train_test_split, normalization, and three print statements.

Scroll up and confirm both sections are present in a single cell, with no code between them except a blank line.

Your dataset is loaded and ready for exploration. Next up, you'll visualize the actual product images to see what 784 pixels look like as clothing items.

Explore and Visualize Product Images

Your data is loaded, but right now it's just a matrix of 10,000 rows and 784 columns. Each row is a product image flattened into a single line of pixel values.

Before training a model, you need to see what these numbers actually look like as images. Visualizing the data builds intuition about which product categories are easy to tell apart and which ones will challenge your classifier.

In this step, get ready to:
  • Display a grid of product images with their category labels.
  • Plot the class distribution to confirm a balanced dataset.
  • Compare visually similar product pairs side by side.
Display one sample from each product category

Each image in the dataset is stored as a flat array of 784 numbers. To see the actual product, you need to reshape those numbers back into a 28x28 grid of pixels.

  • Create a new code cell below your Cell 1 and paste this code to display one sample image from each of the 10 product classes:
# Cell 2: Explore and Visualize Product Images
fig, axes = plt.subplots(2, 5, figsize=(12, 5))
for i, ax in enumerate(axes.flat):
    # Find first sample of each class
    idx = np.where(y_train == i)[0][0]
    ax.imshow(X_train[idx].reshape(28, 28), cmap='gray')
    ax.set_title(class_names[i], fontsize=10)
    ax.axis('off')
plt.suptitle('One Sample Per Product Category', fontsize=14)
plt.tight_layout()
plt.show()

What does this code do?

This code creates a 2x5 grid of subplots and loops through all 10 classes (0 through 9).

  • np.where(y_train == i)[0][0] finds the index of the first training sample belonging to class i.
  • .reshape(28, 28) transforms the flat 784-element array back into its original 28x28 pixel grid so imshow can display it as an image.
  • cmap='gray' renders the image in grayscale, matching how the product photos were originally captured.
  • Run the cell by pressing Shift+Enter.

You should see a 2x5 grid of grayscale product images, each labeled with its category name (T-shirt/top, Trouser, Pullover, etc.).

Not seeing any images?

  • Make sure you ran Cell 1 first. This cell depends on X_train, y_train, and class_names from the previous cell.
  • Check that plt.show() is at the end of the cell. Without it, Colab may not render the plot inline.
  • Help me debug my image grid
Plot the class distribution

A balanced dataset means each category has roughly the same number of training samples. If one class had far fewer examples, the model might learn to ignore it.

  • Create a new code cell and paste this code to visualize how many samples each class has:
# Class distribution
unique, counts = np.unique(y_train, return_counts=True)
plt.figure(figsize=(10, 4))
plt.bar([class_names[i] for i in unique], counts)
plt.xticks(rotation=45, ha='right')
plt.ylabel('Count')
plt.title('Training Set Class Distribution')
plt.tight_layout()
plt.show()

What does this code do?

  • np.unique(y_train, return_counts=True) counts how many training samples belong to each class.
  • The bar chart maps each class integer to its human-readable name using the class_names list from Cell 1.
  • Because you used stratify=y in the train/test split, each class should have roughly 1,000 samples in the 10,000-sample training set.
  • Run the cell with Shift+Enter.

You should see a bar chart with all 10 categories at roughly the same height (around 1,000 samples each). This confirms the dataset is balanced.

Bars look uneven or labels overlap?

  • If labels overlap, check that the rotation=45 and ha='right' arguments are in your plt.xticks() call.
  • Slight variation in bar heights (e.g. 998 vs 1002) is normal. The stratified split keeps things close but not perfectly equal.
  • Help me fix my bar chart
Compare visually similar product categories

Some product categories look almost identical to the human eye. If you can barely tell them apart, your classifier will struggle with them too.

Before you run the next cell, think about which categories you expect a model to confuse. Look back at your sample grid. Which two pairs look the most alike?

  • Create a new code cell and paste this code to compare confusable product pairs side by side:
# Similar items comparison
fig, axes = plt.subplots(2, 4, figsize=(10, 5))
similar_pairs = [(0, 6), (5, 9)]  # T-shirt vs Shirt, Sandal vs Ankle boot
for row, (class_a, class_b) in enumerate(similar_pairs):
    for col in range(2):
        idx_a = np.where(y_train == class_a)[0][col]
        idx_b = np.where(y_train == class_b)[0][col]
        axes[row, col].imshow(X_train[idx_a].reshape(28, 28), cmap='gray')
        axes[row, col].set_title(class_names[class_a], fontsize=9)
        axes[row, col].axis('off')
        axes[row, col + 2].imshow(X_train[idx_b].reshape(28, 28), cmap='gray')
        axes[row, col + 2].set_title(class_names[class_b], fontsize=9)
        axes[row, col + 2].axis('off')
plt.suptitle('Visually Similar Categories', fontsize=14)
plt.tight_layout()
plt.show()

What does this code do?

  • similar_pairs = [(0, 6), (5, 9)] pairs up T-shirt/top (class 0) with Shirt (class 6), and Sandal (class 5) with Ankle boot (class 9).
  • For each pair, the code pulls two examples of each class and displays them side by side in a 2x4 grid. The top row compares T-shirts to Shirts. The bottom row compares Sandals to Ankle boots.
  • This gives you a visual preview of where the classifier will likely make mistakes.
  • Run the cell with Shift+Enter.

You should see a 2x4 grid. Notice how T-shirts and Shirts have nearly identical silhouettes. Sandals and Ankle boots share similar shapes too, just with different coverage.

Seeing blank subplots or errors?

  • Check that similar_pairs uses tuples inside the list: [(0, 6), (5, 9)]. Missing parentheses will cause an unpacking error.
  • Ensure Cell 1 has been run so that X_train and y_train are in memory.
  • Help me debug my comparison grid

Why does visual similarity matter for classification?

A classifier works by finding pixel patterns that distinguish one class from another. When two classes look nearly identical (same shape, same brightness distribution), the model has fewer distinguishing pixels to work with.

In Step 3, you will see these exact pairs (T-shirt vs Shirt, Sandal vs Ankle boot) showing up as the most confused categories in the classification report.

✔️ Awesome, I've got everything!

Great. Make sure all three cells are saved in your notebook.

ⓧ I'd like to double check the full code

Scroll through your notebook and confirm these three code sections are present in order, each in its own cell:

  • The sample grid cell starting with # Cell 2: Explore and Visualize Product Images that creates a 2x5 plt.subplots grid.
  • The class distribution cell starting with # Class distribution that uses np.unique and plt.bar.
  • The similar items cell starting with # Similar items comparison that creates a 2x4 grid with similar_pairs.

You now have a clear picture of your data. You can see what 784 pixels look like as real products, you know the classes are balanced, and you have identified the tricky pairs. Next up, you will train your first classifier and see how well it handles these categories.

Train Your First Classifier

You've explored the product images and seen how different categories look. Now it's time to teach a model to tell them apart automatically.

The fastest way to learn an ML algorithm is to use it. Logistic Regression finds a decision boundary between classes, and scikit-learn lets you train it in two lines. You'll time the training to see how long 784 raw pixel features takes.

In this step, get ready to:
  • Train a Logistic Regression model on 10,000 product images and measure its speed.
  • Visualize predictions on test images to see which products the model gets right and wrong.
  • Print a per-category performance report to identify which product types are hardest to classify.
Create and train a Logistic Regression model

Logistic Regression works by finding a linear boundary between each pair of classes. Think of it as drawing lines that separate T-shirts from Trousers, Sneakers from Bags, and so on.

  • Create a new code cell in your notebook and add the following code to train the model and measure how long it takes:
# Cell 3: Train Your First Classifier
clf = LogisticRegression(max_iter=1000, random_state=42)

start_time = time.time()
clf.fit(X_train, y_train)
train_time = time.time() - start_time

y_pred = clf.predict(X_test)
accuracy = accuracy_score(y_test, y_pred)

print(f"Training time: {train_time:.2f} seconds")
print(f"Test accuracy: {accuracy:.4f}")

What does this code do?

  • LogisticRegression(max_iter=1000, random_state=42) creates the model. max_iter=1000 gives it enough iterations to converge on a solution.
  • time.time() records timestamps before and after training so you can see exactly how many seconds fit() takes.
  • clf.predict(X_test) feeds 2,000 unseen test images through the trained model to get predicted labels.
  • accuracy_score() compares predictions to the true labels and returns the fraction of correct answers.

Before you run this cell, take a guess. How accurate do you think the model will be on 10 product categories using raw pixel values?

  • Run the cell.

You should see the training time printed (likely 10-20 seconds) and a test accuracy around 84%.

Why is training so slow?

The model is crunching through all 784 pixel features for every image. Most of those pixels are just black background and carry no useful information. That's wasted computation you'll fix in the next step with PCA.

Seeing a convergence warning?

If the output shows a ConvergenceWarning, the model didn't fully converge within 1000 iterations. This still produces usable results, but check that you typed max_iter=1000 and not a smaller number.

help me debug a convergence warning in my Logistic Regression model

Visualize predictions on test images

An accuracy number alone doesn't tell you what the model gets wrong. Plotting predicted vs actual labels on real images reveals the patterns behind the mistakes.

  • Add this code below the print statements in the same cell to display a grid of 10 test images with their predictions:
# Show predictions
fig, axes = plt.subplots(2, 5, figsize=(12, 5))
for i, ax in enumerate(axes.flat):
    ax.imshow(X_test[i].reshape(28, 28), cmap='gray')
    pred_label = class_names[y_pred[i]]
    true_label = class_names[y_test[i]]
    color = 'green' if y_pred[i] == y_test[i] else 'red'
    ax.set_title(f"Pred: {pred_label}\nTrue: {true_label}", fontsize=8, color=color)
    ax.axis('off')
plt.suptitle('Predictions on Test Images', fontsize=14)
plt.tight_layout()
plt.show()

What does this visualization show?

  • Each image shows the predicted label on top and the true label below it.
  • Green titles mean the model got it right. Red titles mean it made a mistake.
  • The reshape back to 28x28 converts the flat 784-value array into a viewable image.
  • Run the cell again. You should see a 2x5 grid of product images with color-coded prediction labels.

All titles showing in black?

Check that the color variable uses the comparison y_pred[i] == y_test[i] (double equals). A single equals sign would cause an assignment error.

help me fix my prediction visualization colors

Check per-category performance

Overall accuracy hides important details. A model might nail Trousers (100%) but struggle with Shirts (60%). The classification report breaks accuracy down per category.

  • Add these lines at the bottom of the same cell to print the full classification report:
# Classification report
print("\nClassification Report:")
print(classification_report(y_test, y_pred, target_names=class_names))

What is a classification report?

The report shows three key metrics per category. Precision measures how many items predicted as that category actually belonged to it. Recall measures how many items of that category were correctly found. F1-score combines both into a single number.

  • Run the cell. Look at the report output and find the category with the lowest f1-score.

You should see that Shirt has the lowest score. The model confuses it with T-shirt/top and Coat because they share similar silhouettes.

Seeing a NameError for classification_report?

Make sure your Cell 1 import includes from sklearn.metrics import accuracy_score, classification_report, ConfusionMatrixDisplay. If you skipped this import, add it and re-run Cell 1 before re-running Cell 3.

help me fix a NameError in my classification report

✔️ Awesome, I've got everything!

Nice work. Make sure your Cell 3 is saved and all three outputs are visible: training time, the prediction grid, and the classification report.

ⓧ I'd like to double check the full code

Here is the complete Cell 3 code. Make sure yours matches:

# Cell 3: Train Your First Classifier
clf = LogisticRegression(max_iter=1000, random_state=42)

start_time = time.time()
clf.fit(X_train, y_train)
train_time = time.time() - start_time

y_pred = clf.predict(X_test)
accuracy = accuracy_score(y_test, y_pred)

print(f"Training time: {train_time:.2f} seconds")
print(f"Test accuracy: {accuracy:.4f}")

fig, axes = plt.subplots(2, 5, figsize=(12, 5))
for i, ax in enumerate(axes.flat):
    ax.imshow(X_test[i].reshape(28, 28), cmap='gray')
    pred_label = class_names[y_pred[i]]
    true_label = class_names[y_test[i]]
    color = 'green' if y_pred[i] == y_test[i] else 'red'
    ax.set_title(f"Pred: {pred_label}\nTrue: {true_label}", fontsize=8, color=color)
    ax.axis('off')
plt.suptitle('Predictions on Test Images', fontsize=14)
plt.tight_layout()
plt.show()

print("\nClassification Report:")
print(classification_report(y_test, y_pred, target_names=class_names))

Your first classifier is trained and scoring around 84% accuracy. But did you notice how long training took? All 784 pixel features slowed things down, and most of those pixels were just empty background. Next up, you'll compress the images with PCA to train faster without losing much accuracy.

Compress Images with PCA

In the last step, you trained a Logistic Regression model on all 784 pixel features. It worked, but training took over 10 seconds. That's because most of those 784 pixels are background (always zero) or redundant neighbors carrying the same information.

PCA (Principal Component Analysis) solves this by finding the most important patterns in the data and compressing everything down to far fewer features. This makes training faster and can even improve accuracy by stripping away noise.

In this step, get ready to:
  • Fit PCA on your training data and visualize how many components capture most of the information.
  • Compress the images to 50 components and retrain Logistic Regression to see the speed and accuracy trade-off.
  • Reconstruct compressed images to see what PCA keeps and what it throws away.
Fit PCA and plot the explained variance

Before picking a specific number of components, you need to know how much information each component captures. Fitting PCA with all possible components and plotting the cumulative explained variance tells you exactly where the "useful information" starts to plateau.

  • Create a new code cell below your Cell 3 code. Add the following code to fit PCA on the training data and plot the explained variance:
# Cell 4: Compress Images with PCA
pca_full = PCA(random_state=42)
pca_full.fit(X_train)

# Plot cumulative explained variance
cumulative_variance = np.cumsum(pca_full.explained_variance_ratio_)
plt.figure(figsize=(10, 5))
plt.plot(cumulative_variance, linewidth=2)
plt.axhline(y=0.80, color='r', linestyle='--', label='80% variance')
plt.axhline(y=0.95, color='g', linestyle='--', label='95% variance')
plt.xlabel('Number of Components')
plt.ylabel('Cumulative Explained Variance')
plt.title('PCA: How Many Components Do We Need?')
plt.legend()
plt.grid(True, alpha=0.3)
plt.show()

n_80 = np.argmax(cumulative_variance >= 0.80) + 1
n_95 = np.argmax(cumulative_variance >= 0.95) + 1
print(f"Components for 80% variance: {n_80}")
print(f"Components for 95% variance: {n_95}")

What does this code do?

PCA(random_state=42) without specifying n_components computes ALL possible components. This lets you inspect how much variance each one explains before committing to a number.

explained_variance_ratio_ is an array where each entry tells you what fraction of the total information that component captures. Summing them cumulatively shows the running total.

  • Run the cell.

You should see a curve that rises steeply at first, then flattens. The red dashed line at 80% is crossed at roughly 50 components. The output also prints the exact component counts for 80% and 95% variance thresholds.

Why does the curve flatten?

The first few components capture the big, obvious patterns (overall shape, orientation). Later components capture increasingly tiny details. Most of those details are noise from individual pixels, not useful signal for classification.

Not seeing the plot?

Make sure you ran all previous cells first (Cells 1-3). PCA needs X_train to be defined and normalized.

Help me debug my PCA explained variance plot

Compress to 50 components and retrain

The variance plot shows that 50 components capture over 80% of the information. Now you can compress the data and retrain the same Logistic Regression model to see if the speed improves without sacrificing much accuracy.

  • Add this code in the same cell, directly below the print statements you just wrote:
# Compress to 50 components and retrain
pca = PCA(n_components=50, random_state=42)
X_train_pca = pca.fit_transform(X_train)
X_test_pca = pca.transform(X_test)

clf_pca = LogisticRegression(max_iter=1000, random_state=42)

start_time = time.time()
clf_pca.fit(X_train_pca, y_train)
train_time_pca = time.time() - start_time

accuracy_pca = clf_pca.score(X_test_pca, y_test)

print(f"\nWith PCA (50 components):")
print(f"Training time: {train_time_pca:.2f} seconds")
print(f"Test accuracy: {accuracy_pca:.4f}")
print(f"\nSpeedup: {train_time / train_time_pca:.1f}x faster")

What does this code do?

  • fit_transform(X_train) learns the 50 most important patterns from the training data and compresses it in one step. The result is a (10000, 50) array instead of (10000, 784).
  • transform(X_test) applies the same compression to test data using the patterns learned from training. This prevents data leakage.
  • The speedup ratio compares the original train_time from Step 3 against the new compressed training time.
  • Before you run this, think about it. Do you expect the compressed model to be faster, slower, or the same speed as the original 784-feature model?
  • Run the cell.

You should see a significant speedup (several times faster) and accuracy that is close to the original. The model trained on 50 features instead of 784, so it finished much faster. The accuracy stays similar because those 50 components contain over 80% of the useful information.

Seeing a NameError for train_time?

The variable train_time was defined in Cell 3 when you timed the original training. Go back and re-run Cell 3 first so the variable exists in memory.

Help me fix my PCA retraining code

Visualize what PCA keeps and loses

Numbers tell you the accuracy stayed close. But what does "losing 20% of variance" actually look like? You can reverse the compression using inverse_transform to reconstruct the images from only 50 components and compare them visually to the originals.

  • Add this code at the bottom of your Cell 4 to reconstruct and display the compressed images:
# Visualize reconstruction
X_reconstructed = pca.inverse_transform(X_test_pca)

fig, axes = plt.subplots(2, 5, figsize=(12, 5))
for i in range(5):
    axes[0, i].imshow(X_test[i].reshape(28, 28), cmap='gray')
    axes[0, i].set_title('Original', fontsize=9)
    axes[0, i].axis('off')
    axes[1, i].imshow(X_reconstructed[i].reshape(28, 28), cmap='gray')
    axes[1, i].set_title('50 Components', fontsize=9)
    axes[1, i].axis('off')
plt.suptitle('Original vs PCA-Reconstructed Images', fontsize=14)
plt.tight_layout()
plt.show()

What does this code do?

inverse_transform takes the compressed 50-feature representation and expands it back to 784 pixels. The reconstructed image is slightly blurry because the fine details (the other 20% of variance) are gone. The overall shape and category are still clearly recognizable.

  • Run the cell.

You should see a 2-row grid. The top row shows original images and the bottom row shows their reconstructions from 50 components. The reconstructed images look slightly softer but the shapes are clearly recognizable. That's why the model still classifies them accurately.

Reconstructed images look completely garbled?

Make sure you used pca.inverse_transform(X_test_pca) and not pca_full.inverse_transform(...). The pca object (50 components) matches the shape of X_test_pca.

Help me debug my PCA image reconstruction

✔️ Awesome, I've got everything!

Great. Make sure Cell 4 is saved and all three outputs are visible: the variance plot, the accuracy/speedup printout, and the reconstruction grid.

ⓧ I'd like to double check the full code

Your Cell 4 should contain three sections, in this order:

  • The PCA full fit and explained variance plot (starting with pca_full = PCA(random_state=42), ending with the two print statements for n_80 and n_95).
  • The 50-component compression and retraining (starting with pca = PCA(n_components=50, random_state=42), ending with the speedup print statement).
  • The reconstruction visualization (starting with X_reconstructed = pca.inverse_transform(X_test_pca), ending with plt.show()).

Scroll up to each chunk above and confirm your cell matches. All three sections should be in a single cell.

You just compressed 784 features into 50, trained a model several times faster, and barely lost any accuracy. Next up, you'll wrap this PCA-then-model pattern into a Pipeline and race three different algorithms against each other.

Build a Pipeline and Compare Models

In the last step, you proved that PCA compresses 784 features down to 50 while keeping most of the image information intact.

But there was a hidden problem with how you did it. You fitted PCA on the training data separately, then passed the result to the model. If you ever forget to transform the test data the same way, you get garbage predictions. A Pipeline chains preprocessing and modeling into a single object, so the steps always run in the correct order.

In this step, get ready to:
  • Create a Pipeline that chains PCA and a classifier into one unit.
  • Run three different algorithms on the same data using cross-validation.
  • Visualize which model wins and where it still makes mistakes.
Create your first Pipeline

In Step 4, you manually applied PCA, then passed the compressed arrays to Logistic Regression. That works, but it means you have to remember to call pca.transform() on every new dataset.

A Pipeline bundles the preprocessing step and the model into a single object. You call fit() once and it handles everything in order.

  • Create a new code cell in your notebook and add the following Pipeline:
# Cell 5: Build a Pipeline and Compare Models
pipe_lr = make_pipeline(
    PCA(n_components=50, random_state=42),
    LogisticRegression(max_iter=1000, random_state=42)
)

What does this code do?

The make_pipeline function chains steps together in order. When you call pipe_lr.fit(X_train, y_train), it automatically fits PCA on the training data, transforms it, then trains Logistic Regression on the compressed result.

When you call pipe_lr.score(X_test, y_test), it transforms the test data with the same PCA and scores predictions. No manual step-tracking needed.

Why Pipelines instead of manual steps?

The manual approach (fit PCA separately, then feed results to a model) risks data leakage. If you accidentally fit PCA on your test set, your accuracy looks better than it really is. A Pipeline guarantees that each step only sees the data it should.

Pipelines also let you swap models in one line. You can replace LogisticRegression() with any classifier and the PCA step stays the same.

Race three models with cross-validation

One model is not enough to know you picked the best algorithm. Cross-validation splits the training data into 5 folds, trains on 4, and tests on the 1 left out. It repeats this 5 times so every sample gets tested once.

  • Add the following code below your pipe_lr definition. This creates two more Pipelines and runs all three through cross-validation:
# Pipeline 2: PCA + Random Forest
pipe_rf = make_pipeline(
    PCA(n_components=50, random_state=42),
    RandomForestClassifier(n_estimators=100, random_state=42)
)

# Pipeline 3: PCA + KNN
pipe_knn = make_pipeline(
    PCA(n_components=50, random_state=42),
    KNeighborsClassifier(n_neighbors=5)
)

models = {
    'Logistic Regression': pipe_lr,
    'Random Forest': pipe_rf,
    'KNN': pipe_knn
}

results = {}
for name, pipe in models.items():
    scores = cross_val_score(pipe, X_train, y_train, cv=5, scoring='accuracy')
    results[name] = {'mean': scores.mean(), 'std': scores.std()}
    print(f"{name}: {scores.mean():.4f} (+/- {scores.std():.4f})")

What does this code do?

You create two more Pipelines with different classifiers. Random Forest builds 100 decision trees and takes a majority vote. KNN (K-Nearest Neighbors) finds the 5 most similar training images and picks the most common label among them.

The cross_val_score function runs 5-fold cross-validation on each Pipeline. It returns 5 accuracy scores, one per fold. You store the mean and standard deviation so you can compare fairly.

  • Run the cell by pressing Shift+Enter.

This takes 20-40 seconds because cross-validation trains each model 5 times. You should see three lines of output with accuracy scores for each algorithm.

Cell taking more than 2 minutes?

Make sure you used the 10,000-sample training subset from Step 1. If you accidentally passed the full 70,000-sample X instead of X_train, cross-validation will be extremely slow.

Check that your cross_val_score call uses X_train and y_train, not X and y.

help me debug my cross-validation code

Visualize the comparison and pick the winner

Numbers in a terminal are hard to compare at a glance. A bar chart with error bars makes the winner obvious and shows how consistent each model is across folds.

  • In a new code cell, add this visualization code:
# Visualize cross-validation results
names = list(results.keys())
means = [results[n]['mean'] for n in names]
stds = [results[n]['std'] for n in names]

plt.figure(figsize=(8, 5))
plt.bar(names, means, yerr=stds, capsize=5, color=['steelblue', 'forestgreen', 'coral'])
plt.ylabel('Accuracy')
plt.title('Model Comparison (5-Fold Cross-Validation)')
plt.ylim(0.75, 0.95)
plt.tight_layout()
plt.show()

What does this code do?

It extracts the mean accuracy and standard deviation for each model, then plots them as a bar chart. The yerr=stds adds error bars showing how much the accuracy varied across folds. Smaller error bars mean a more stable model.

The plt.ylim(0.75, 0.95) zooms the y-axis so differences between models are visible instead of compressed near the top.

  • Run the cell to see your comparison chart.

You should see a bar chart with three colored bars. The taller the bar, the better the model performed.

Now that you can see which model wins, let's formally identify the best one and inspect exactly where it makes mistakes.

  • In the same cell, add the following code below your chart code to fit the best model and show its confusion matrix:
# Identify and evaluate the best model
best_name = max(results, key=lambda k: results[k]['mean'])
best_pipe = models[best_name]
best_pipe.fit(X_train, y_train)

print(f"\nBest model: {best_name}")
print(f"Test accuracy: {best_pipe.score(X_test, y_test):.4f}")

fig, ax = plt.subplots(figsize=(10, 8))
ConfusionMatrixDisplay.from_estimator(
    best_pipe, X_test, y_test,
    display_labels=class_names,
    xticks_rotation='vertical',
    ax=ax
)
plt.title(f'Confusion Matrix - {best_name}')
plt.tight_layout()
plt.show()

What does this code do?

The max() call finds which model had the highest mean cross-validation score. It then fits that model on the full training set and prints its test accuracy.

The confusion matrix is a grid showing every prediction the model made. Each row is a true class. Each column is a predicted class. Bright squares on the diagonal mean correct predictions. Off-diagonal values show which classes get confused with each other.

Before you run this, what do you think the confusion matrix will reveal about which clothing categories are hardest to tell apart?

  • Run the cell.

You should see a bar chart, the best model name printed (likely Logistic Regression or Random Forest), a test accuracy around 87%, and a confusion matrix. Look at the off-diagonal squares. The brightest ones show Shirt being confused with T-shirt/top and Coat.

Seeing a NameError for results?

The results dictionary was created in the previous cell. If you restarted your runtime or ran cells out of order, run the previous cell (the one with cross_val_score) again first.

help me fix my model comparison code

✔️ Awesome, I've got everything!

Great. Double check you've saved your notebook.

ⓧ I'd like to double check the full code

Here is the complete Cell 5 code. Confirm your cell matches this:

# Cell 5: Build a Pipeline and Compare Models
pipe_lr = make_pipeline(
    PCA(n_components=50, random_state=42),
    LogisticRegression(max_iter=1000, random_state=42)
)

pipe_rf = make_pipeline(
    PCA(n_components=50, random_state=42),
    RandomForestClassifier(n_estimators=100, random_state=42)
)

pipe_knn = make_pipeline(
    PCA(n_components=50, random_state=42),
    KNeighborsClassifier(n_neighbors=5)
)

models = {
    'Logistic Regression': pipe_lr,
    'Random Forest': pipe_rf,
    'KNN': pipe_knn
}

results = {}
for name, pipe in models.items():
    scores = cross_val_score(pipe, X_train, y_train, cv=5, scoring='accuracy')
    results[name] = {'mean': scores.mean(), 'std': scores.std()}
    print(f"{name}: {scores.mean():.4f} (+/- {scores.std():.4f})")

And the visualization cell:

names = list(results.keys())
means = [results[n]['mean'] for n in names]
stds = [results[n]['std'] for n in names]

plt.figure(figsize=(8, 5))
plt.bar(names, means, yerr=stds, capsize=5, color=['steelblue', 'forestgreen', 'coral'])
plt.ylabel('Accuracy')
plt.title('Model Comparison (5-Fold Cross-Validation)')
plt.ylim(0.75, 0.95)
plt.tight_layout()
plt.show()

best_name = max(results, key=lambda k: results[k]['mean'])
best_pipe = models[best_name]
best_pipe.fit(X_train, y_train)

print(f"\nBest model: {best_name}")
print(f"Test accuracy: {best_pipe.score(X_test, y_test):.4f}")

fig, ax = plt.subplots(figsize=(10, 8))
ConfusionMatrixDisplay.from_estimator(
    best_pipe, X_test, y_test,
    display_labels=class_names,
    xticks_rotation='vertical',
    ax=ax
)
plt.title(f'Confusion Matrix - {best_name}')
plt.tight_layout()
plt.show()

You now have a structured ML workflow: load data, compress with PCA, chain into a Pipeline, and compare models with cross-validation. Next up, you'll take this Pipeline into the Secret Mission and use GridSearchCV to automatically find the best hyperparameters.

Secret mission

Hyperparameter Tuning with GridSearchCV

You've been guessing hyperparameters this whole time. 50 PCA components, default regularization strength. What if there's a better combination hiding in plain sight? GridSearchCV systematically tries every combination and finds the optimal settings using cross-validation.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely in Google Colab, so there are no ongoing costs.

Resources you used:

  • Google Colab notebook (your fashion classifier code and outputs).

Keep everything running

No action needed. Choose this if you're still actively experimenting with your classifier or want to try new models.

  • Your Colab notebook stays in your Google Drive and you can reopen it anytime.
  • Colab will disconnect the runtime after a period of inactivity, but your saved notebook and outputs remain.

Pause - I'll come back to this later

Disconnect the Colab runtime to free up the allocated GPU/CPU resources. Your notebook file stays safely in Google Drive.

  • In your Colab notebook, click Runtime in the top menu bar.
  • Click Disconnect and delete runtime.
  • When you return, reopen the notebook and click Runtime then Run all to re-execute your cells.

Delete - I don't want to use this again

Remove the notebook entirely from your Google Drive.

  • In your Colab notebook, click File in the top menu bar.
  • Click Locate in Drive to find the notebook file in Google Drive.
  • Right-click the notebook file and select Move to trash.
  • The Fashion-MNIST dataset was only loaded into the runtime memory. It is not stored in your Drive, so no additional cleanup is needed.

That's a Wrap!

That's a Wrap!

Nice work! You just built a complete machine learning pipeline that classifies fashion product images into 10 categories from raw pixel data.

You've learned how to:

  • Build a fashion product image classifier that predicts clothing categories from 784 pixel features using scikit-learn.
  • Compress high-dimensional data with PCA to make training faster while preserving the most important patterns.
  • Train and compare three ML algorithms (Logistic Regression, Random Forest, KNN) using cross-validation and Pipelines to chain preprocessing and modeling steps together safely.
  • Secret Mission: Used GridSearchCV to automatically find optimal hyperparameters across the entire pipeline.

Ready to quiz yourself?