Build a Handwritten Digit Classifier

Train and audit a digit classifier with unseen-test evaluation.

Introduction

30 Second Summary

A result can look flawless when it is tested on questions it has already seen. The real challenge begins with new examples.

In this project, you will use supervised learning in Google Colab to build a random forest that recognizes handwritten digits. You will compare its performance on familiar images with its performance on unseen images.

What You'll Build

Your finished notebook turns handwritten images into predicted labels while revealing the exact digits that fooled the model.

By the end of this project, you'll have:

  • A visual digit check that pairs an 8 by 8 handwritten image with its known label.
  • Working predictions that let you compare ten actual labels with ten labels from your trained model.
  • An honest evaluation that compares training accuracy with accuracy on unseen digits. A confusion matrix and misclassified image show where the model fails.
  • Secret Mission: Repeat the evaluation across five stratified data splits. Compare the mean, minimum, and maximum accuracy.

Are there any prerequisites?

You need a Google account plus basic Python syntax. No prior machine learning experience or local Python setup is required.

Before We Start

Before the hands-on work begins, lock in what your classifier must predict. You will judge success by its accuracy on unseen digits.

Set Up Google Colab

A handwritten-digit classifier needs a dependable Python environment. Setting one up locally on Windows could pull your attention away from the machine learning workflow.

Google Colab provides a browser-based notebook with a managed runtime. You can focus on the classifier without configuring a local editor.

scikit-learn version 1.9.1 will provide the classification tools. Matplotlib version 3.11.2 will provide the plotting tools.

In this step, get ready to:
  • Create a Colab notebook named digit_classifier.ipynb.
  • Install the pinned project libraries in the current runtime.
  • Prove that both libraries import successfully.
Create the Colab notebook

Colab stores notebooks in Google Drive. Your code remains available after the temporary runtime is removed.

  • Open Google Colab in your browser.
  • Sign in to your Google account if Colab asks.
  • Click File in the top menu bar.
  • Select New notebook.
  • Click the current notebook title in the top-left corner.
  • Replace the title with digit_classifier.ipynb.
  • Press Enter to save the notebook name.

You'll see digit_classifier.ipynb in the notebook header. This confirms that your project has a named home in Google Drive.

Can't create the notebook?

Make sure you are signed in to a Google account. Wait for the Colab page to finish loading before using the File menu.

If the new title does not remain visible, click the title again. Press Enter after typing the filename.

Help me create and rename a Google Colab notebook.

Install the pinned libraries

Each Colab session runs inside a temporary virtual machine. The %pip command installs packages in the current notebook kernel.

  • Click inside the first code cell.
  • Install the pinned project libraries by pasting this command into the cell:
%pip install scikit-learn==1.9.1 matplotlib==3.11.2

What does this command do?

  • The %pip command targets the notebook's current kernel.
  • scikit-learn==1.9.1 pins the dataset and classification APIs to the project version.
  • matplotlib==3.11.2 pins the plotting API to the project version.
  • Run the installation cell by pressing Ctrl+Enter.

The cell prints package installation progress beneath the command. Use the outcome that matches what you see.

✔️ Both pinned versions installed

The cell completes without an installation error. Your runtime now has scikit-learn 1.9.1 plus Matplotlib 3.11.2 ready for the import check.

ⓧ I see an older version

An earlier package state can mention a different version in the installation output. The pinned installation cell applies the versions required by this project.

  • Run the installation cell again to apply the pinned versions.
  • Wait for the cell to finish before continuing to the import check.

ⓧ The installation failed

A failed installation leaves the runtime without dependable project libraries. Resolve the connection problem before continuing.

  • Confirm that your browser has an active internet connection.
  • Run the installation cell again.

Installation still failing?

Check that the installation command matches the cell shown above. A changed package name or version prevents the correct packages from installing.

Help me troubleshoot the package installation in Google Colab.

Verify the imports

An import check proves that the active runtime can find the installed libraries. The visible confirmation gives you a clean starting point for the classifier.

  • Use the notebook toolbar to add a second code cell beneath the installation cell.
  • Paste this import check into the new cell:
from sklearn.datasets import load_digits
import matplotlib.pyplot as plt

print("Setup ready")

What does this code do?

  • load_digits imports the function that provides the handwritten-digit dataset.
  • plt gives the notebook a short name for Matplotlib's plotting interface.
  • print("Setup ready") produces a visible confirmation after both imports succeed.

Before you run this check, do you expect both imports to succeed in the runtime where you installed the pinned packages?

  • Run the import-check cell by pressing Ctrl+Enter.

You'll see Setup ready directly beneath the cell without an import error. You now have a working runtime that can load both project libraries.

Don't see the setup message?

Check that the installation cell completed before the import-check cell. Run the installation cell again if its output stopped with an error.

Check the import cell against the code above. A changed module name prevents Python from finding the required API.

Help me fix the Colab import check.

✔️ Awesome, I've got everything!

  • Confirm that the notebook title reads digit_classifier.ipynb.
  • Confirm that the final cell output reads Setup ready.

ⓧ I'd like to double check the full code

Your first code cell installs the two pinned packages:

%pip install scikit-learn==1.9.1 matplotlib==3.11.2

Your second code cell checks the project imports:

from sklearn.datasets import load_digits
import matplotlib.pyplot as plt

print("Setup ready")

Your Colab notebook is saved with the required libraries ready. Next up, you'll load a real handwritten digit and give the classifier its first data.

Make the Model Recognize Digits

Your Google Colab notebook is ready. Now you can turn that setup into the project's first visible result.

Supervised learning links measurable image features to known labels. This step gives you a real digit image before a classifier starts producing predictions.

In this step, get ready to:
  • Load the handwritten-digits dataset.
  • Display its first 8 by 8 image.
  • Train a classifier that predicts familiar examples.
Load and inspect the digits dataset

Scikit-learn includes a dataset of small handwritten-digit images. Each sample pairs 64 pixel features with one known label.

  • Insert a new code cell beneath the import-check cell with the notebook's add-code control.
  • Paste the dataset code below into the new cell:
# Cell 1: Load and inspect the digits dataset
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import ConfusionMatrixDisplay, accuracy_score
from sklearn.model_selection import train_test_split

digits = load_digits()
X = digits.data
y = digits.target

print(f"Samples: {X.shape[0]}")
print(f"Features per sample: {X.shape[1]}")
print(f"Classes: {len(digits.target_names)}")
print(f"First label: {y[0]}")

plt.matshow(digits.images[0], cmap="gray")
plt.show()

What does this code do?

  • The imports bring the dataset loader into the notebook. They also prepare the model and evaluation tools used in later cells.
  • The digits variable holds the complete handwritten-digit dataset.
  • The X variable holds the flattened pixel features for every image.
  • The y variable holds the correct digit label for every sample.
  • The printed shape details reveal how many samples and features the classifier receives.
  • Matplotlib turns the first 8 by 8 pixel array into a grayscale image you can inspect.
  • Run the dataset cell by pressing Ctrl+Enter.

You'll see four dataset details followed by a grayscale digit image. The output shows that the notebook has loaded images and their known answers.

Good progress. Your notebook can now turn stored pixel values into a digit you can inspect.

Don't see the digit image?

Check that plt.matshow(digits.images[0], cmap="gray") appears before plt.show() in the same cell.

If an import fails, switch back to the installation cell from the previous step. Rerun that cell before rerunning this dataset cell.

Help me debug why my Colab cell does not display the first digit image.

Train a random forest

A random forest is a classifier that combines predictions from many decision trees. Fitting lets it learn patterns that connect the pixel features in X to the labels in y.

Why a random forest?

A random forest follows scikit-learn's standard fit and predict workflow. It also works directly with the numerical features already stored in X.

A neural network would add architecture and optimization choices. A random forest keeps this first experiment focused on the training and prediction loop.

  • Insert another code cell directly beneath the dataset cell.
  • Paste the classifier code below into the new cell:
# Cell 2: Train a naive model and predict familiar data
naive_model = RandomForestClassifier(n_estimators=100, random_state=42)
naive_model.fit(X, y)
naive_predictions = naive_model.predict(X)

print("First 10 actual labels:", y[:10])
print("First 10 predictions:", naive_predictions[:10])

What does this code do?

  • The naive_model variable holds a random forest containing 100 decision trees.
  • The fixed random_state=42 value makes the model's fitting behavior reproducible.
  • The fit method trains the classifier with every feature row in X and every corresponding label in y.
  • The predict method produces one predicted digit for each feature row in X.
  • The two slices make the first ten known labels easy to compare with the first ten predictions.

Before you run this cell, predict whether the first ten model predictions will resemble the first ten actual labels.

  • Run the classifier cell by pressing Ctrl+Enter.

You'll see First 10 actual labels: followed by ten values. You'll then see First 10 predictions: followed by ten model outputs.

Your first classifier is working. It now produces digit predictions from real image features.

Missing the prediction output?

If the cell reports that X or y is missing, rerun the dataset cell first. Notebook cells depend on values created by earlier cells.

Check that naive_model.fit(X, y) appears before naive_model.predict(X).

Help me troubleshoot why my random forest cell does not print digit predictions.

✔️ Awesome, I've got everything!

Your two project cells now load a digit image and produce ten model predictions. Confirm that both cells have finished running before you continue.

ⓧ I'd like to double check the full code

Compare your two project cells with the complete code below. Keep each commented section in its own consecutive notebook cell.

# Cell 1: Load and inspect the digits dataset
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import ConfusionMatrixDisplay, accuracy_score
from sklearn.model_selection import train_test_split

digits = load_digits()
X = digits.data
y = digits.target

print(f"Samples: {X.shape[0]}")
print(f"Features per sample: {X.shape[1]}")
print(f"Classes: {len(digits.target_names)}")
print(f"First label: {y[0]}")

plt.matshow(digits.images[0], cmap="gray")
plt.show()

# Cell 2: Train a naive model and predict familiar data
naive_model = RandomForestClassifier(n_estimators=100, random_state=42)
naive_model.fit(X, y)
naive_predictions = naive_model.predict(X)

print("First 10 actual labels:", y[:10])
print("First 10 predictions:", naive_predictions[:10])

Your classifier can now recognize familiar digit images. Next, you'll score those predictions and examine what that result can actually prove.

Expose the Training-Accuracy Trap

Your random forest can now label the same digit samples it learned from. The next question is whether its correct-looking predictions prove anything about new images.

In this step, you will calculate training accuracy across the familiar samples. The impressive result sets up an audit of what this number can actually support.

In this step, get ready to:
  • Calculate the model's accuracy across the examples used for fitting.
  • Explain why the score cannot prove performance on unseen digits.
Measure accuracy on familiar digits

The accuracy_score function calculates the fraction of labels the model predicted correctly. This first measurement compares naive_predictions with the same y labels used during fitting.

  • Add a new code cell directly beneath the prediction output using the notebook toolbar.
  • Paste the following calculation into the new cell:
# Cell 3: Expose the training-accuracy trap
training_accuracy = accuracy_score(y, naive_predictions)
print(f"Training accuracy: {training_accuracy:.3f}")

What does this code do?

  • The training_accuracy variable stores the fraction of labels predicted correctly.
  • The :.3f format displays the score to three decimal places.
  • The Training accuracy: label makes the result easy to identify in the output.

Before you run the cell, predict whether a score built from familiar digits will look convincing.

  • Run the cell by pressing Ctrl+Enter.

You'll see Training accuracy: followed by a score to three decimal places. The score should look extremely strong because the model is scoring data that was already available during fitting.

That impressive score is the trap in action. Your notebook now demonstrates how an attractive metric can overstate the available evidence.

No Training Score?

  • Check that the new code cell contains all three lines shown above.
  • Rerun the earlier model cell if the output mentions naive_predictions as missing.

Help me debug the missing training-accuracy output.

Audit what the score proves

A training score describes how well the model matches familiar data. Generalization describes how well the model handles examples that were unavailable during fitting.

  • Add a text cell directly beneath the training-accuracy output using the notebook toolbar.
  • Write a sentence explaining that every scored example was available during fitting.
  • Add a second sentence explaining that the score is not independent evidence of generalization.

Before you rerun the cell, predict what this number can prove about images the model never saw during fitting.

  • Rerun the training-accuracy cell by pressing Ctrl+Enter.

You'll see Training accuracy: above a text cell that explains the examples were familiar. Your notebook now shows why the score cannot establish generalization.

✔️ Awesome, I've got everything!

Great. Your third project cell now measures training accuracy across the familiar examples.

ⓧ I'd like to double check the full code

Compare the cumulative Python code in your first three project cells with this reference.

# Cell 1: Load and inspect the digits dataset
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import ConfusionMatrixDisplay, accuracy_score
from sklearn.model_selection import train_test_split

digits = load_digits()
X = digits.data
y = digits.target

print(f"Samples: {X.shape[0]}")
print(f"Features per sample: {X.shape[1]}")
print(f"Classes: {len(digits.target_names)}")
print(f"First label: {y[0]}")

plt.matshow(digits.images[0], cmap="gray")
plt.show()

# Cell 2: Train a naive model and predict familiar data
naive_model = RandomForestClassifier(n_estimators=100, random_state=42)
naive_model.fit(X, y)
naive_predictions = naive_model.predict(X)

print("First 10 actual labels:", y[:10])
print("First 10 predictions:", naive_predictions[:10])

# Cell 3: Expose the training-accuracy trap
training_accuracy = accuracy_score(y, naive_predictions)
print(f"Training accuracy: {training_accuracy:.3f}")

You have caught the key evaluation flaw behind the impressive score. Next, you'll test a freshly trained model on unseen digits and inspect where it makes mistakes.

Test Unseen Digits and Inspect Mistakes

Your Google Colab notebook currently reports an impressive training score. That score cannot establish generalization because the model already saw every scored image.

A stratified train/test split gives each example one evaluation role. A fresh random forest reveals performance on unseen digits through a confusion matrix.

In this step, get ready to:
  • Create a reproducible stratified 80/20 split.
  • Train a fresh classifier on the training examples.
  • Inspect the model's mistakes visually.
Create an independent evaluation

An independent evaluation keeps the test examples hidden during fitting. This makes the resulting accuracy evidence about unseen digits.

  • Add a new code cell beneath the training-accuracy explanation.
  • Paste this complete evaluation cell into the new code cell:
# Cell 4: Create an independent test split and retrain
all_indices = list(range(len(y)))
train_indices, test_indices = train_test_split(
    all_indices,
    test_size=0.20,
    random_state=42,
    stratify=y,
)

X_train = X[train_indices]
X_test = X[test_indices]
y_train = y[train_indices]
y_test = y[test_indices]

model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train, y_train)
test_predictions = model.predict(X_test)

test_accuracy = accuracy_score(y_test, test_predictions)
print(f"Unseen test accuracy: {test_accuracy:.3f}")

ConfusionMatrixDisplay.from_predictions(y_test, test_predictions)
plt.show()

What does this code do?

  • The all_indices list preserves each example's original image position.
  • The test_size=0.20 setting reserves 20 percent of the examples for testing.
  • The stratify=y setting preserves the class proportions across the split.
  • The model fits only on X_train with labels from y_train.
  • The ConfusionMatrixDisplay.from_predictions(y_test, test_predictions) call organizes the test results into a visual grid.

Before you run this cell, do you expect its unseen test accuracy to match the training accuracy exactly?

  • Run the cell by pressing Ctrl+Enter.

You'll see Unseen test accuracy: followed by a score. You'll also see a confusion matrix for the held-out examples.

Seeing a split or plotting error?

Run the earlier notebook cells in order. This restores X with y before the split runs.

Check the indentation inside train_test_split. Each argument should align beneath all_indices.

Ask for help if the cell still fails: help me debug this Colab evaluation cell

Read the confusion matrix

The matrix places matching label pairs on its main diagonal. Off-diagonal cells reveal mistakes between two digit classes.

  • Trace the main diagonal from the top left to identify correct predictions.
  • Scan outside the diagonal to find digit classes the model confused.
Inspect the first mistake

A single misclassified image turns an off-diagonal count into a concrete example. Its original position links the test result back to the correct 8 by 8 image.

  • Add another code cell beneath the confusion matrix output.
  • Paste the mistake-inspection code into the new code cell:
# Cell 5: Inspect the first mistake
wrong_positions = [
    position
    for position, (actual, predicted) in enumerate(zip(y_test, test_predictions))
    if actual != predicted
]

if wrong_positions:
    wrong_position = wrong_positions[0]
    print(f"Actual label: {y_test[wrong_position]}")
    print(f"Predicted label: {test_predictions[wrong_position]}")
    plt.matshow(digits.images[test_indices[wrong_position]], cmap="gray")
    plt.show()
else:
    print("No misclassified digit was found in this test split.")

How does the mistake lookup work?

  • The list comprehension compares every actual label with its prediction.
  • The wrong_positions list stores test positions where those labels differ.
  • The first stored position selects one mistake for inspection.
  • The matching test_indices value retrieves the original image from digits.images.
  • The final branch reports when the split contains no misclassified digit.

Before you run this cell, do you think the first mistake will still look recognizable as its actual digit?

  • Run the cell by pressing Ctrl+Enter.

You'll see an actual label followed by the model's predicted label. An 8 by 8 image of that misclassified digit appears beneath them.

That's the evaluation gap closed. Your notebook now tests unseen data before showing exactly where the classifier falls short.

No misclassified image appears?

Run the independent evaluation cell first. The mistake lookup needs test_predictions with test_indices from that cell.

Check the indentation beneath if wrong_positions:. Every image-display line belongs inside that branch.

Ask for help with the lookup: help me find why the misclassified digit is missing

✔️ Awesome, I've got everything!

Your notebook now evaluates unseen digits. It also exposes mistakes through a confusion matrix plus a misclassified image.

ⓧ I'd like to double check the full code

Compare your project cells with this complete cumulative version of digit_classifier.py.

# Cell 1: Load and inspect the digits dataset
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import ConfusionMatrixDisplay, accuracy_score
from sklearn.model_selection import train_test_split

digits = load_digits()
X = digits.data
y = digits.target

print(f"Samples: {X.shape[0]}")
print(f"Features per sample: {X.shape[1]}")
print(f"Classes: {len(digits.target_names)}")
print(f"First label: {y[0]}")

plt.matshow(digits.images[0], cmap="gray")
plt.show()

# Cell 2: Train a naive model and predict familiar data
naive_model = RandomForestClassifier(n_estimators=100, random_state=42)
naive_model.fit(X, y)
naive_predictions = naive_model.predict(X)

print("First 10 actual labels:", y[:10])
print("First 10 predictions:", naive_predictions[:10])

# Cell 3: Expose the training-accuracy trap
training_accuracy = accuracy_score(y, naive_predictions)
print(f"Training accuracy: {training_accuracy:.3f}")

# Cell 4: Create an independent test split and retrain
all_indices = list(range(len(y)))
train_indices, test_indices = train_test_split(
    all_indices,
    test_size=0.20,
    random_state=42,
    stratify=y,
)

X_train = X[train_indices]
X_test = X[test_indices]
y_train = y[train_indices]
y_test = y[test_indices]

model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train, y_train)
test_predictions = model.predict(X_test)

test_accuracy = accuracy_score(y_test, test_predictions)
print(f"Unseen test accuracy: {test_accuracy:.3f}")

ConfusionMatrixDisplay.from_predictions(y_test, test_predictions)
plt.show()

# Cell 5: Inspect the first mistake
wrong_positions = [
    position
    for position, (actual, predicted) in enumerate(zip(y_test, test_predictions))
    if actual != predicted
]

if wrong_positions:
    wrong_position = wrong_positions[0]
    print(f"Actual label: {y_test[wrong_position]}")
    print(f"Predicted label: {test_predictions[wrong_position]}")
    plt.matshow(digits.images[test_indices[wrong_position]], cmap="gray")
    plt.show()
else:
    print("No misclassified digit was found in this test split.")

Secret mission

Stress-Test the Split

Repeat your unseen-data evaluation across five split seeds. Compare the resulting range to see how stable one test result really is.

Clean Up Your Resources

Clean Up Your Resources

Your project uses Google Colab's free standard runtime. This small CPU workflow has no ongoing cost.

Choose whether to keep the notebook available, release the temporary runtime, or remove both resources.

Resources you used:

  • The digit_classifier.ipynb notebook stored in Google Drive.
  • The temporary Colab virtual machine containing the pinned packages.

Keep everything running

No action is needed. Choose this option if you want to rerun the classifier or share the notebook.

  • Your digit_classifier.ipynb notebook stays in Google Drive.
  • Your current Colab runtime remains available while the managed session stays active.
  • Free Colab resources are not guaranteed or unlimited. You may need to rerun the installation cell after Colab replaces the runtime.

Pause - I'll come back to this later

Release the temporary virtual machine to clear the current session. Your saved notebook remains available in Google Drive.

  • Click Runtime in the Colab menu bar.
  • Select Disconnect and delete runtime.

That's the temporary runtime cleaned up. Your digit_classifier.ipynb notebook stays saved for your next session.

Delete - I don't want to use this again

Remove the temporary runtime first. Remove the saved notebook next to clear the project resources.

  • Click Runtime in the Colab menu bar.
  • Select Disconnect and delete runtime.

The temporary virtual machine is now removed. The pinned packages installed inside it are removed with it.

  • Open Google Drive in a new browser tab.
  • Locate digit_classifier.ipynb in your Drive files.
  • Right-click digit_classifier.ipynb.
  • Select Remove.

Cleanup complete. You should no longer see digit_classifier.ipynb in its original Google Drive location.

Nice Work!

Nice Work!

Well done. You built a Google Colab notebook that classifies handwritten digits using a model evaluated on unseen data.

You've learned how to:

  • Store the samples in X. Use their features to connect each image with its label in y. Train a random forest classifier to recognize handwritten digits.
  • Reveal why training accuracy cannot prove generalization when naive_model scores data used during fitting.
  • Create a stratified train/test split for independent evaluation. Read class-level mistakes from a confusion matrix. Inspect a misclassified image. Compare its actual label with its predicted label.
  • Secret Mission: Measure how evaluation changes across five random data splits. Use the mean accuracy to summarize the runs. Use the minimum-to-maximum range to judge stability.

Ready to quiz yourself?