Build Your First ML Workflow

Build and evaluate a handwritten-digit classifier with Python and scikit-learn.

Introduction

30 Second Summary

A high score can look convincing while hiding every wrong answer. Trust starts when you can see what the score beats plus exactly where it fails.

In this project, you will build a local Python program that recognizes small images of handwritten digits. You will follow a complete supervised machine learning workflow from raw data to visible error analysis.

What You'll Build

One terminal run leaves you with scored predictions plus saved images of the digits your classifier misread.

By the end of this project, you'll have:

  • A labeled digit preview showing the first ten grayscale images. Each image maps back to one sample of 64 features. Its caption reveals the target label.
  • An honest model comparison that measures a scaled logistic regression pipeline against a most-frequent baseline. Five-fold cross-validation tests the pipeline across the training data. An untouched test set provides the final score.
  • A classification report that reveals class-by-class performance beyond accuracy. A saved confusion matrix pinpoints class-level errors. A misclassification gallery lets you inspect the actual images.
  • Secret Mission: an automatic audit that names the pair of digits the model confuses most often.

Are there any prerequisites?

You need basic Python syntax. No machine learning theory is required because the guide includes isolated environment setup plus package installation.

Before We Start

Before any hands-on work begins, take a moment to commit to the classifier you are building. You will also define why a deliberately weak baseline must come before trusting the real model.

Set Up a Clean ML Workspace

Your handwritten-digit classifier depends on package versions that work with its Python interpreter. An incompatible interpreter can stop the project before any model code runs.

A virtual environment keeps this project's packages separate from your other Python work. The version check determines which setup route fits your computer.

scikit-learn provides the machine learning tools. Matplotlib creates the visual results.

In this step, get ready to:
  • Create the ml-digit-basics workspace in Visual Studio Code.
  • Activate an isolated environment with Python 3.11 or newer.
  • Install the pinned packages inside the active environment.
Create and inspect your workspace

A workspace gives every project file one predictable home. The integrated terminal starts inside the folder you open.

  • Start Visual Studio Code from your Linux application launcher.
  • Select File from the menu bar.
  • Select Open Folder....
  • Choose Desktop in the folder picker.
  • Select New Folder.
  • Enter ml-digit-basics as the folder name.
  • Select the new ml-digit-basics folder.
  • Select Select Folder.

Visual Studio Code may display a Workspace Trust prompt. You created this empty folder yourself.

  • Select Yes, I trust the authors if the Workspace Trust prompt appears.

The Explorer view now shows ml-digit-basics as the open workspace. Future project files now have the correct destination.

  • Select View from the menu bar.
  • Select Terminal.
  • Check the installed Python version by running this command:
python3 --version

Why check the version?

The pinned package releases require Python 3.11 or newer. Your output determines which environment route you follow.

A compatible interpreter can use Python's standard environment tool. An older interpreter uses the pinned fallback.

Activate a compatible environment

The environment stores its own interpreter plus project packages. Choose the tab that matches the result from your version check.

✔️ I see version 3.11 or higher

Your installed interpreter meets the package requirement. You can create the environment with Python's standard library.

  • Create the sklearn-env environment by running this command:
python3 -m venv sklearn-env

What does this command do?

The venv module creates an isolated Python environment inside sklearn-env. Packages installed there stay scoped to this project.

Environment creation failed?

Your Linux installation may be missing standard virtual environment support. Follow your Linux distribution's Python package instructions before retrying the command.

Help me diagnose the virtual environment error.

  • Activate sklearn-env by running this command:
source sklearn-env/bin/activate

What does activation change?

Activation points the current terminal at the Python interpreter inside sklearn-env. Package installation now targets this isolated environment.

  • Confirm that sklearn-env appears at the start of the terminal prompt.

Your terminal now uses the compatible project environment. The system interpreter remains separate.

Environment name missing?

Check that the terminal is inside the open ml-digit-basics workspace. Run the activation command again from that folder.

Help me activate the project environment.

ⓧ I see an older version

The installed interpreter is older than the packages require. uv version 0.13.5 can supply Python 3.11 for this project.

  • Install the pinned uv release by running this command:
curl -LsSf https://astral.sh/uv/0.13.5/install.sh | sh

What does this command do?

The official installer downloads uv version 0.13.5 for your Linux user account. A fresh terminal loads the updated command path.

Installer did not finish?

Check that your internet connection can reach the installer address. Retry the command after the connection is stable.

Help me troubleshoot the uv installation.

  • Close the current terminal from the terminal panel.
  • Select Terminal from the menu bar.
  • Select New Terminal.
  • Create a Python 3.11 environment by running this command:
uv venv --python 3.11

How does the fallback work?

uv creates its default environment at .venv. It downloads Python 3.11 when that interpreter is unavailable locally.

uv command unavailable?

Confirm that you opened a fresh terminal after the installation completed. The old terminal may not have the updated command path.

Help me make uv available in the terminal.

  • Activate .venv by running this command:
source .venv/bin/activate

What does activation change?

Activation points the current terminal at the Python interpreter inside .venv. Package installation now uses Python 3.11.

  • Confirm that .venv appears at the start of the terminal prompt.

Your project now has a compatible interpreter. The older system interpreter remains unchanged.

Environment name missing?

Check that the terminal is inside the open ml-digit-basics workspace. Run the activation command again from that folder.

Help me activate the uv environment.

ⓧ Command not found

The terminal cannot find the expected Python command. uv version 0.13.5 can download Python 3.11 while creating this project's environment.

  • Install the pinned uv release by running this command:
curl -LsSf https://astral.sh/uv/0.13.5/install.sh | sh

What does this command do?

The official installer downloads uv version 0.13.5 for your Linux user account. A fresh terminal loads the new command path.

Installer did not finish?

Check that your internet connection can reach the installer address. Retry the command after the connection is stable.

Help me troubleshoot the uv installation.

  • Close the current terminal from the terminal panel.
  • Select Terminal from the menu bar.
  • Select New Terminal.
  • Create a Python 3.11 environment by running this command:
uv venv --python 3.11

How does the fallback work?

uv creates .venv inside ml-digit-basics. Its managed Python support supplies version 3.11.

uv command unavailable?

Confirm that you opened a fresh terminal after the installation completed. The old terminal may not have the updated command path.

Help me make uv available in the terminal.

  • Activate .venv by running this command:
source .venv/bin/activate

What does activation change?

Activation makes the Python interpreter inside .venv the current terminal's interpreter. Package installation can now use Python 3.11.

  • Confirm that .venv appears at the start of the terminal prompt.

Your project now has a usable interpreter. The missing system command no longer blocks the workflow.

Environment name missing?

Check that the terminal is inside the open ml-digit-basics workspace. Run the activation command again from that folder.

Help me activate the uv environment.

Install and verify the pinned packages

A requirements file records the exact package versions used by the workflow. These pins make the environment reproducible when you return to the project.

  • Select the Explorer icon in the Activity Bar.
  • Select the new-file control at the top of the Explorer sidebar.
  • Enter requirements.txt as the file name.
  • Press Enter to create the file.

The Explorer now lists requirements.txt inside ml-digit-basics.

  • Add the pinned project packages by pasting this content into requirements.txt:
scikit-learn==1.9.1
matplotlib==3.11.2

What do these pins control?

  • scikit-learn version 1.9.1 provides the dataset plus the modeling tools used later.
  • Matplotlib version 3.11.2 creates the digit preview plus the error-analysis images.
  • Press Ctrl+S on Linux to save requirements.txt.
  • Confirm that the editor contains exactly two pinned package lines.

Your dependency file is now saved. It provides one source for every package installation in this project.

File missing from Explorer?

Check that the Explorer heading shows ml-digit-basics. Create the file inside that open workspace if another folder appears.

Help me place the requirements file correctly.

✔️ Awesome, I've got everything!

Good work. Confirm that requirements.txt is saved before installing the packages.

ⓧ I'd like to double check the full code

scikit-learn==1.9.1
matplotlib==3.11.2

What should match?

Your file should contain these two lines in this order. Each package name must use the version pin shown here.

The installation command depends on the environment you activated. Choose the route that matches the name at the start of your terminal prompt.

sklearn-env route

  • Install the pinned packages into sklearn-env by running this command:
python -m pip install -r requirements.txt

What does this command do?

The active environment's Python interpreter runs pip against requirements.txt. The two pinned releases are installed inside sklearn-env.

The installation may take a little while while package files download. The terminal returns to the prompt when the process finishes.

Package installation failed?

Confirm that sklearn-env still appears at the start of the terminal prompt. Compare both lines in requirements.txt with the reference above.

Help me troubleshoot the pip installation.

uv fallback route

  • Install the pinned packages into .venv by running this command:
uv pip install -r requirements.txt

What does this command do?

uv reads the two pins from requirements.txt. It installs those releases into the discovered .venv environment.

The installation may take a little while while package files download. The terminal returns to the prompt when the process finishes.

Package installation failed?

Confirm that .venv still appears at the start of the terminal prompt. Compare both package pins with the reference above.

Help me troubleshoot the uv installation.

A package inventory provides concrete proof that the active environment matches requirements.txt.

Before you run the final check, do you expect both pinned package versions to appear?

sklearn-env route

  • List the packages installed in sklearn-env by running this command:
python -m pip freeze

What should you see?

The output includes scikit-learn==1.9.1 plus matplotlib==3.11.2. These lines confirm that the active environment matches the project pins.

Pinned line missing?

Confirm that sklearn-env still appears at the start of the prompt. Return to the matching installation route if either pinned line is absent.

Help me repair the standard environment.

uv fallback route

  • List the packages installed in .venv by running this command:
uv pip freeze

What should you see?

The output includes scikit-learn==1.9.1 plus matplotlib==3.11.2. These lines confirm that the active environment matches the project pins.

Pinned line missing?

Confirm that .venv still appears at the start of the prompt. Return to the matching installation route if either pinned line is absent.

Help me repair the uv environment.

That is the setup complete. Your isolated workspace now has the exact packages needed for the digit workflow.

Your clean machine learning workspace is ready. Next, you will load the handwritten digits and connect their feature rows to visible images.

Inspect the Digits as Data

Your isolated Python environment now has the packages this workflow needs. The project can move from setup to a result you can see.

A supervised machine learning model connects each feature row to its target label. Inspecting the array shape beside the original digit images makes that relationship concrete.

In this step, get ready to:
  • Create a runnable script that loads the handwritten-digit dataset.
  • Connect flattened feature rows to original images through printed dimensions.
  • Save a labeled preview of the first ten digits.
Load the digits dataset

scikit-learn includes a digits dataset that works without a separate download. The load_digits() function exposes flattened rows for modeling plus 8 by 8 images for inspection.

  • Use the Explorer sidebar in Visual Studio Code to create digit_workflow.py inside ml-digit-basics.

You'll see digit_workflow.py beside requirements.txt in the Explorer sidebar.

  • Build the first runnable dataset inspection by pasting this code into digit_workflow.py:
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits


digits = load_digits()
X = digits.data
y = digits.target

print(f"Samples: {X.shape[0]}")
print(f"Features per sample: {X.shape[1]}")
print(f"Original image shape: {digits.images[0].shape}")

What Does This Code Do?

  • The digits variable holds the complete handwritten-digit dataset.
  • The X variable stores each sample as a row of 64 features.
  • The y variable stores the matching label for each row.
  • The digits.images[0] value exposes the first sample in its original 8 by 8 image shape.
  • Save digit_workflow.py.

Before you run it, how many feature values do you expect after flattening an 8 by 8 image?

  • Inspect the dataset by running this command in the integrated terminal:
python digit_workflow.py

What Does This Command Do?

The python command uses your active project environment to execute digit_workflow.py. The three printed dimensions confirm that the dataset loaded successfully.

You'll see Samples: 1797 followed by Features per sample: 64. The final line reads Original image shape: (8, 8).

Imports Failing?

  • Return to the integrated terminal from Step 1.
  • Confirm that its prompt still shows either sklearn-env or .venv.
  • Repeat the matching environment activation route from Step 1 if the environment name is missing.

Help me diagnose why the digits dataset imports fail in my active environment.

Plot a labeled preview

Matplotlib turns the first ten numeric samples back into visible grayscale digits. The title above each image makes its target label visible.

  • Extend digit_workflow.py after the print(f"Original image shape: {digits.images[0].shape}") line by pasting this code:
preview_figure, preview_axes = plt.subplots(2, 5, figsize=(10, 4))
for axis, image, label in zip(preview_axes.flat, digits.images[:10], y[:10]):
    axis.imshow(image, cmap="gray")
    axis.set_title(f"Label: {label}")
    axis.set_axis_off()
preview_figure.suptitle("Handwritten digits and their labels")
plt.tight_layout()
preview_figure.savefig("digits_preview.png", dpi=150, bbox_inches="tight")
plt.close(preview_figure)
print("Saved digits_preview.png")

How Does the Preview Work?

  • The preview_figure variable holds the complete image grid.
  • The preview_axes variable gives the loop ten positions across a two-row grid.
  • The axis variable receives one image plus its matching label during each loop.
  • The final lines save the grid as digits_preview.png before closing the completed figure.
  • Save digit_workflow.py.

Before you run the updated script, what should each title reveal about the image below it?

  • Generate the labeled preview by running this command in the integrated terminal:
python digit_workflow.py

Why Run the Script Again?

The command now executes the new preview section after printing the dataset dimensions. That section writes the labeled image grid into your ml-digit-basics folder.

You'll see Saved digits_preview.png as the final terminal line. You'll also see digits_preview.png in the Explorer sidebar.

  • Select digits_preview.png in the Explorer sidebar to inspect the saved image.

You'll see ten grayscale handwritten digits arranged in two rows. Each digit has a matching label above it.

Preview Missing or Blank?

  • Confirm that digit_workflow.py was saved before you ran it.
  • Check that the preview code sits below the three dataset dimension lines.
  • Confirm that the integrated terminal is still inside ml-digit-basics.

Help me find why my digit preview file is missing or blank.

The finished script should match this reference.

✔️ Awesome, I've got everything!

Your script prints the dataset dimensions. It also saves a labeled preview of the first ten digits.

ⓧ I'd like to double check the full code

import matplotlib.pyplot as plt
from sklearn.datasets import load_digits


digits = load_digits()
X = digits.data
y = digits.target

print(f"Samples: {X.shape[0]}")
print(f"Features per sample: {X.shape[1]}")
print(f"Original image shape: {digits.images[0].shape}")

preview_figure, preview_axes = plt.subplots(2, 5, figsize=(10, 4))
for axis, image, label in zip(preview_axes.flat, digits.images[:10], y[:10]):
    axis.imshow(image, cmap="gray")
    axis.set_title(f"Label: {label}")
    axis.set_axis_off()
preview_figure.suptitle("Handwritten digits and their labels")
plt.tight_layout()
preview_figure.savefig("digits_preview.png", dpi=150, bbox_inches="tight")
plt.close(preview_figure)
print("Saved digits_preview.png")

How to Use This Reference

Compare the imports first. Continue downward through the data assignments plus the printed dimensions.

Finish by checking the preview variables plus the saved filename. Matching this reference gives you the exact runnable artifact for this step.

Your dataset is now visible in two forms. Next, you will give every real classifier a minimum score to beat.

Establish a Baseline

You can now see the digits as labeled images plus feature rows. The next question is whether a model's score is genuinely useful.

Accuracy needs a point of comparison before it means anything. A most-frequent baseline supplies that context through valid predictions based on class frequency.

In this step, get ready to:
  • Create a reproducible stratified split between training rows plus test rows.
  • Train a most-frequent baseline on the training labels.
  • Run the workflow to observe the minimum score a useful classifier must beat.
Create a stratified train/test split

A train/test split separates rows used for learning from rows reserved for evaluation. Stratification keeps every digit class represented in both groups.

  • Return to digit_workflow.py in Visual Studio Code.
  • Find the import block at the top of the file.
  • Compare the current import block with this reference:
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits

What is already imported?

These imports provide the plotting tools plus the digits dataset. The next block adds the tools needed to split the rows before measuring a baseline.

  • Replace the current import block with this expanded block:
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.dummy import DummyClassifier
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split

What do the new imports provide?

  • The DummyClassifier class creates the deliberately simple baseline.
  • The accuracy_score function measures the fraction of correct test predictions.
  • The train_test_split function separates the dataset before training begins.
  • All three tools come from scikit-learn.
  • Scroll to the final print("Saved digits_preview.png") line.
  • Add the split below that line by pasting this code:
X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.25,
    random_state=42,
    stratify=y,
)
print(f"Training samples: {len(X_train)}")
print(f"Test samples: {len(X_test)}")

How does the split work?

  • The test_size=0.25 setting reserves one quarter of the samples for testing.
  • The random_state=42 setting produces the same split on every run.
  • The stratify=y setting preserves the digit class proportions.
  • The four output variables keep the feature rows paired with their labels.
  • Save digit_workflow.py.
  • Run the updated workflow with this command:
python digit_workflow.py

What does this run check?

The script still rebuilds the digit preview. It now also prints the sizes of the two dataset groups.

You'll see Training samples: 1347. You'll also see Test samples: 450.

You have the first evaluation guardrail in place. Those test rows can now measure performance on data excluded from training.

Split sizes missing?

Confirm that the split appears below print("Saved digits_preview.png"). Check that all three new imports appear at the top of digit_workflow.py.

Help me debug the train and test split in my digit workflow.

Fit the most-frequent baseline

A baseline model gives later scores a minimum target to beat. The most-frequent strategy learns the dominant training label.

It uses that single label for every test prediction. Every prediction still has the correct output format.

  • Add the baseline block at the bottom of digit_workflow.py by pasting this code:
baseline = DummyClassifier(strategy="most_frequent")
baseline.fit(X_train, y_train)
baseline_predictions = baseline.predict(X_test)
baseline_accuracy = accuracy_score(y_test, baseline_predictions)
print(f"Baseline accuracy: {baseline_accuracy:.3f}")

What does the baseline do?

  • The strategy="most_frequent" setting chooses the most common training label.
  • The fit() call learns which label occurs most often.
  • The predict() call repeats that label across the test rows.
  • The baseline_accuracy value records how often those repeated predictions match the true labels.
  • Save digit_workflow.py.

Before you run this version, do you think one repeated digit can classify the full test set well?

  • Test the most-frequent strategy by running this command:
python digit_workflow.py

What does this run test?

This run applies the same repeated prediction to all 450 test rows. The final line measures those predictions against the true test labels.

You'll see Baseline accuracy: 0.102 as the final line.

That low score is the intended shortfall. Predicting one class ignores all 64 pixel features in every sample.

Baseline result missing?

A missing baseline line usually means the new block is above the split. Place the baseline code after the lines that create X_train plus y_train.

Help me find why my baseline accuracy is missing.

Confirm what the baseline proves

The baseline marks the performance floor for the real classifier. A useful model must beat this score by learning patterns from the pixels.

Why is this a valid comparison?

The baseline trains on the same training labels used by the later model. It is evaluated against the same untouched test labels.

That shared evaluation makes the comparison honest. Any improvement must come from learning useful feature patterns.

  • Choose the tab that matches how you want to check digit_workflow.py.

✔️ Awesome, I've got everything!

Your file now creates the split before training the most-frequent baseline. Make sure your latest changes are saved.

ⓧ I'd like to double check the full code

import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.dummy import DummyClassifier
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split


digits = load_digits()
X = digits.data
y = digits.target

print(f"Samples: {X.shape[0]}")
print(f"Features per sample: {X.shape[1]}")
print(f"Original image shape: {digits.images[0].shape}")

preview_figure, preview_axes = plt.subplots(2, 5, figsize=(10, 4))
for axis, image, label in zip(preview_axes.flat, digits.images[:10], y[:10]):
    axis.imshow(image, cmap="gray")
    axis.set_title(f"Label: {label}")
    axis.set_axis_off()
preview_figure.suptitle("Handwritten digits and their labels")
plt.tight_layout()
preview_figure.savefig("digits_preview.png", dpi=150, bbox_inches="tight")
plt.close(preview_figure)
print("Saved digits_preview.png")

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.25,
    random_state=42,
    stratify=y,
)
print(f"Training samples: {len(X_train)}")
print(f"Test samples: {len(X_test)}")

baseline = DummyClassifier(strategy="most_frequent")
baseline.fit(X_train, y_train)
baseline_predictions = baseline.predict(X_test)
baseline_accuracy = accuracy_score(y_test, baseline_predictions)
print(f"Baseline accuracy: {baseline_accuracy:.3f}")

How should you use this reference?

Compare the import block first. Then check that the split appears before the baseline code.

Before the final run, do you expect the reproducible split to produce the same baseline result?

  • Verify the completed baseline workflow by running this command:
python digit_workflow.py

What does the final run confirm?

The final run confirms that the preview generation still works. It also confirms that the split plus baseline execute in the correct order.

You'll see the training size plus test size again. The final line remains Baseline accuracy: 0.102.

Seeing a different result?

Check that random_state=42 appears inside train_test_split(). Confirm that the baseline uses strategy="most_frequent".

Help me compare my baseline output with the expected result.

You have now measured a deliberately weak prediction strategy. Your next classifier has a clear result to beat.

Your evaluation now has a trustworthy starting point. Next, you'll train a model that learns from the pixel features.

Train a Real Classifier

Your most-frequent baseline made valid predictions while ignoring every pixel. Its one-class result exposed why frequency alone cannot recognize handwritten digits.

You will now train a logistic regression model that learns from the pixel features. A pipeline keeps feature scaling attached to the classifier. Cross-validation checks whether its improvement holds across several training folds.

In this step, get ready to:
  • Build a scaled logistic regression pipeline.
  • Measure the pipeline with five-fold cross-validation.
  • Evaluate the trained pipeline against the baseline on untouched test rows.
Build the model pipeline

Each pixel position becomes a feature column. StandardScaler learns how to scale those columns from the training rows.

The pipeline applies that transformation whenever the classifier trains or predicts. This keeps preprocessing connected to the model that depends on it.

  • In digit_workflow.py, replace the import section at the top with this expanded import section:
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
from sklearn.model_selection import cross_val_score, train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

What do these imports add?

  • The LogisticRegression classifier learns the relationship between pixel values and digit labels.
  • The cross_val_score function evaluates the same model across several folds.
  • The make_pipeline function keeps preprocessing connected to classification.
  • The StandardScaler transformer learns a mean and standard deviation for each feature.
  • Save digit_workflow.py.
  • Verify the expanded imports by running this command:
python digit_workflow.py

What does this run prove?

You should see the existing dataset details and baseline result complete successfully. This confirms that every new import is available in your project environment.

Seeing an import problem?

  • Check that the project environment remains activated in the Visual Studio Code terminal.
  • Compare each import with the expanded section above.

Ask for help with debugging the classifier imports.

  • In digit_workflow.py, place this code directly below the line that prints Baseline accuracy:
model = make_pipeline(
    StandardScaler(),
    LogisticRegression(max_iter=1000),
)

What does this code do?

  • The model variable holds one ordered workflow for scaling and classification.
  • The pipeline fits StandardScaler before fitting LogisticRegression.
  • The max_iter=1000 setting gives the classifier enough iterations to fit this dataset.
  • Save digit_workflow.py.
  • Confirm that the pipeline can be constructed by running this command:
python digit_workflow.py

What does this check prove?

You should see the script complete through the baseline result again. The successful run proves that Python can construct the scaled classifier pipeline.

Pipeline construction failing?

  • Check that the closing parenthesis appears after LogisticRegression(max_iter=1000).
  • Confirm that model starts at the left edge of the file.

Ask for help with fixing the model pipeline.

Cross-validate the training data

One train/test split can produce a fortunate result. Five-fold cross-validation rotates through five training and validation partitions to show how consistently the pipeline performs.

The held-out test rows stay outside this process. They remain reserved for the final check.

  • In digit_workflow.py, place this code directly below the closing parenthesis of model:
cross_validation_scores = cross_val_score(model, X_train, y_train, cv=5)
formatted_fold_scores = ", ".join(
    f"{score:.3f}" for score in cross_validation_scores
)
mean_cross_validation_score = sum(cross_validation_scores) / len(
    cross_validation_scores
)
print(f"Cross-validation fold scores: {formatted_fold_scores}")
print(f"Mean cross-validation accuracy: {mean_cross_validation_score:.3f}")

How does cross-validation work here?

  • The cv=5 value requests five evaluation folds from the training data.
  • The cross_validation_scores array holds one accuracy score from each fold.
  • The formatted_fold_scores string makes every score readable to three decimal places.
  • The mean_cross_validation_score value summarizes performance across all five folds.
  • Save digit_workflow.py.
  • Run the expanded workflow with this command:
python digit_workflow.py

What should you see?

You should see five decimal values after Cross-validation fold scores. The next line shows their mean accuracy.

Scores that stay close together show that the improvement is consistent across the training folds. This gives you stronger evidence than a result from one split.

Missing the fold scores?

  • Check that the cross-validation code sits below the completed model pipeline.
  • Confirm that X_train and y_train use the same capitalization as the train/test split.
  • Check the indentation inside the two multiline expressions.

Ask for help with debugging the cross-validation output.

Evaluate the untouched test set

Cross-validation measures consistency within the training rows. The untouched test set now gives the pipeline its final evaluation on data that played no role in fitting.

Placing this result beside baseline_accuracy answers the important question. The real classifier must beat the deliberate one-class strategy.

  • In digit_workflow.py, place this code directly below the mean cross-validation print line:
model.fit(X_train, y_train)
predictions = model.predict(X_test)
model_accuracy = accuracy_score(y_test, predictions)
print(f"Model test accuracy: {model_accuracy:.3f}")

What does this code do?

  • The fit call trains the complete pipeline on every training row.
  • The predictions array holds one predicted digit for each untouched test row.
  • The model_accuracy value measures the fraction of correct test predictions.
  • The final print line makes the model result easy to compare with Baseline accuracy.
  • Save digit_workflow.py.

Before you run the finished workflow, do you expect the pixel-based model to beat the one-class baseline?

  • Evaluate the finished classifier by running this command:
python digit_workflow.py

What should you see?

You should see five fold scores followed by Mean cross-validation accuracy. The final line shows Model test accuracy.

Compare the final value with Baseline accuracy earlier in the output. The model test accuracy should be higher because the pipeline learns from the pixel features.

You have trained a classifier that clearly beats the deliberate baseline. Its five fold scores also show that the improvement survives beyond one lucky partition.

Model accuracy missing or unexpectedly low?

  • Check that model.fit uses X_train and y_train.
  • Confirm that model.predict uses X_test.
  • Verify that accuracy_score compares y_test with predictions.

Ask for help with debugging the final model evaluation.

✔️ Awesome, I've got everything!

Your saved workflow now cross-validates the pipeline before evaluating it on the untouched test set.

ⓧ I'd like to double check the full code

  • Compare digit_workflow.py with this complete reference:
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
from sklearn.model_selection import cross_val_score, train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler


digits = load_digits()
X = digits.data
y = digits.target

print(f"Samples: {X.shape[0]}")
print(f"Features per sample: {X.shape[1]}")
print(f"Original image shape: {digits.images[0].shape}")

preview_figure, preview_axes = plt.subplots(2, 5, figsize=(10, 4))
for axis, image, label in zip(preview_axes.flat, digits.images[:10], y[:10]):
    axis.imshow(image, cmap="gray")
    axis.set_title(f"Label: {label}")
    axis.set_axis_off()
preview_figure.suptitle("Handwritten digits and their labels")
plt.tight_layout()
preview_figure.savefig("digits_preview.png", dpi=150, bbox_inches="tight")
plt.close(preview_figure)
print("Saved digits_preview.png")

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.25,
    random_state=42,
    stratify=y,
)
print(f"Training samples: {len(X_train)}")
print(f"Test samples: {len(X_test)}")

baseline = DummyClassifier(strategy="most_frequent")
baseline.fit(X_train, y_train)
baseline_predictions = baseline.predict(X_test)
baseline_accuracy = accuracy_score(y_test, baseline_predictions)
print(f"Baseline accuracy: {baseline_accuracy:.3f}")

model = make_pipeline(
    StandardScaler(),
    LogisticRegression(max_iter=1000),
)

cross_validation_scores = cross_val_score(model, X_train, y_train, cv=5)
formatted_fold_scores = ", ".join(
    f"{score:.3f}" for score in cross_validation_scores
)
mean_cross_validation_score = sum(cross_validation_scores) / len(
    cross_validation_scores
)
print(f"Cross-validation fold scores: {formatted_fold_scores}")
print(f"Mean cross-validation accuracy: {mean_cross_validation_score:.3f}")

model.fit(X_train, y_train)
predictions = model.predict(X_test)
model_accuracy = accuracy_score(y_test, predictions)
print(f"Model test accuracy: {model_accuracy:.3f}")

What should match?

The complete file preserves the dataset preview and baseline from earlier steps. The new section starts with model = make_pipeline( before finishing with the untouched test-set accuracy.

Your workflow now proves that the real classifier beats its baseline across repeated folds and an untouched test set. Next, you will turn that score into per-class evidence and visible prediction mistakes.

Explain the Model Results

Your scaled logistic regression pipeline now reports cross-validation scores. It also beats the baseline on untouched test data.

One accuracy score hides which digit classes fail. Visual error analysis exposes the patterns behind those mistakes.

In this step, get ready to:
  • Generate a classification report with precision, recall, F1 score, and support for every digit.
  • Save a confusion matrix that reveals which digit pairs the model mixes up.
  • Build a gallery that shows the first eight misclassified test images.
Generate the classification report

A classification report measures performance separately for each digit. This reveals weaknesses that the overall model accuracy cannot show.

  • In digit_workflow.py, find the metrics import near the top of the file.
  • Replace that import with the expanded line below:
from sklearn.metrics import ConfusionMatrixDisplay, accuracy_score, classification_report

Why expand this import?

The expanded scikit-learn import keeps the existing accuracy function. It also adds the report generator and the confusion matrix display used in this step.

  • Find the final model_accuracy print statement.
  • Add the following code directly below it:
report = classification_report(y_test, predictions)
print("Classification report:")
print(report)
with open("classification_report.txt", "w", encoding="utf-8") as report_file:
    report_file.write(report)

What does this code do?

  • The classification_report() function compares the true test labels with the model predictions.
  • Precision measures how often predictions for a digit are correct.
  • Recall measures how many real examples of a digit the model finds.
  • F1 score balances precision with recall. Support counts the test examples for each digit.
  • The file-writing block preserves the same report in classification_report.txt for later inspection.
  • Save digit_workflow.py.
  • Generate the classification report by running this command:
python digit_workflow.py

What should you see?

The terminal prints a row for each digit beneath Classification report:. Each row contains precision, recall, F1 score, and support.

You also see classification_report.txt in the project folder. The saved file contains the same report shown in the terminal.

Report missing or incomplete?

Check that the expanded metrics import includes classification_report. Confirm that the report code sits below the line that creates predictions.

Check that report_file.write(report) remains indented inside the file-writing block.

Ask for help with the report code.

Save the confusion matrix

A confusion matrix arranges true digits into rows. Its columns represent the digits predicted by the model.

  • Find report_file.write(report) near the bottom of digit_workflow.py.
  • Add the following code directly below the report block:
confusion_display = ConfusionMatrixDisplay.from_predictions(
    y_test,
    predictions,
    colorbar=False,
)
confusion_display.figure_.suptitle("Digit confusion matrix")
plt.tight_layout()
confusion_display.figure_.savefig(
    "confusion_matrix.png",
    dpi=150,
    bbox_inches="tight",
)
plt.close(confusion_display.figure_)

How does the matrix reveal errors?

  • The display compares y_test with predictions.
  • Cells on the diagonal count correct predictions because their true digit matches their predicted digit.
  • Cells away from the diagonal count mistakes between different digit classes.
  • The Matplotlib figure is saved as confusion_matrix.png before it is closed.
  • Save digit_workflow.py.
  • Regenerate the workflow artifacts by running this command:
python digit_workflow.py

What should you see?

The script completes after printing the classification report. You see confusion_matrix.png in the project folder.

  • Open confusion_matrix.png from the Explorer sidebar.
  • Confirm that the largest counts form a diagonal from the top left to the bottom right.
  • Locate smaller counts away from the diagonal to identify wrong predictions.

The diagonal shows that most test images are classified correctly. Every off-diagonal count points to a specific confusion between two digits.

Confusion matrix missing?

Confirm that ConfusionMatrixDisplay appears in the metrics import. Check that the filename inside savefig() is confusion_matrix.png.

Ask for help with the confusion matrix.

Inspect misclassified digits

Matrix counts identify error patterns across the test set. A misclassification gallery lets you inspect the actual handwriting behind those counts.

  • Find plt.close(confusion_display.figure_) near the bottom of digit_workflow.py.
  • Add the following gallery code directly below it:
mistake_indices = [
    index
    for index, (actual, predicted) in enumerate(zip(y_test, predictions))
    if actual != predicted
][:8]

mistake_figure, mistake_axes = plt.subplots(2, 4, figsize=(10, 5))
for axis in mistake_axes.flat:
    axis.set_axis_off()
for axis, index in zip(mistake_axes.flat, mistake_indices):
    axis.imshow(X_test[index].reshape(8, 8), cmap="gray")
    axis.set_title(f"True: {y_test[index]} | Pred: {predictions[index]}")
mistake_figure.suptitle("Misclassified digits")
plt.tight_layout()
mistake_figure.savefig(
    "misclassified_digits.png",
    dpi=150,
    bbox_inches="tight",
)
plt.close(mistake_figure)

print("Saved classification_report.txt")
print("Saved confusion_matrix.png")
print("Saved misclassified_digits.png")

How does the gallery work?

  • The list comprehension keeps test positions where actual differs from predicted.
  • The final slice limits the gallery to the first eight mistakes.
  • Each selected feature row is reshaped into its original 8 by 8 image.
  • Each title shows the true digit beside the model prediction.
  • The final print statements confirm the three new result files saved by this step.
  • Save digit_workflow.py.

Before you run this, consider which digit pairs you expect to appear in the gallery.

  • Run the completed workflow with this command:
python digit_workflow.py

What should you see?

The terminal prints precision, recall, F1 score, and support for each digit. It finishes with confirmation lines for the three new result files.

The project folder now contains digits_preview.png, classification_report.txt, confusion_matrix.png, and misclassified_digits.png.

  • In Visual Studio Code, open misclassified_digits.png from the Explorer sidebar.
  • Confirm that the gallery shows eight grayscale digit images.
  • Read each title to compare its true digit with its predicted digit.
  • Confirm that all four result artifacts appear in the Explorer sidebar.

Gallery missing or blank?

Check that mistake_indices compares actual with predicted. Confirm that each selected row uses reshape(8, 8) before it is displayed.

Check that the gallery filename is misclassified_digits.png inside savefig().

Ask for help with the misclassification gallery.

✔️ Awesome, I've got everything!

Your file now matches the completed model evaluation workflow.

ⓧ I'd like to double check the full code

  • Compare your digit_workflow.py file with this completed version:
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import ConfusionMatrixDisplay, accuracy_score, classification_report
from sklearn.model_selection import cross_val_score, train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler


digits = load_digits()
X = digits.data
y = digits.target

print(f"Samples: {X.shape[0]}")
print(f"Features per sample: {X.shape[1]}")
print(f"Original image shape: {digits.images[0].shape}")

preview_figure, preview_axes = plt.subplots(2, 5, figsize=(10, 4))
for axis, image, label in zip(preview_axes.flat, digits.images[:10], y[:10]):
    axis.imshow(image, cmap="gray")
    axis.set_title(f"Label: {label}")
    axis.set_axis_off()
preview_figure.suptitle("Handwritten digits and their labels")
plt.tight_layout()
preview_figure.savefig("digits_preview.png", dpi=150, bbox_inches="tight")
plt.close(preview_figure)
print("Saved digits_preview.png")

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.25,
    random_state=42,
    stratify=y,
)
print(f"Training samples: {len(X_train)}")
print(f"Test samples: {len(X_test)}")

baseline = DummyClassifier(strategy="most_frequent")
baseline.fit(X_train, y_train)
baseline_predictions = baseline.predict(X_test)
baseline_accuracy = accuracy_score(y_test, baseline_predictions)
print(f"Baseline accuracy: {baseline_accuracy:.3f}")

model = make_pipeline(
    StandardScaler(),
    LogisticRegression(max_iter=1000),
)

cross_validation_scores = cross_val_score(model, X_train, y_train, cv=5)
formatted_fold_scores = ", ".join(
    f"{score:.3f}" for score in cross_validation_scores
)
mean_cross_validation_score = sum(cross_validation_scores) / len(
    cross_validation_scores
)
print(f"Cross-validation fold scores: {formatted_fold_scores}")
print(f"Mean cross-validation accuracy: {mean_cross_validation_score:.3f}")

model.fit(X_train, y_train)
predictions = model.predict(X_test)
model_accuracy = accuracy_score(y_test, predictions)
print(f"Model test accuracy: {model_accuracy:.3f}")

report = classification_report(y_test, predictions)
print("Classification report:")
print(report)
with open("classification_report.txt", "w", encoding="utf-8") as report_file:
    report_file.write(report)

confusion_display = ConfusionMatrixDisplay.from_predictions(
    y_test,
    predictions,
    colorbar=False,
)
confusion_display.figure_.suptitle("Digit confusion matrix")
plt.tight_layout()
confusion_display.figure_.savefig(
    "confusion_matrix.png",
    dpi=150,
    bbox_inches="tight",
)
plt.close(confusion_display.figure_)

mistake_indices = [
    index
    for index, (actual, predicted) in enumerate(zip(y_test, predictions))
    if actual != predicted
][:8]

mistake_figure, mistake_axes = plt.subplots(2, 4, figsize=(10, 5))
for axis in mistake_axes.flat:
    axis.set_axis_off()
for axis, index in zip(mistake_axes.flat, mistake_indices):
    axis.imshow(X_test[index].reshape(8, 8), cmap="gray")
    axis.set_title(f"True: {y_test[index]} | Pred: {predictions[index]}")
mistake_figure.suptitle("Misclassified digits")
plt.tight_layout()
mistake_figure.savefig(
    "misclassified_digits.png",
    dpi=150,
    bbox_inches="tight",
)
plt.close(mistake_figure)

print("Saved classification_report.txt")
print("Saved confusion_matrix.png")
print("Saved misclassified_digits.png")

What does the completed file contain?

The completed file preserves the dataset preview, split, baseline, pipeline, cross-validation, and test accuracy from earlier steps. It adds the classification report, confusion matrix, and misclassification gallery from this step.

You have turned one model score into evidence you can inspect. Your workflow now shows where the classifier succeeds and where its predictions break down.

Secret mission

Find the Most Confusing Digit Pair

Your confusion matrix already shows every classification error. Extend the workflow so its final line automatically names the most common true-digit to predicted-digit mismatch.

Clean Up Your Resources

Clean Up Your Resources

Your next choice is whether to keep your local resources available, pause your work, or delete the project entirely. Everything runs locally. There are no ongoing costs.

Resources you used:

  • Project source files: `digit_workflow.py` contains your complete machine learning workflow. `requirements.txt` records the project dependencies.
  • Virtual environment: virtual environment inside `ml-digit-basics` holds scikit-learn `1.9.1` plus Matplotlib `3.11.2`.
  • Generated evidence: `digits_preview.png`, `classification_report.txt`, `confusion_matrix.png`, and `misclassified_digits.png` show the workflow's results.

Keep everything running

No action is needed. Choose this option if you plan to rerun the classifier or continue investigating its prediction errors.

  • Keep the `ml-digit-basics` folder on your Desktop.
  • Keep the project environment available in the integrated terminal.
  • Use the saved reports and images when you want to review the model's performance.
  • Leave every resource in place without worrying about ongoing costs.

Pause - I'll come back to this later

There is no server or background process to stop. Your project files can remain on your Desktop until you return.

  • Leave every file inside `ml-digit-basics` unchanged.
  • Deactivate the project environment by running this command:
deactivate

What does this command do?

The `deactivate` command leaves the project environment in your current terminal session. Your installed packages remain inside the project folder.

  • Close Visual Studio Code when you finish reviewing the project.

Delete - I don't want to use this again

Deleting the folder permanently removes your code, environment, reports, and images. The commands below target only `ml-digit-basics` on your Desktop.

  • Remove the complete project safely by running these commands:
deactivate
cd ~/Desktop
rm -rf ml-digit-basics

What do these commands do?

  • The `deactivate` command leaves the active project environment.
  • The `cd` command returns your terminal to the Desktop.
  • The `rm` command permanently removes the `ml-digit-basics` folder plus everything inside it.
  • Confirm that the project folder is gone by listing the Desktop contents:
ls

What does this check do?

The `ls` command lists the folders and files on your Desktop. This gives you a visible cleanup check.

You should no longer see `ml-digit-basics` in the output. Your source files, project environment, reports, and images are now removed.

Nice Work!

Nice Work!

You made it! Your program now carries handwritten digits through a complete supervised machine learning workflow with evidence you can inspect.

What you learned:

  • Built a labeled digit preview that connects each sample to 64 features. You matched each image with its label.
  • Established a deliberately weak baseline before trusting the real classifier. You evaluated a scaled logistic regression pipeline with five-fold cross-validation. You kept the test set untouched for the final accuracy check.
  • Turned a single accuracy score into model error evidence. You read precision, recall, F1 score, and support in classification_report.txt. You inspected class-level errors in confusion_matrix.png. You reviewed real mistakes in misclassified_digits.png.
  • Secret Mission: Automated most-common confusion detection by scanning off-diagonal counts. The final line names the true digit, wrong prediction, and error count.

Ready to quiz yourself?