Build a Wine Classifier with scikit-learn

Train and compare 3 ML algorithms to classify wines by chemical properties.

Introduction

30 Second Summary

Every time you ask your email app to filter spam or your phone to recognize a face, a machine learning classifier is making that decision. It learns patterns from examples, then applies those patterns to new data it has never seen before.

In this project, you will build a complete machine learning pipeline that classifies wines into three categories based on their chemical properties. You will train three different algorithms, compare their performance, and make predictions on new data.

What You'll Build

A Python script that loads a real wine dataset, trains multiple classification models, evaluates them with industry-standard metrics, and predicts wine types from chemical measurements.

By the end of this project, you'll have:

  • A working ML classification pipeline that predicts wine types from 13 chemical properties.
  • Trained and compared three different algorithms with honest evaluation on unseen data.
  • A clear mental model of the end-to-end ML workflow used in every supervised learning project: load, explore, split, train, evaluate, predict.
  • Secret Mission: Add cross-validation and feature importance analysis to identify which chemical properties matter most for classification.

Are there any prerequisites?

You should be comfortable with Python basics like variables, loops, and functions. You do not need any prior machine learning experience.

Set Up Your ML Environment

Every machine learning project starts with the same foundation: a clean, isolated environment with the right libraries installed.

In this step, you will create a project folder, set up a virtual environment, and install the four core ML libraries you need. Then you will create your main Python file with all the imports ready to go.

In this step, get ready to:
  • Create a project folder and virtual environment.
  • Install scikit-learn, pandas, numpy, and matplotlib.
  • Create the project file with all imports.
Create a project folder and virtual environment

A virtual environment keeps your project's packages separate from the rest of your system. This prevents version conflicts between different projects.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir wine-classifier
cd wine-classifier
  • Confirm the folder exists by running:
ls ~/Desktop

You should see wine-classifier in the output.

  • Create a virtual environment by running this command:
python3 -m venv venv

What does this command do?

python3 -m venv venv tells Python to create a virtual environment in a folder called venv inside your project directory.

This gives you an isolated space where pip install puts packages without affecting your system Python.

  • Activate the virtual environment by running the command for your operating system:

macOS

source venv/bin/activate

Windows

.\venv\Scripts\activate

You should see (venv) appear at the start of your terminal prompt. This confirms the virtual environment is active.

Don't see (venv) in your prompt?

On Windows, if you see a permission error in PowerShell, run Set-ExecutionPolicy RemoteSigned -Scope CurrentUser first, then try the activate command again.

On macOS, make sure you typed source before the path. Without it, the command will not work.

help me activate my Python virtual environment

  • Install the four ML libraries by running this command:
pip install scikit-learn pandas numpy matplotlib

What are these libraries?

  • scikit-learn provides machine learning algorithms and tools for training, evaluating, and making predictions.
  • pandas gives you DataFrames for loading and exploring tabular data.
  • numpy handles numerical arrays and math operations under the hood.
  • matplotlib creates charts and visualizations from your data.

The install may take a minute. Once it finishes, you should see a success message ending with "Successfully installed" followed by a list of packages.

Seeing an install error?

If you see an error mentioning an incompatible Python version, your Python may be older than 3.11. scikit-learn 1.9.0 requires Python 3.11 or higher.

Check your version with python3 --version. If it is below 3.11, visit python.org to download and install the latest Python, then recreate your virtual environment.

help me fix my pip install error for scikit-learn

  • Verify that scikit-learn installed correctly by running:
python -c "import sklearn; print(sklearn.__version__)"

You should see a version number like 1.9.0 printed in the terminal.

Create the project file

Now you need to open your project folder in VS Code and create the main Python file.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code and press Enter to open it.
  • In VS Code, click File in the top menu bar.
  • Click Open Folder.
  • Navigate to your Desktop and select the wine-classifier folder, then click Open.
  • Create a new file by pressing Cmd+N (macOS) or Ctrl+N (Windows).
  • Save the file immediately with Cmd+S (macOS) or Ctrl+S (Windows).
  • Name it wine_classifier.py and save it in the wine-classifier folder.
  • Add the following import statements to wine_classifier.py:
# Wine Classifier - Machine Learning Pipeline
# Classifies wines into 3 categories based on chemical properties

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix

What do these imports do?

  • numpy and pandas handle data loading and manipulation.
  • matplotlib.pyplot creates visualizations of your results.
  • load_wine gives you a built-in wine dataset with 178 samples and 13 chemical features.
  • train_test_split divides your data into training and test sets so you can evaluate honestly.
  • The three classifiers (DecisionTreeClassifier, RandomForestClassifier, KNeighborsClassifier) are the three algorithms you will train and compare.
  • accuracy_score, classification_report, and confusion_matrix measure how well your models perform.
  • Save wine_classifier.py with Cmd+S (macOS) or Ctrl+S (Windows).

✔️ Awesome, I've got everything!

Great. Make sure you have saved the file before moving on.

ⓧ I'd like to double check the full code

Here is the complete wine_classifier.py file so far:

# Wine Classifier - Machine Learning Pipeline
# Classifies wines into 3 categories based on chemical properties

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix
Verify everything works

The final check is to run your script and confirm that all the imports resolve without errors.

  • In your terminal (make sure you are still in the wine-classifier folder with (venv) active), run:
python wine_classifier.py

You should see no output and no errors. The script exits cleanly because the imports loaded successfully but there is no executable code yet.

Seeing a ModuleNotFoundError?

This means a package did not install correctly. Check that (venv) appears in your terminal prompt. If it does not, activate the virtual environment again and re-run pip install scikit-learn pandas numpy matplotlib.

If the error mentions a specific package, try installing just that one package again (for example, pip install scikit-learn).

help me fix my ModuleNotFoundError in wine_classifier.py

Your environment is ready. Next up, you will load the wine dataset and explore what is inside it.

Load and Explore the Wine Dataset

Your environment is ready and your imports are confirmed working. Now it is time to get some data into your pipeline.

Before you can train any machine learning model, you need data to learn from. In this step, you will load a real wine chemistry dataset, explore its structure, and visualize how balanced the classes are.

In this step, get ready to:
  • Load the wine dataset from scikit-learn into Python variables.
  • Explore the data with summary statistics and class counts.
  • Visualize the class distribution as a bar chart.
Load the wine dataset from scikit-learn

scikit-learn ships with several built-in datasets designed for learning. The Wine dataset contains chemical measurements of 178 wines from three different cultivators in Italy.

Each wine has 13 measured properties (alcohol content, color intensity, flavonoids, and more). Your classifier will learn to predict which cultivator grew the wine based on those properties.

  • In wine_classifier.py, find the last import line at the bottom of the file.
  • Add the following code below it to load the dataset and print an overview:
# --- Step 2: Load and Explore the Data ---

# Load the wine dataset
wine = load_wine()
X = wine.data
y = wine.target
feature_names = wine.feature_names
target_names = wine.target_names

# Convert to DataFrame for exploration
df = pd.DataFrame(X, columns=feature_names)
df['target'] = y

print("=== Wine Dataset Overview ===")
print(f"Samples: {X.shape[0]}, Features: {X.shape[1]}")
print(f"Feature names: {list(feature_names)}")
print(f"Target classes: {list(target_names)}")
print()

What does this code do?

load_wine() returns a Bunch object (a dictionary-like container) with data, target, feature_names, and target_names attributes.

X holds the 178 rows of measurements (one per wine). y holds the class label (0, 1, or 2) for each wine. Converting everything into a pandas DataFrame makes it easier to explore with built-in summary methods.

  • Save wine_classifier.py.
  • Run the script in your terminal by running:
python wine_classifier.py

You should see output showing 178 samples, 13 features, the list of feature names, and three target classes (class_0, class_1, class_2).

Seeing a NameError?

If you see NameError: name 'load_wine' is not defined, make sure the import from sklearn.datasets import load_wine is still at the top of your file.

If you see ModuleNotFoundError, confirm your virtual environment is activated. You should see (venv) at the start of your terminal prompt.

help me debug my wine_classifier.py script

Explore the data with summary statistics

Before training a model, you need to understand what your data looks like. Summary statistics reveal the range and spread of each feature, while class counts tell you whether one class dominates the dataset.

  • Add the following code below the print() statement you just added:
# Summary statistics
print("=== Summary Statistics ===")
print(df.describe().round(2))
print()

# Class distribution
print("=== Class Distribution ===")
class_counts = df['target'].value_counts().sort_index()
for i, count in enumerate(class_counts):
    print(f"  {target_names[i]}: {count} samples")
print()

What does this code do?

df.describe() prints the mean, standard deviation, min, max, and quartiles for every numeric column. This helps you spot features with wildly different scales.

value_counts() counts how many samples belong to each class. A balanced dataset (roughly equal class sizes) is easier for a classifier to learn from. Heavily imbalanced data can trick a model into always predicting the majority class.

  • Save wine_classifier.py and run it again:
python wine_classifier.py

You should now see a statistics table and a class breakdown showing class_0 with 59 samples, class_1 with 71 samples, and class_2 with 48 samples.

Not seeing the class distribution output?

Make sure the new code is placed after the first print() block, not inside it. Each section should be at the same indentation level (no extra indentation).

help me fix my data exploration code

Visualize the class distribution

Numbers are useful, but a chart makes class balance instantly obvious. You will save a bar chart as a PNG file so you can reference it later.

  • Add the following code below the class distribution print block:
# Visualize class distribution
plt.figure(figsize=(8, 5))
plt.bar(target_names, class_counts.values, color=['#2196F3', '#4CAF50', '#FF9800'])
plt.title('Wine Dataset - Class Distribution')
plt.xlabel('Wine Class')
plt.ylabel('Number of Samples')
plt.tight_layout()
plt.savefig('class_distribution.png', dpi=100)
plt.close()
print("Saved: class_distribution.png")
print()

What does this code do?

plt.figure() creates a new chart canvas. plt.bar() draws one bar per class, colored blue, green, and orange.

plt.savefig() writes the chart to a PNG file instead of displaying it in a window. plt.close() frees memory by closing the figure after saving. This pattern is standard for scripts that generate charts without a GUI.

  • Save wine_classifier.py and run it again:
python wine_classifier.py
  • You should see Saved: class_distribution.png at the bottom of the output.
  • Confirm the file was created by running:
ls

You should see class_distribution.png alongside wine_classifier.py and the venv folder.

Don't see class_distribution.png?

Check that import matplotlib.pyplot as plt is still at the top of your file. If matplotlib is not installed, you will see a ModuleNotFoundError before the script reaches the chart code.

Also confirm your terminal is inside the wine-classifier folder. The PNG saves to whatever directory the script runs from.

help me figure out why my chart file is not being saved

✔️ Awesome, I've got everything!

Great. Double check you have saved wine_classifier.py and that class_distribution.png exists in your project folder.

ⓧ I'd like to double check the full code

Here is the complete wine_classifier.py file as it should look at the end of this step:

# Wine Classifier - Machine Learning Pipeline
# Classifies wines into 3 categories based on chemical properties

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix

# --- Step 2: Load and Explore the Data ---

# Load the wine dataset
wine = load_wine()
X = wine.data
y = wine.target
feature_names = wine.feature_names
target_names = wine.target_names

# Convert to DataFrame for exploration
df = pd.DataFrame(X, columns=feature_names)
df['target'] = y

print("=== Wine Dataset Overview ===")
print(f"Samples: {X.shape[0]}, Features: {X.shape[1]}")
print(f"Feature names: {list(feature_names)}")
print(f"Target classes: {list(target_names)}")
print()

# Summary statistics
print("=== Summary Statistics ===")
print(df.describe().round(2))
print()

# Class distribution
print("=== Class Distribution ===")
class_counts = df['target'].value_counts().sort_index()
for i, count in enumerate(class_counts):
    print(f"  {target_names[i]}: {count} samples")
print()

# Visualize class distribution
plt.figure(figsize=(8, 5))
plt.bar(target_names, class_counts.values, color=['#2196F3', '#4CAF50', '#FF9800'])
plt.title('Wine Dataset - Class Distribution')
plt.xlabel('Wine Class')
plt.ylabel('Number of Samples')
plt.tight_layout()
plt.savefig('class_distribution.png', dpi=100)
plt.close()
print("Saved: class_distribution.png")
print()

You now have a clear picture of what your data looks like. Next up, you will split this dataset into training and test sets, then train your first classifier to predict wine types.

Split Data and Train Your First Model

You now have a clear picture of your dataset. You know you have 178 wines, 13 chemical measurements per wine, and 3 balanced classes. The next question is: how do you actually teach a computer to classify wines from these numbers?

The answer is a two-part process. First, you hold back some data the model has never seen so you can test it honestly. Then, you train your first algorithm on the remaining data and measure how well it predicts the held-back samples.

In this step, get ready to:
  • Split the dataset into training and test sets with stratification.
  • Train a Decision Tree classifier on the training data.
  • Evaluate your first model's accuracy on unseen data.
Split data into training and test sets

If you train a model on all 178 samples and then test it on those same samples, it can just memorize the answers. That tells you nothing about how it handles new, unseen wines.

The standard solution is a train/test split. You reserve a portion of data exclusively for evaluation.

  • In wine_classifier.py, scroll to the bottom of your file (after the print("Saved: class_distribution.png") line).
  • Add the train/test split by pasting this code at the bottom:
# --- Step 3: Split Data and Train First Model ---

# Split into train (80%) and test (20%)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)
print(f"Training samples: {X_train.shape[0]}, Test samples: {X_test.shape[0]}")
print()

What does this code do?

The train_test_split function shuffles your data and divides it into two groups.

  • test_size=0.2 holds back 20% of samples (36 wines) for testing. The other 80% (142 wines) are used for training.
  • random_state=42 locks the shuffle so you get the same split every time you run the script. This makes your results reproducible.
  • stratify=y ensures both the training and test sets keep the same proportion of each wine class. Without this, you might accidentally put all of one class into training and none into testing.

Why stratify?

Imagine your test set ended up with zero class_2 wines by chance. Your accuracy score would completely ignore how the model handles that class. Stratification prevents this by preserving the original class ratios in both halves of the split.

  • Save wine_classifier.py.
  • Run the script in your terminal by running:
python wine_classifier.py
  • Near the bottom of the output, you should see Training samples: 142, Test samples: 36.

Not seeing the training/test split output?

  • Make sure you pasted the new code below the existing exploration code, not inside or above it.
  • Check that you saved the file before running it.
  • If you see a NameError for X or y, confirm the data loading code from the previous step is still intact above your new code.

My train/test split is throwing an error. What's wrong?

Train a Decision Tree classifier

A Decision Tree learns a series of yes/no questions about your features. For example: "Is alcohol content above 13?" If yes, go left. If no, go right. It keeps splitting until it can classify each sample.

  • In wine_classifier.py, add the Decision Tree training code directly below the print() line you just added:
# Train Decision Tree
dt_model = DecisionTreeClassifier(random_state=42)
dt_model.fit(X_train, y_train)
dt_predictions = dt_model.predict(X_test)
dt_accuracy = accuracy_score(y_test, dt_predictions)
print(f"Decision Tree Accuracy: {dt_accuracy:.4f}")

What does this code do?

  • DecisionTreeClassifier(random_state=42) creates a fresh, untrained Decision Tree model.
  • .fit(X_train, y_train) trains the model by finding the best split questions from the 142 training samples.
  • .predict(X_test) asks the trained model to classify the 36 test wines it has never seen.
  • accuracy_score(y_test, dt_predictions) compares the model's guesses to the actual answers and calculates the percentage correct.
  • Save wine_classifier.py and run it again:
python wine_classifier.py
  • At the very bottom of the output, you should see a line like Decision Tree Accuracy: 0.9444 (your exact number may vary slightly depending on your scikit-learn version).

Seeing a NameError for X_train?

  • The Decision Tree code must appear below the train_test_split code. If it is above or disconnected, X_train will not exist yet.
  • Check for typos in variable names. Python is case-sensitive, so x_train is not the same as X_train.

Help me debug my Decision Tree training code.

What just happened?

Your model learned patterns from 142 training samples. It was then evaluated on 36 samples it had never seen before. The accuracy score tells you what percentage of those 36 predictions were correct.

This is the core loop of supervised machine learning: train on known data, test on unseen data. If you tested on the same data you trained on, the model could just memorize answers and score perfectly without actually learning useful patterns.

✔️ Awesome, I've got everything!

Great. Double check you have saved wine_classifier.py before moving on.

ⓧ I'd like to double check the full code

Here is the complete wine_classifier.py up to this point:

# Wine Classifier - Machine Learning Pipeline
# Classifies wines into 3 categories based on chemical properties

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix

# --- Step 2: Load and Explore the Data ---

# Load the wine dataset
wine = load_wine()
X = wine.data
y = wine.target
feature_names = wine.feature_names
target_names = wine.target_names

# Convert to DataFrame for exploration
df = pd.DataFrame(X, columns=feature_names)
df['target'] = y

print("=== Wine Dataset Overview ===")
print(f"Samples: {X.shape[0]}, Features: {X.shape[1]}")
print(f"Feature names: {list(feature_names)}")
print(f"Target classes: {list(target_names)}")
print()

# Summary statistics
print("=== Summary Statistics ===")
print(df.describe().round(2))
print()

# Class distribution
print("=== Class Distribution ===")
class_counts = df['target'].value_counts().sort_index()
for i, count in enumerate(class_counts):
    print(f"  {target_names[i]}: {count} samples")
print()

# Visualize class distribution
plt.figure(figsize=(8, 5))
plt.bar(target_names, class_counts.values, color=['#2196F3', '#4CAF50', '#FF9800'])
plt.title('Wine Dataset - Class Distribution')
plt.xlabel('Wine Class')
plt.ylabel('Number of Samples')
plt.tight_layout()
plt.savefig('class_distribution.png', dpi=100)
plt.close()
print("Saved: class_distribution.png")
print()

# --- Step 3: Split Data and Train First Model ---

# Split into train (80%) and test (20%)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)
print(f"Training samples: {X_train.shape[0]}, Test samples: {X_test.shape[0]}")
print()

# Train Decision Tree
dt_model = DecisionTreeClassifier(random_state=42)
dt_model.fit(X_train, y_train)
dt_predictions = dt_model.predict(X_test)
dt_accuracy = accuracy_score(y_test, dt_predictions)
print(f"Decision Tree Accuracy: {dt_accuracy:.4f}")

Your first model is trained and evaluated. Next, you will train two more algorithms on the same data and compare all three to find the best performer.

Train and Compare Multiple Models

You've trained your first model and have a Decision Tree accuracy score. One model is a start, but how do you know it's the best algorithm for this data?

Different algorithms learn patterns in fundamentally different ways. In this step, you'll train a Random Forest and a K-Nearest Neighbors classifier, then compare all three head-to-head to find the strongest performer.

In this step, get ready to:
  • Train a Random Forest classifier on the wine data.
  • Train a K-Nearest Neighbors classifier on the same split.
  • Compare all three models and visualize the results.

Why three different algorithms?

Each algorithm approaches classification differently. Decision Trees split data with simple rules. Random Forests combine many trees for a more robust answer. K-Nearest Neighbors classifies based on what's nearby in the feature space.

Comparing them teaches you that algorithm choice matters. The same data can produce very different results depending on which model you pick.

Train a Random Forest classifier

A Random Forest builds many Decision Trees on random subsets of the data, then combines their votes for a final prediction. This ensemble approach typically outperforms a single tree.

  • In wine_classifier.py, add the following code at the bottom of your file, below the Decision Tree accuracy line:
# --- Step 4: Train and Compare Multiple Models ---

# Train Random Forest
rf_model = RandomForestClassifier(n_estimators=100, random_state=42)
rf_model.fit(X_train, y_train)
rf_predictions = rf_model.predict(X_test)
rf_accuracy = accuracy_score(y_test, rf_predictions)
print(f"Random Forest Accuracy: {rf_accuracy:.4f}")

What does this code do?

  • n_estimators=100 tells the Random Forest to build 100 individual decision trees and average their predictions.
  • random_state=42 locks the randomness so you get the same result every time you run the script.
  • The fit/predict/score pattern is identical to the Decision Tree. Every scikit-learn model follows this same interface.
  • Save wine_classifier.py.
  • Run the script in your terminal:
python wine_classifier.py
  • Scroll to the bottom of the output. You should see Random Forest Accuracy: followed by a decimal number (likely higher than the Decision Tree).

Not seeing the Random Forest accuracy?

  • Make sure the new code is placed after the last line of your existing file, not inside an existing block.
  • Check that RandomForestClassifier is already imported at the top of the file. It should be from the previous step.
  • help me debug why my Random Forest isn't printing accuracy
Train a K-Nearest Neighbors classifier

K-Nearest Neighbors takes a completely different approach. Instead of learning rules, it classifies new samples by finding the 5 closest training samples and taking a majority vote.

  • Add the following code below the Random Forest accuracy print:
# Train K-Nearest Neighbors
knn_model = KNeighborsClassifier(n_neighbors=5)
knn_model.fit(X_train, y_train)
knn_predictions = knn_model.predict(X_test)
knn_accuracy = accuracy_score(y_test, knn_predictions)
print(f"KNN (k=5) Accuracy: {knn_accuracy:.4f}")
print()

What does this code do?

  • n_neighbors=5 means the model looks at the 5 nearest data points to make each prediction.
  • KNN doesn't build an internal model during training. It memorizes the training data and does all the work at prediction time.
  • The extra print() adds a blank line in the output to separate the individual results from the comparison section coming next.
  • Save wine_classifier.py and run the script again:
python wine_classifier.py
  • At the bottom of the output, you should now see KNN (k=5) Accuracy: followed by a decimal number.

Seeing a NameError for KNeighborsClassifier?

  • Confirm that from sklearn.neighbors import KNeighborsClassifier exists in your imports at the top of the file.
  • Check spelling. The class is KNeighborsClassifier with a capital K and capital C.
  • help me fix my KNN import error
Compare all three models

You now have three accuracy scores. Printing them side by side makes it easy to see which algorithm handles the wine data best.

  • Add the following comparison code below the KNN section:
# Compare models
print("=== Model Comparison ===")
models = ['Decision Tree', 'Random Forest', 'KNN (k=5)']
accuracies = [dt_accuracy, rf_accuracy, knn_accuracy]
for name, acc in zip(models, accuracies):
    print(f"  {name}: {acc:.4f}")
best_idx = np.argmax(accuracies)
print(f"\nBest model: {models[best_idx]} ({accuracies[best_idx]:.4f})")
print()

What does this code do?

  • models and accuracies are parallel lists. Each model name lines up with its accuracy score.
  • np.argmax(accuracies) returns the index of the highest value. This automatically picks the winner without hardcoding.
  • The zip() function pairs each name with its accuracy for clean printing.
  • Save wine_classifier.py and run the script:
python wine_classifier.py
  • At the bottom of the output, you should see a === Model Comparison === header followed by three accuracy scores and a "Best model" line.

Seeing a NameError for dt_accuracy?

  • Make sure your Step 3 code (the Decision Tree section) is still intact above this new code. The comparison references variables defined there.
  • Check that import numpy as np is at the top of your file. The np.argmax() call needs it.
  • help me debug my model comparison code

Now add a bar chart to visualize the comparison. A visual makes it instantly clear which model leads.

  • Add the following visualization code directly below the comparison print section:
# Visualize model comparison
plt.figure(figsize=(8, 5))
colors = ['#2196F3', '#4CAF50', '#FF9800']
plt.bar(models, accuracies, color=colors)
plt.title('Model Accuracy Comparison')
plt.xlabel('Model')
plt.ylabel('Accuracy')
plt.ylim(0.5, 1.05)
for i, acc in enumerate(accuracies):
    plt.text(i, acc + 0.01, f'{acc:.3f}', ha='center', fontsize=11)
plt.tight_layout()
plt.savefig('model_comparison.png', dpi=100)
plt.close()
print("Saved: model_comparison.png")
print()

What does this code do?

  • plt.ylim(0.5, 1.05) sets the y-axis range so differences between models are visible. Starting at 0 would compress all bars to the top.
  • The plt.text() loop places the exact accuracy value above each bar for easy reading.
  • plt.close() frees memory after saving. Without it, matplotlib keeps the figure open in the background.
  • Save wine_classifier.py and run the script:
python wine_classifier.py
  • You should see Saved: model_comparison.png at the bottom of the output.
  • Confirm the file exists by running:
ls
  • You should see model_comparison.png alongside class_distribution.png and wine_classifier.py in the output.

Don't see model_comparison.png?

  • Make sure you saved wine_classifier.py before running the script.
  • Check that import matplotlib.pyplot as plt is still at the top of your file.
  • Confirm you are running the script from inside the wine-classifier folder. The image saves to whatever directory you run the script from.
  • help me figure out why model_comparison.png isn't being created

✔️ Awesome, I've got everything!

Great. Double check you've saved your file, then move on to the next step.

ⓧ I'd like to double check the full code

Here is the complete wine_classifier.py up to this point:

# Wine Classifier - Machine Learning Pipeline
# Classifies wines into 3 categories based on chemical properties

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix

# --- Step 2: Load and Explore the Data ---

# Load the wine dataset
wine = load_wine()
X = wine.data
y = wine.target
feature_names = wine.feature_names
target_names = wine.target_names

# Convert to DataFrame for exploration
df = pd.DataFrame(X, columns=feature_names)
df['target'] = y

print("=== Wine Dataset Overview ===")
print(f"Samples: {X.shape[0]}, Features: {X.shape[1]}")
print(f"Feature names: {list(feature_names)}")
print(f"Target classes: {list(target_names)}")
print()

# Summary statistics
print("=== Summary Statistics ===")
print(df.describe().round(2))
print()

# Class distribution
print("=== Class Distribution ===")
class_counts = df['target'].value_counts().sort_index()
for i, count in enumerate(class_counts):
    print(f"  {target_names[i]}: {count} samples")
print()

# Visualize class distribution
plt.figure(figsize=(8, 5))
plt.bar(target_names, class_counts.values, color=['#2196F3', '#4CAF50', '#FF9800'])
plt.title('Wine Dataset - Class Distribution')
plt.xlabel('Wine Class')
plt.ylabel('Number of Samples')
plt.tight_layout()
plt.savefig('class_distribution.png', dpi=100)
plt.close()
print("Saved: class_distribution.png")
print()

# --- Step 3: Split Data and Train First Model ---

# Split into train (80%) and test (20%)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)
print(f"Training samples: {X_train.shape[0]}, Test samples: {X_test.shape[0]}")
print()

# Train Decision Tree
dt_model = DecisionTreeClassifier(random_state=42)
dt_model.fit(X_train, y_train)
dt_predictions = dt_model.predict(X_test)
dt_accuracy = accuracy_score(y_test, dt_predictions)
print(f"Decision Tree Accuracy: {dt_accuracy:.4f}")

# --- Step 4: Train and Compare Multiple Models ---

# Train Random Forest
rf_model = RandomForestClassifier(n_estimators=100, random_state=42)
rf_model.fit(X_train, y_train)
rf_predictions = rf_model.predict(X_test)
rf_accuracy = accuracy_score(y_test, rf_predictions)
print(f"Random Forest Accuracy: {rf_accuracy:.4f}")

# Train K-Nearest Neighbors
knn_model = KNeighborsClassifier(n_neighbors=5)
knn_model.fit(X_train, y_train)
knn_predictions = knn_model.predict(X_test)
knn_accuracy = accuracy_score(y_test, knn_predictions)
print(f"KNN (k=5) Accuracy: {knn_accuracy:.4f}")
print()

# Compare models
print("=== Model Comparison ===")
models = ['Decision Tree', 'Random Forest', 'KNN (k=5)']
accuracies = [dt_accuracy, rf_accuracy, knn_accuracy]
for name, acc in zip(models, accuracies):
    print(f"  {name}: {acc:.4f}")
best_idx = np.argmax(accuracies)
print(f"\nBest model: {models[best_idx]} ({accuracies[best_idx]:.4f})")
print()

# Visualize model comparison
plt.figure(figsize=(8, 5))
colors = ['#2196F3', '#4CAF50', '#FF9800']
plt.bar(models, accuracies, color=colors)
plt.title('Model Accuracy Comparison')
plt.xlabel('Model')
plt.ylabel('Accuracy')
plt.ylim(0.5, 1.05)
for i, acc in enumerate(accuracies):
    plt.text(i, acc + 0.01, f'{acc:.3f}', ha='center', fontsize=11)
plt.tight_layout()
plt.savefig('model_comparison.png', dpi=100)
plt.close()
print("Saved: model_comparison.png")
print()

You now have three trained models and a clear winner. Next up, you'll dig deeper into the best model's predictions to understand exactly where it gets things right and where it stumbles.

Evaluate the Best Model in Depth

You now have three trained models and know which one scores highest on overall accuracy. Accuracy is a great starting point, but it only tells you the total percentage of correct predictions.

In this step, you will dig deeper into how the best model performs on each individual class. You will also use the model to classify a brand-new wine sample, proving the full pipeline works end-to-end.

In this step, get ready to:
  • Generate a classification report with per-class precision, recall, and F1-score.
  • Generate a confusion matrix to see which classes get misclassified.
  • Make a prediction on a new unseen wine sample.
Generate a classification report

Accuracy tells you the overall score, but it hides important details. A model might be great at classifying class_0 wines but terrible at class_2.

A classification report breaks performance down per class using three metrics: precision (of all wines predicted as class X, how many actually were class X), recall (of all actual class X wines, how many did the model catch), and F1-score (the balanced average of both).

  • In wine_classifier.py, scroll to the bottom of your file. Add the following code below the model comparison chart block:
# --- Step 5: Evaluate the Best Model ---

# Use Random Forest for detailed evaluation (typically the best performer)
print("=== Classification Report (Random Forest) ===")
print(classification_report(y_test, rf_predictions, target_names=target_names))

What does this code do?

  • The section comment # --- Step 5: Evaluate the Best Model --- marks this as the evaluation phase of the pipeline.
  • classification_report() computes precision, recall, and F1-score for each of the 3 wine classes. It uses the Random Forest predictions since that model typically scores highest.
  • target_names=target_names labels each row with the human-readable class name instead of a number.
  • Save wine_classifier.py.
  • Run your script in the terminal:
python wine_classifier.py
  • Scroll through the terminal output. Near the bottom you should see a table with columns for precision, recall, f1-score, and support for each wine class.

Not seeing the classification report?

  • Make sure your new code is below the model comparison chart block (after plt.close() and the print("Saved: model_comparison.png") line).
  • Check that you saved the file before running it.
  • If you see a NameError, confirm that rf_predictions and y_test are defined earlier in your file from the previous step.

Why isn't my classification report printing?

Generate a confusion matrix

A confusion matrix shows exactly which classes the model confuses with each other. Diagonal values are correct predictions. Off-diagonal values are mistakes.

  • Add the following code directly below the classification report block in wine_classifier.py:
print("=== Confusion Matrix (Random Forest) ===")
cm = confusion_matrix(y_test, rf_predictions)
print(cm)
print("(Rows = actual class, Columns = predicted class)")
print()

What does this code do?

  • confusion_matrix() produces a 3x3 grid (one row and column per class). The value at row i, column j is the number of wines from class i that were predicted as class j.
  • High numbers on the diagonal mean the model is classifying correctly. Any non-zero off-diagonal value tells you exactly which two classes are getting mixed up.
  • Save wine_classifier.py and run the script again:
python wine_classifier.py
  • You should now see a 3x3 number grid below the classification report. The diagonal values should be the largest numbers in each row.

Seeing zeros on the diagonal?

  • Confirm you are passing rf_predictions (not dt_predictions or knn_predictions) to confusion_matrix().
  • Check that the arguments are in the correct order: y_test first (actual), then rf_predictions second (predicted).

Help me understand my confusion matrix output.

Make a prediction on new data

The whole point of training a model is to use it on data it has never seen before. You will now create a fake wine sample with 13 chemical measurements and ask the model to predict which class it belongs to.

  • Add the following code at the bottom of wine_classifier.py, below the confusion matrix block:
# Make a prediction on new data
print("=== Prediction on New Data ===")
new_wine = np.array([[13.0, 2.0, 2.3, 20.0, 100.0, 2.5, 2.0, 0.3, 1.5, 5.0, 1.0, 3.0, 800.0]])
prediction = rf_model.predict(new_wine)
print(f"Input features: {new_wine[0]}")
print(f"Predicted class: {target_names[prediction[0]]}")
print()
print("Pipeline complete!")

What does this code do?

  • new_wine is a NumPy array with 13 values. Each value represents one of the chemical properties the model was trained on (alcohol, color intensity, flavonoids, etc.).
  • rf_model.predict(new_wine) feeds this sample through the trained Random Forest and returns a class label (0, 1, or 2).
  • The final print confirms the entire pipeline ran successfully from loading data through to making a live prediction.
  • Save wine_classifier.py and run the script one final time:
python wine_classifier.py
  • At the very bottom of the output, you should see the predicted class name for your new wine sample, followed by Pipeline complete!.

Seeing a shape error?

  • Make sure new_wine is a 2D array with double brackets: [[...]]. A single set of brackets creates a 1D array, which scikit-learn rejects.
  • Confirm your array has exactly 13 values. The model expects one value per feature.

Help me fix my prediction error.

✔️ Awesome, I've got everything!

Great. Double check you have saved wine_classifier.py and that your terminal shows the classification report, confusion matrix, predicted class, and Pipeline complete! at the end.

ⓧ I'd like to double check the full code

# Wine Classifier - Machine Learning Pipeline
# Classifies wines into 3 categories based on chemical properties

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix

# --- Step 2: Load and Explore the Data ---

# Load the wine dataset
wine = load_wine()
X = wine.data
y = wine.target
feature_names = wine.feature_names
target_names = wine.target_names

# Convert to DataFrame for exploration
df = pd.DataFrame(X, columns=feature_names)
df['target'] = y

print("=== Wine Dataset Overview ===")
print(f"Samples: {X.shape[0]}, Features: {X.shape[1]}")
print(f"Feature names: {list(feature_names)}")
print(f"Target classes: {list(target_names)}")
print()

# Summary statistics
print("=== Summary Statistics ===")
print(df.describe().round(2))
print()

# Class distribution
print("=== Class Distribution ===")
class_counts = df['target'].value_counts().sort_index()
for i, count in enumerate(class_counts):
    print(f"  {target_names[i]}: {count} samples")
print()

# Visualize class distribution
plt.figure(figsize=(8, 5))
plt.bar(target_names, class_counts.values, color=['#2196F3', '#4CAF50', '#FF9800'])
plt.title('Wine Dataset - Class Distribution')
plt.xlabel('Wine Class')
plt.ylabel('Number of Samples')
plt.tight_layout()
plt.savefig('class_distribution.png', dpi=100)
plt.close()
print("Saved: class_distribution.png")
print()

# --- Step 3: Split Data and Train First Model ---

# Split into train (80%) and test (20%)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)
print(f"Training samples: {X_train.shape[0]}, Test samples: {X_test.shape[0]}")
print()

# Train Decision Tree
dt_model = DecisionTreeClassifier(random_state=42)
dt_model.fit(X_train, y_train)
dt_predictions = dt_model.predict(X_test)
dt_accuracy = accuracy_score(y_test, dt_predictions)
print(f"Decision Tree Accuracy: {dt_accuracy:.4f}")

# --- Step 4: Train and Compare Multiple Models ---

# Train Random Forest
rf_model = RandomForestClassifier(n_estimators=100, random_state=42)
rf_model.fit(X_train, y_train)
rf_predictions = rf_model.predict(X_test)
rf_accuracy = accuracy_score(y_test, rf_predictions)
print(f"Random Forest Accuracy: {rf_accuracy:.4f}")

# Train K-Nearest Neighbors
knn_model = KNeighborsClassifier(n_neighbors=5)
knn_model.fit(X_train, y_train)
knn_predictions = knn_model.predict(X_test)
knn_accuracy = accuracy_score(y_test, knn_predictions)
print(f"KNN (k=5) Accuracy: {knn_accuracy:.4f}")
print()

# Compare models
print("=== Model Comparison ===")
models = ['Decision Tree', 'Random Forest', 'KNN (k=5)']
accuracies = [dt_accuracy, rf_accuracy, knn_accuracy]
for name, acc in zip(models, accuracies):
    print(f"  {name}: {acc:.4f}")
best_idx = np.argmax(accuracies)
print(f"\nBest model: {models[best_idx]} ({accuracies[best_idx]:.4f})")
print()

# Visualize model comparison
plt.figure(figsize=(8, 5))
colors = ['#2196F3', '#4CAF50', '#FF9800']
plt.bar(models, accuracies, color=colors)
plt.title('Model Accuracy Comparison')
plt.xlabel('Model')
plt.ylabel('Accuracy')
plt.ylim(0.5, 1.05)
for i, acc in enumerate(accuracies):
    plt.text(i, acc + 0.01, f'{acc:.3f}', ha='center', fontsize=11)
plt.tight_layout()
plt.savefig('model_comparison.png', dpi=100)
plt.close()
print("Saved: model_comparison.png")
print()

# --- Step 5: Evaluate the Best Model ---

# Use Random Forest for detailed evaluation (typically the best performer)
print("=== Classification Report (Random Forest) ===")
print(classification_report(y_test, rf_predictions, target_names=target_names))

print("=== Confusion Matrix (Random Forest) ===")
cm = confusion_matrix(y_test, rf_predictions)
print(cm)
print("(Rows = actual class, Columns = predicted class)")
print()

# Make a prediction on new data
print("=== Prediction on New Data ===")
new_wine = np.array([[13.0, 2.0, 2.3, 20.0, 100.0, 2.5, 2.0, 0.3, 1.5, 5.0, 1.0, 3.0, 800.0]])
prediction = rf_model.predict(new_wine)
print(f"Input features: {new_wine[0]}")
print(f"Predicted class: {target_names[prediction[0]]}")
print()
print("Pipeline complete!")

Secret mission

Cross-Validation and Feature Importance

Your single train/test split gave you one accuracy number. But how confident can you be in that result? In this mission, you will validate your model more rigorously and discover which chemical properties actually drive the classification.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally, so there are no ongoing costs.

Resources you used:

  • The wine-classifier project folder on your Desktop.
  • A Python virtual environment (venv) inside the project folder.
  • Python packages (scikit-learn, pandas, numpy, matplotlib) installed inside the virtual environment.
  • Generated image files (class_distribution.png, model_comparison.png, feature_importance.png).

Keep everything running

No action needed. Choose this if you are still actively building or want to revisit the project later.

  • Your wine-classifier folder stays on your Desktop with all files intact.
  • The virtual environment preserves your installed packages so you can pick up right where you left off.
  • To resume later, open a terminal in the wine-classifier folder and activate the environment with source venv/bin/activate.

Pause - I'll come back to this later

Deactivate your virtual environment to free up your terminal session. All files stay on disk.

  • If your virtual environment is still active, deactivate it by running:
deactivate
  • Your project folder and all installed packages remain on disk for when you return.
  • To resume, navigate back to ~/Desktop/wine-classifier and run source venv/bin/activate.

Delete - I don't want to use this again

Remove all project files, the virtual environment, and installed packages. This frees up disk space completely.

  • If your virtual environment is still active, deactivate it first:
deactivate
  • Delete the entire project folder (including the virtual environment, script, and generated images) by running:
cd ~/Desktop
rm -rf wine-classifier
  • Confirm the folder is gone by running:
ls ~/Desktop | grep wine-classifier
  • If you see no output, the folder has been successfully removed.

What about Python itself?

Python remains installed on your system. All project-specific packages (scikit-learn, pandas, numpy, matplotlib) lived inside the virtual environment you just deleted, so they are gone too.

Nice Work!

Nice Work!

You just built a complete machine learning classification pipeline from scratch. Your script takes raw chemical data, trains three different algorithms, and predicts wine types on data it has never seen before.

You've learned how to:

  • Load, explore, and visualize a real dataset (178 wines, 13 features, 3 classes) then split it with stratification to prevent data leakage.
  • Train and compare 3 classification algorithms (Decision Tree, Random Forest, KNN) on the same train/test split to see how algorithm choice affects performance.
  • Evaluate your best model with accuracy, a classification report, and a confusion matrix, then make predictions on new unseen data.
  • Secret Mission: Used cross-validation for robust evaluation and identified the most important features driving classification.

Ready to quiz yourself?