Track ML Experiments with MLflow

Train, compare, and register ML models using MLflow experiment tracking.

Introduction

30 Second Summary

When you're building machine learning models, trying different settings and comparing results quickly becomes a mess of spreadsheets and forgotten configurations. Imagine if every experiment you ran was automatically tracked, versioned, and visually comparable.

In this project, you will build a complete experiment tracking system using MLflow that trains multiple wine quality prediction models, logs every run with parameters and metrics, and registers the best model for making predictions on new data.

What You'll Build

You will set up a local MLflow Tracking Server, train three different machine learning models with different algorithms and hyperparameters, compare their performance in a visual dashboard, and register the winning model for inference.

By the end of this project, you'll have:

  • A local MLflow Tracking Server with a SQLite backend and a visual UI for inspecting experiments.
  • An experiment containing multiple logged runs with different hyperparameters, accuracy metrics, and model artifacts you can compare side-by-side.
  • A registered production model loaded back from the model registry and making predictions on new data.
  • Secret Mission: Add automated hyperparameter tuning with Optuna and log every trial as a separate MLflow run.

Are there any prerequisites?

You only need basic Python knowledge (variables, functions, loops). No machine learning or MLOps experience is required.

Set Up Your Environment

Before you can track machine learning experiments, you need the right tools installed on your machine.

In this step, you will install Python, MLflow, and scikit-learn. Then you will start a local MLflow Tracking Server and create the project files you will work with throughout this project.

In this step, get ready to:
  • Install Python, MLflow, and scikit-learn.
  • Start the MLflow Tracking Server and confirm the UI loads.
  • Create your project directory and Python script files.
Install Python, MLflow, and scikit-learn

MLflow requires Python 3.10 or higher. Let's check whether you already have it installed.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal and press Enter to open it.
  • Check your Python version by running this command:
python3 --version

✔️ I see version 3.10 or higher

Your Python version is compatible. Move on to installing MLflow and scikit-learn below.

ⓧ I see an older version

Your system has an older version of Python that won't work with MLflow. You need to install Python 3.10+ alongside it.

  • Visit the official Python downloads page at python.org/downloads.
  • Download and install the latest Python 3 release for your operating system.
  • After installation, close and reopen your Terminal, then run python3 --version again to confirm the new version.

macOS users with a shadowed version

macOS ships with an older Python by default. After installing a newer version, your system may still resolve python3 to the old one. Try running python3.12 --version (or whichever version you installed) to confirm the new binary is available. Use that binary throughout this project if python3 still resolves to the old version.

ⓧ Command not found

Python is not installed yet. Let's install it.

  • Visit the official Python downloads page at python.org/downloads.
  • Download and install the latest Python 3 release for your operating system.
  • After installation, close and reopen your Terminal, then run python3 --version to confirm it works.

Now install MLflow and scikit-learn using pip.

  • Install MLflow by running:
pip install mlflow
  • Install scikit-learn by running:
pip install scikit-learn

What are these packages?

MLflow is an open-source platform for tracking machine learning experiments. It logs your model parameters, metrics, and artifacts so you can compare different runs.

scikit-learn is a popular Python library for machine learning. It provides the models and datasets you will train and track in this project.

  • Verify both packages installed correctly by running:
mlflow --version
  • You should see a version number printed (e.g. mlflow, version 3.12.0).
  • Verify scikit-learn by running:
python3 -c "import sklearn; print(sklearn.__version__)"
  • You should see a version number like 1.8.0 printed.

Command not found after installing?

If mlflow --version returns "command not found", your terminal may not have the path to pip-installed binaries. Try closing and reopening your Terminal, then run the command again.

If the issue persists, try running python3 -m mlflow --version instead.

help me fix mlflow not being found after pip install

Start the MLflow Tracking Server

The MLflow Tracking Server is a local web server that stores your experiment data and serves a visual UI for inspecting runs. You will start it now and leave it running in the background while you work.

  • Start the MLflow Tracking Server by running this command:
mlflow server --backend-store-uri sqlite:///mlflow.db --default-artifact-root ./mlartifacts --port 5000

What does this command do?

--backend-store-uri sqlite:///mlflow.db tells MLflow to store all experiment metadata (parameters, metrics, run info) in a local SQLite database file called mlflow.db.

--default-artifact-root ./mlartifacts sets the folder where MLflow stores large files like trained model artifacts.

--port 5000 makes the server listen on port 5000 so you can access the UI in your browser.

  • You should see output indicating the server is running. It will say something like "Listening at: http://127.0.0.1:5000".
  • Open your browser and navigate to http://localhost:5000.
  • You should see the MLflow UI with an empty experiments list.

Keep this terminal open

The MLflow server needs to keep running for the rest of this project. Leave this terminal window open and use a new terminal window for all future commands.

Port 5000 already in use?

If you see an error about port 5000 being in use, another process is occupying that port. On macOS, AirPlay Receiver sometimes uses port 5000. Disable it in System Settings > General > AirDrop & Handoff, then try the command again.

help me free up port 5000 so I can start my MLflow server

Create your project directory and script files

Now you need a dedicated folder for your project code. You will create three Python files that you will fill in throughout the next steps.

  • Open a new terminal window (keep the MLflow server running in the other one).
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir mlflow-wine-tracking
cd mlflow-wine-tracking
  • Create the three Python script files you will use in this project by running:
touch train_autolog.py train_manual.py register_and_predict.py
  • Confirm the files exist by running:
ls
  • You should see train_autolog.py, train_manual.py, and register_and_predict.py in the output.

What are these files for?

train_autolog.py will use MLflow's autologging feature to track a model with zero manual effort.

train_manual.py will train additional models with explicit, manual logging for full control over what gets tracked.

register_and_predict.py will register the best model and load it back to make predictions on new data.

Touch command not found on Windows?

The touch command is a macOS/Linux utility. On Windows, use PowerShell and run New-Item train_autolog.py, train_manual.py, register_and_predict.py instead.

help me create empty files on Windows without the touch command

Your environment is ready. Next up, you will write your first model training script and watch MLflow automatically log everything about the run.

Train Your First Model with Autologging

Your MLflow server is running and ready to receive experiment data. The next step is to train a machine learning model and see how MLflow captures every detail of the training run automatically.

MLflow's autologging feature watches your model training and records parameters, metrics, and the model artifact without you writing any logging code. You'll use the scikit-learn wine quality dataset to train a Random Forest classifier and see this in action.

In this step, get ready to:
  • Load the wine quality dataset and split it into training and test sets.
  • Enable MLflow autologging and train a Random Forest classifier.
  • View the auto-logged run in the MLflow UI.
Load the wine dataset and configure MLflow

Before training a model, you need to load your data and tell MLflow where to send the experiment logs. The wine dataset has 178 samples with 13 features, making it a good playground for comparing different models.

  • Open train_autolog.py in your code editor.
  • Add the following imports and MLflow configuration code:
import mlflow
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier

# Connect to the local MLflow Tracking Server
mlflow.set_tracking_uri("http://localhost:5000")

# Create or set the experiment
mlflow.set_experiment("Wine Quality Prediction")

# Load data
X, y = load_wine(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

What does this code do?

  • mlflow.set_tracking_uri("http://localhost:5000") tells your script to send all logging data to the MLflow server you started in Step 1.
  • mlflow.set_experiment("Wine Quality Prediction") creates a named experiment on the server. All runs in this project will be grouped under this name.
  • train_test_split holds back 20% of the data for testing. The random_state=42 ensures you get the same split every time you run the script.
  • Save train_autolog.py.
  • You won't see a change yet. The next chunk adds the training code that produces output.
Enable autologging and train a Random Forest

With your data loaded, one line of code is all it takes to turn on MLflow's automatic tracking for scikit-learn models. Every parameter you pass to the model and every metric it calculates gets captured without any extra effort.

  • In train_autolog.py, add the following code below the train_test_split line:
# Enable autologging for scikit-learn
mlflow.sklearn.autolog()

# Train a Random Forest model (MLflow logs everything automatically)
rf = RandomForestClassifier(n_estimators=100, max_depth=5, random_state=42)
rf.fit(X_train, y_train)

print(f"Random Forest accuracy: {rf.score(X_test, y_test):.4f}")
print("Run logged! Check the MLflow UI at http://localhost:5000")

What does this code do?

  • mlflow.sklearn.autolog() activates automatic logging for all scikit-learn operations that follow. It captures parameters like n_estimators and max_depth, training metrics like accuracy and F1 score, and saves the trained model as an artifact.
  • rf.fit(X_train, y_train) triggers the actual training. This is the moment MLflow detects a model being trained and starts logging.
  • The print statements give you immediate feedback in the terminal, confirming the model's accuracy on the test set.
  • Save train_autolog.py.
  • Run the script in a new terminal tab (keep the MLflow server running in its original tab) by running:
python3 train_autolog.py
  • You should see output similar to:
Random Forest accuracy: 0.9722
Run logged! Check the MLflow UI at http://localhost:5000

Why might my accuracy differ slightly?

Your accuracy should be very close to 0.9722. Small differences can occur due to library version changes across scikit-learn releases. The important thing is that the script ran without errors and reported an accuracy above 0.90.

Seeing a connection error?

  • Make sure your MLflow server is still running in its original terminal tab. If you accidentally closed it, restart it with mlflow server --port 5000.
  • Check that the URL in your script matches exactly: http://localhost:5000 (no trailing slash, no https).
  • Make sure you're running the script from your project directory where train_autolog.py is located.

Why is my MLflow script failing to connect to the server?

✔️ Awesome, I've got everything!

Great. Double check you've saved your file before moving on.

ⓧ I'd like to double check the full code

Here's the complete train_autolog.py file. Make sure yours matches exactly:

import mlflow
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier

# Connect to the local MLflow Tracking Server
mlflow.set_tracking_uri("http://localhost:5000")

# Create or set the experiment
mlflow.set_experiment("Wine Quality Prediction")

# Load data
X, y = load_wine(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Enable autologging for scikit-learn
mlflow.sklearn.autolog()

# Train a Random Forest model (MLflow logs everything automatically)
rf = RandomForestClassifier(n_estimators=100, max_depth=5, random_state=42)
rf.fit(X_train, y_train)

print(f"Random Forest accuracy: {rf.score(X_test, y_test):.4f}")
print("Run logged! Check the MLflow UI at http://localhost:5000")
View the auto-logged run in the MLflow UI

The real power of MLflow is that everything your model just did is now captured in a visual dashboard. You can inspect every parameter, metric, and artifact without writing any extra code.

  • Switch to your browser and open http://localhost:5000.
  • In the left sidebar, click the Wine Quality Prediction experiment.
  • You should see one run listed in the table. Click on the run name to open it.

Inside the run details page, you'll find three key sections that autologging captured for you.

  • Look at the Parameters section. You should see entries like n_estimators, max_depth, and random_state. These are the exact values you passed to the Random Forest constructor.
  • Look at the Metrics section. Autologging captured training metrics like accuracy, F1 score, precision, and recall.
  • Look at the Artifacts section. MLflow saved the entire trained model as an artifact. You'll use this later to load the model back for predictions.

What is autologging?

Autologging is MLflow's zero-effort tracking mode. It hooks into supported libraries (like scikit-learn) and automatically records everything that happens during training.

This is great for getting started quickly. In the next step, you'll learn manual logging, which gives you full control over exactly what gets recorded.

Don't see the Wine Quality Prediction experiment?

  • Refresh the MLflow UI page in your browser.
  • Confirm your script ran successfully. Check your terminal for the accuracy output. If it printed an error instead, re-run the script after fixing the issue.
  • Make sure the set_tracking_uri in your script points to the same port your server is running on (5000).

Why isn't my MLflow experiment showing up in the UI?

You've trained your first model and seen MLflow capture everything automatically. Next up, you'll train two more models with manual logging to get full control over what gets tracked.

Train Multiple Models with Manual Logging

In the previous step, you trained a Random Forest model and let MLflow autolog everything for you. That's great for speed, but sometimes you need finer control over what gets tracked.

In this step, you will train two more models with manual logging. Manual logging lets you choose exactly which parameters, metrics, and artifacts to record. By the end, you will have three runs in your experiment that you can compare side-by-side.

In this step, get ready to:
  • Train a Logistic Regression model with explicit parameter and metric logging.
  • Train a Gradient Boosting model with different hyperparameters using manual logging.
  • Compare all three runs side-by-side in the MLflow UI.
Train a Logistic Regression model with manual logging

Autologging captures everything automatically, but manual logging gives you full control. You decide which metrics matter, which parameters to track, and what name to give the saved model artifact.

  • Open train_manual.py in your code editor.
  • Add the imports and setup code by pasting the following:
import mlflow
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.metrics import accuracy_score, precision_score, recall_score

# Connect to the local MLflow Tracking Server
mlflow.set_tracking_uri("http://localhost:5000")

# Set the experiment
mlflow.set_experiment("Wine Quality Prediction")

# Load data
X, y = load_wine(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

What does this code do?

  • The imports bring in two new model types (LogisticRegression and GradientBoostingClassifier) plus three metric functions.
  • mlflow.set_tracking_uri("http://localhost:5000") connects to the same tracking server you started earlier.
  • mlflow.set_experiment("Wine Quality Prediction") ensures the new runs land in the same experiment as your Random Forest run from the previous step.

You won't run the script yet. The next chunk adds the first training run.

  • Below the data loading code, add the Logistic Regression training block:
# --- Run 1: Logistic Regression ---
with mlflow.start_run(run_name="Logistic Regression"):
    params = {"solver": "lbfgs", "max_iter": 2000, "random_state": 42}
    mlflow.log_params(params)

    lr = LogisticRegression(**params)
    lr.fit(X_train, y_train)

    y_pred = lr.predict(X_test)
    accuracy = accuracy_score(y_test, y_pred)
    precision = precision_score(y_test, y_pred, average="weighted")
    recall = recall_score(y_test, y_pred, average="weighted")

    mlflow.log_metric("accuracy", accuracy)
    mlflow.log_metric("precision", precision)
    mlflow.log_metric("recall", recall)

    mlflow.sklearn.log_model(sk_model=lr, name="logistic_regression_model")
    mlflow.set_tag("model_type", "Logistic Regression")

    print(f"Logistic Regression - Accuracy: {accuracy:.4f}")

What does this code do?

  • mlflow.start_run(run_name="Logistic Regression") opens a new tracked run. Everything inside the with block is recorded under that run.
  • mlflow.log_params(params) logs all the hyperparameters at once so you can see exactly which settings produced this result.
  • mlflow.log_metric() records each evaluation metric individually. You are logging accuracy, precision, and recall.
  • mlflow.sklearn.log_model() saves the trained model as an artifact you can reload later.
  • mlflow.set_tag() adds a searchable label so you can filter runs by model type in the UI.

Why manual logging instead of autologging?

Autologging captures a default set of metrics and parameters chosen by the library integration. Manual logging lets you log custom metrics (like weighted precision), add tags for filtering, and name your model artifacts explicitly. In production workflows, this precision matters.

You won't run the script yet. The next chunk adds the Gradient Boosting model, then you will run everything together.

Train a Gradient Boosting model

Your second manual run uses a completely different algorithm with its own hyperparameters. This gives you a meaningful comparison across model types.

  • Below the Logistic Regression block, add the Gradient Boosting training block:
# --- Run 2: Gradient Boosting ---
with mlflow.start_run(run_name="Gradient Boosting"):
    params = {"n_estimators": 200, "max_depth": 3, "learning_rate": 0.1, "random_state": 42}
    mlflow.log_params(params)

    gb = GradientBoostingClassifier(**params)
    gb.fit(X_train, y_train)

    y_pred = gb.predict(X_test)
    accuracy = accuracy_score(y_test, y_pred)
    precision = precision_score(y_test, y_pred, average="weighted")
    recall = recall_score(y_test, y_pred, average="weighted")

    mlflow.log_metric("accuracy", accuracy)
    mlflow.log_metric("precision", precision)
    mlflow.log_metric("recall", recall)

    mlflow.sklearn.log_model(sk_model=gb, name="gradient_boosting_model")
    mlflow.set_tag("model_type", "Gradient Boosting")

    print(f"Gradient Boosting - Accuracy: {accuracy:.4f}")

print("\nAll runs logged! Compare them in the MLflow UI at http://localhost:5000")

What does this code do?

  • This block follows the same pattern as the Logistic Regression run: log params, train, predict, log metrics, log model, and set a tag.
  • The hyperparameters are different: n_estimators=200, max_depth=3, and learning_rate=0.1. These control how the Gradient Boosting algorithm builds its ensemble of trees.
  • The final print() statement confirms both runs completed successfully.
  • Save train_manual.py.
  • Run the script by executing this command in your terminal:
python train_manual.py
  • You should see output showing the accuracy for both models, followed by the confirmation message.

Seeing a connection error?

  • Make sure your MLflow Tracking Server is still running in its own terminal window. If you closed it, restart it with mlflow server --port 5000.
  • Check that mlflow.set_tracking_uri("http://localhost:5000") is at the top of your script. A missing or incorrect URI means the script cannot find the server.
  • If you see an import error, confirm you installed all packages. Run pip install mlflow scikit-learn again to be safe.

help me debug my train_manual.py connection error

✔️ Awesome, I've got everything!

Great. Double check you have saved train_manual.py before moving on.

ⓧ I'd like to double check the full code

import mlflow
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.metrics import accuracy_score, precision_score, recall_score

# Connect to the local MLflow Tracking Server
mlflow.set_tracking_uri("http://localhost:5000")

# Set the experiment
mlflow.set_experiment("Wine Quality Prediction")

# Load data
X, y = load_wine(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# --- Run 1: Logistic Regression ---
with mlflow.start_run(run_name="Logistic Regression"):
    params = {"solver": "lbfgs", "max_iter": 2000, "random_state": 42}
    mlflow.log_params(params)

    lr = LogisticRegression(**params)
    lr.fit(X_train, y_train)

    y_pred = lr.predict(X_test)
    accuracy = accuracy_score(y_test, y_pred)
    precision = precision_score(y_test, y_pred, average="weighted")
    recall = recall_score(y_test, y_pred, average="weighted")

    mlflow.log_metric("accuracy", accuracy)
    mlflow.log_metric("precision", precision)
    mlflow.log_metric("recall", recall)

    mlflow.sklearn.log_model(sk_model=lr, name="logistic_regression_model")
    mlflow.set_tag("model_type", "Logistic Regression")

    print(f"Logistic Regression - Accuracy: {accuracy:.4f}")

# --- Run 2: Gradient Boosting ---
with mlflow.start_run(run_name="Gradient Boosting"):
    params = {"n_estimators": 200, "max_depth": 3, "learning_rate": 0.1, "random_state": 42}
    mlflow.log_params(params)

    gb = GradientBoostingClassifier(**params)
    gb.fit(X_train, y_train)

    y_pred = gb.predict(X_test)
    accuracy = accuracy_score(y_test, y_pred)
    precision = precision_score(y_test, y_pred, average="weighted")
    recall = recall_score(y_test, y_pred, average="weighted")

    mlflow.log_metric("accuracy", accuracy)
    mlflow.log_metric("precision", precision)
    mlflow.log_metric("recall", recall)

    mlflow.sklearn.log_model(sk_model=gb, name="gradient_boosting_model")
    mlflow.set_tag("model_type", "Gradient Boosting")

    print(f"Gradient Boosting - Accuracy: {accuracy:.4f}")

print("\nAll runs logged! Compare them in the MLflow UI at http://localhost:5000")
Compare all three runs in the MLflow UI

You now have three runs in your experiment: the auto-logged Random Forest from the previous step, plus the two manually-logged runs you just created. The MLflow UI lets you compare them all in one view.

  • Open your browser and go to http://localhost:5000.
  • Click on the Wine Quality Prediction experiment in the left sidebar.
  • You should see three runs listed: Logistic Regression, Gradient Boosting, and the Random Forest run from Step 2.
  • Select all three runs using the checkboxes on the left side of each row.
  • Click the Compare button that appears above the run list.

The comparison view shows parameters and metrics for all selected runs in a table. You can quickly see which model achieved the highest accuracy.

What should I see?

The comparison table displays each run as a column with its logged parameters and metrics in rows. You should see accuracy, precision, and recall metrics for the manually-logged runs. The Random Forest run shows the auto-logged metrics from scikit-learn's autolog integration.

Look at the accuracy row to see which model performed best on the wine quality dataset. Different model types and hyperparameters produce different results.

Don't see three runs?

  • Make sure you ran both train_autolog.py (in Step 2) and train_manual.py (in this step). Each script must complete without errors to log its runs.
  • Refresh the browser page. The UI does not always auto-update when new runs arrive.
  • Confirm you are looking at the Wine Quality Prediction experiment, not the Default experiment.

help me find my missing MLflow runs

You now have a complete experiment with three different models tracked and compared. Next up, you will register the best-performing model and load it back for predictions on new data.

Register and Load the Best Model

You now have three trained models sitting in your MLflow experiment, each with logged metrics. The real power of experiment tracking comes next: programmatically finding the best performer and promoting it to a registered model.

In this step, you will query your experiment runs, register the winning model in the MLflow Model Registry, and load it back to make predictions on new data.

In this step, get ready to:
  • Identify the best-performing model by comparing accuracy in the MLflow UI.
  • Register the best model in the MLflow Model Registry.
  • Load the registered model and run predictions on new data.
Identify the best model in the MLflow UI

Before writing any code, you can visually confirm which model performed best using the MLflow UI.

  • Open the MLflow UI at http://localhost:5000 in your browser.
  • Click on the Wine Quality Prediction experiment in the left sidebar.
  • You should see three runs listed: Random Forest, Logistic Regression, and Gradient Boosting.
  • Click the accuracy column header to sort runs by accuracy in descending order.
  • Note which model has the highest accuracy score.

Why compare in the UI first?

Sorting by accuracy in the UI gives you a quick visual sanity check before automating anything. In a real workflow, you would inspect confusion matrices and other metrics too.

For this project, accuracy is the single metric you will use to pick the winner programmatically in the next substep.

Register the best model programmatically

The MLflow Model Registry is a central store where you can version and manage production models. Your script will query the experiment, find the run with the highest accuracy, and register its model artifact.

  • Install pandas (needed for displaying predictions later) by running this command:
pip install pandas
  • Open register_and_predict.py in your code editor and add the following imports and connection setup:
import mlflow
from mlflow import MlflowClient
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
import pandas as pd

# Connect to the local MLflow Tracking Server
mlflow.set_tracking_uri("http://localhost:5000")

client = MlflowClient()

What does this code do?

`MlflowClient()` gives you a programmatic interface to query experiments, runs, and the model registry. It connects to the same tracking server your training scripts used.

  • Below the client setup, add the code that finds the best run and registers it:
# Find the best run by accuracy in the "Wine Quality Prediction" experiment
experiment = client.get_experiment_by_name("Wine Quality Prediction")
runs = client.search_runs(
    experiment_ids=[experiment.experiment_id],
    order_by=["metrics.accuracy DESC"],
    max_results=1,
)

best_run = runs[0]
print(f"Best run ID: {best_run.info.run_id}")
print(f"Best accuracy: {best_run.data.metrics['accuracy']:.4f}")

# Register the best model
model_uri = f"runs:/{best_run.info.run_id}/model"
registered_model = mlflow.register_model(model_uri, "WineQualityBestModel")
print(f"Model registered as: {registered_model.name}, version: {registered_model.version}")

What does this code do?

`client.search_runs()` queries the experiment and sorts results by accuracy in descending order, returning only the top result.

`mlflow.register_model()` takes the model artifact from the best run (using the runs:/<run_id>/model URI format) and registers it as WineQualityBestModel in the Model Registry with version 1.

Load the registered model and run predictions

A registered model can be loaded by any downstream service using its name and version. This decouples training from inference.

  • Below the registration code in register_and_predict.py, add the inference code:
# Load the registered model for inference
model = mlflow.pyfunc.load_model(f"models:/{registered_model.name}/{registered_model.version}")

# Make predictions on new data
X, y = load_wine(return_X_y=True)
_, X_test, _, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

predictions = model.predict(X_test)

# Display results
feature_names = load_wine().feature_names
result = pd.DataFrame(X_test, columns=feature_names)
result["actual_class"] = y_test
result["predicted_class"] = predictions

print("\nSample predictions:")
print(result[["actual_class", "predicted_class"]].head(10))

What does this code do?

`mlflow.pyfunc.load_model()` loads the registered model using the models:/<name>/<version> URI format. This is the same interface any production service would use to fetch a model.

The script then recreates the same test split (using random_state=42 for reproducibility), runs predictions, and prints the first 10 results comparing actual vs predicted class labels.

  • Save register_and_predict.py.
  • Run the script by executing this command in your terminal:
python register_and_predict.py
  • You should see output showing the best run ID, its accuracy score, a model registration confirmation, and a table of 10 predictions with actual vs predicted class values.
  • Switch to the MLflow UI at http://localhost:5000 and click Models in the top navigation bar.
  • You should see WineQualityBestModel listed with version 1.

Seeing a "RESOURCE_DOES_NOT_EXIST" error?

This usually means the experiment name in your script does not match exactly. Confirm the experiment is named Wine Quality Prediction (case-sensitive) by checking the MLflow UI sidebar.

Also make sure your MLflow Tracking Server is still running in a separate terminal window. If it stopped, restart it with mlflow server --port 5000.

help me debug my register_and_predict.py script

✔️ Awesome, I've got everything!

Great. Double check you have saved register_and_predict.py and confirmed the model appears in the MLflow UI under Models.

ⓧ I'd like to double check the full code

import mlflow
from mlflow import MlflowClient
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
import pandas as pd

# Connect to the local MLflow Tracking Server
mlflow.set_tracking_uri("http://localhost:5000")

client = MlflowClient()

# Find the best run by accuracy in the "Wine Quality Prediction" experiment
experiment = client.get_experiment_by_name("Wine Quality Prediction")
runs = client.search_runs(
    experiment_ids=[experiment.experiment_id],
    order_by=["metrics.accuracy DESC"],
    max_results=1,
)

best_run = runs[0]
print(f"Best run ID: {best_run.info.run_id}")
print(f"Best accuracy: {best_run.data.metrics['accuracy']:.4f}")

# Register the best model
model_uri = f"runs:/{best_run.info.run_id}/model"
registered_model = mlflow.register_model(model_uri, "WineQualityBestModel")
print(f"Model registered as: {registered_model.name}, version: {registered_model.version}")

# Load the registered model for inference
model = mlflow.pyfunc.load_model(f"models:/{registered_model.name}/{registered_model.version}")

# Make predictions on new data
X, y = load_wine(return_X_y=True)
_, X_test, _, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

predictions = model.predict(X_test)

# Display results
feature_names = load_wine().feature_names
result = pd.DataFrame(X_test, columns=feature_names)
result["actual_class"] = y_test
result["predicted_class"] = predictions

print("\nSample predictions:")
print(result[["actual_class", "predicted_class"]].head(10))

You have a fully tracked, registered, and serving-ready ML model. Next up, challenge yourself with the Secret Mission to automate hyperparameter tuning and log every trial.

Secret mission

Add Hyperparameter Tuning with Optuna

You've trained three models with hand-picked hyperparameters. But what if you could automate the search? In this secret mission, you'll use Optuna to run 20 automated trials, each with different Random Forest hyperparameters, and log every single one to MLflow.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your resources, pause them to come back later, or delete them entirely.

Resources you used:

  • MLflow SQLite database (mlflow.db) in your project directory.
  • MLflow artifact store (mlartifacts/ directory) containing logged model artifacts.
  • Project Python scripts (train_autolog.py, train_manual.py, register_and_predict.py).
  • Installed Python packages (mlflow, scikit-learn).

Keep everything running

No action needed. Choose this if you plan to continue experimenting with your models or adding new runs to your experiments.

  • Your experiment data stays in mlflow.db and mlartifacts/ until you decide to remove them.
  • Restart the MLflow server anytime by navigating to your project folder in the terminal and running mlflow server --port 5000.
  • The mlflow and scikit-learn packages remain installed in your Python environment.

Pause - I'll come back to this later

Stop the MLflow server to free up your terminal. Local files cost nothing when the server is stopped.

  • If the MLflow server is still running, press Ctrl+C in the terminal where you started it.
  • Your experiment data and model artifacts remain on disk, ready for when you come back.
  • To resume later, open a terminal in your mlflow-tracking folder and run mlflow server --port 5000 again.

Delete - I don't want to use this again

Remove all project files and optionally uninstall the Python packages to start fresh.

  • If the MLflow server is still running, press Ctrl+C in the terminal where you started it.
  • Delete the project directory by running this command (macOS/Linux):
rm -rf ~/Desktop/mlflow-tracking
  • On Windows, run this in PowerShell instead:
Remove-Item -Recurse -Force ~\Desktop\mlflow-tracking
  • Optionally, uninstall the Python packages by running:
pip uninstall mlflow scikit-learn -y

Nice Work!

Nice Work!

You just built a complete local MLOps experiment tracking system using MLflow. You trained multiple models, tracked every run, compared results visually, and registered your best model for production inference.

You've learned how to:

  • Train and track three different ML models using autologging and manual logging in MLflow.
  • Compare experiment runs side-by-side in the MLflow UI to identify the best-performing model.
  • Register a production model in the Model Registry and load it back for predictions on new data.
  • Secret Mission: Added automated hyperparameter tuning with Optuna and logged every trial as a separate MLflow run.

Ready to quiz yourself?