Deploy an ML Model as an API

Containerize a PyTorch image classifier and deploy it to Google Cloud Run.

Introduction

30 Second Summary

Every time your phone identifies a dog in a photo or a search engine recognizes a landmark, a machine learning model is running behind the scenes. That model does not live on your device. It sits on a server somewhere, waiting for requests.

In this project, you will build that server. You will wrap a pre-trained image classification model in a web API, package it in a Docker container, and deploy it to Google Cloud Run as a live endpoint anyone can send images to.

What You'll Build

An image classification API powered by MobileNetV3-Small, a lightweight PyTorch neural network that can recognize 1,000 different object categories.

By the end of this project, you'll have:

  • A FastAPI application that loads a pre-trained PyTorch model and serves image classification predictions through a REST API.
  • A Dockerfile that packages the application with CPU-optimized PyTorch into a single deployable container.
  • A live Cloud Run service running your ML inference API, accessible via a public HTTPS URL with automatic scaling.
  • Secret Mission: add performance monitoring that tracks inference time and exposes model metadata through a dedicated endpoint.

Are there any prerequisites?

This project assumes familiarity with Python, REST APIs, basic Docker usage, and terminal commands. You should have Python 3.10+, Docker Desktop, and the Google Cloud CLI (gcloud) installed before starting.

Set Up Your Project

Before you can serve ML predictions through an API, you need a clean project environment. The right dependencies need to be installed and isolated from your system Python.

In this step, you will create a dedicated project folder, set up a virtual environment, and install PyTorch alongside FastAPI and the image processing libraries your API needs.

In this step, get ready to:
  • Create the ml-classifier-api project directory and a Python virtual environment.
  • Install PyTorch, FastAPI, and supporting dependencies.
  • Verify that all packages import correctly.
Create the project directory and virtual environment

A virtual environment gives your project its own isolated set of Python packages. This prevents version conflicts between different projects on your machine.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir ml-classifier-api
cd ml-classifier-api
  • Confirm you are in the right folder by running:
pwd

You should see a path ending in ml-classifier-api.

  • Create a Python virtual environment and activate it by running these commands:

macOS

python3 -m venv .venv
source .venv/bin/activate

Windows

python -m venv .venv
.venv\Scripts\activate
  • Check your terminal prompt. It should now start with (.venv), confirming the virtual environment is active.

Don't see (.venv) in your prompt?

Make sure you ran the activation command (source .venv/bin/activate on macOS or .venv\Scripts\activate on Windows) after creating the virtual environment.

On Windows, if PowerShell blocks the script, run Set-ExecutionPolicy -Scope CurrentUser RemoteSigned first, then try the activation command again.

Why isn't my Python virtual environment activating?

Install Python dependencies
  • Install all the packages your project needs by running this command:
pip install torch torchvision fastapi uvicorn python-multipart Pillow

What do these packages do?

  • torch and torchvision are the PyTorch ML framework and its computer vision model library.
  • fastapi is the web framework for building your API.
  • uvicorn is the ASGI server that runs FastAPI applications.
  • python-multipart enables file upload handling in FastAPI.
  • Pillow handles image opening and processing.
  • Wait for the installation to finish. This may take a few minutes since PyTorch is a large download.
  • Look at the end of the terminal output. You should see Successfully installed followed by a list of package names.

Installation failing?

If you see pip: command not found, your virtual environment is not activated. Your prompt should start with (.venv).

If the download times out or stalls, check your internet connection and run the command again. Pip skips packages that are already installed.

Help me fix my pip install error.

Verify the installation
  • Confirm that all key packages import correctly by running this command:
python -c "import torch; import torchvision; import fastapi; print('PyTorch version:', torch.__version__)"

You should see a PyTorch version number printed, such as 2.11.0.

What does this command do?

The python -c flag runs a one-line Python script directly from the terminal. This script imports the three key libraries and prints the PyTorch version to confirm everything is wired up.

Seeing a ModuleNotFoundError?

This means the package did not install into your active environment. Confirm your prompt shows (.venv) and re-run the pip install command from the previous substep.

If only one package fails, try reinstalling it individually (for example, pip install torch).

Help me fix my ModuleNotFoundError after installing PyTorch.

Your environment is ready. Next up, you will build the FastAPI application that loads a pre-trained ML model and serves image classification predictions.

Build the ML Inference API

Your development environment is ready with all the packages installed. Now it's time to write the core of this project: a REST API that accepts images and returns ML predictions.

You'll build a FastAPI web server that loads a pre-trained MobileNetV3-Small model when it starts up. When someone uploads an image, the model classifies it and returns the top 5 predictions with confidence scores.

In this step, get ready to:
  • Load a pre-trained MobileNetV3-Small model into a FastAPI application.
  • Build a prediction endpoint that classifies uploaded images.
  • Create a requirements file for Docker deployment.
Create main.py with model loading

The first thing your API needs is a model to make predictions with. You'll load MobileNetV3-Small, a lightweight neural network pre-trained on ImageNet to recognize 1,000 object categories.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code and press Enter to open it.
  • Click File in the top menu bar.
  • Click Open Folder.
  • Select the ml-classifier-api folder on your Desktop, then click Open.

You should see the ml-classifier-api folder name in the Explorer sidebar on the left, with the .venv folder nested inside.

  • Create a new file by clicking the New File icon at the top of the Explorer sidebar (it looks like a page with a plus sign).
  • Name the file main.py and press Enter.
  • Set up the application foundation by copying the following code into main.py:
import io

from fastapi import FastAPI, File, UploadFile, HTTPException
from fastapi.responses import JSONResponse
from PIL import Image
import torch
from torchvision.models import mobilenet_v3_small, MobileNet_V3_Small_Weights

app = FastAPI(title="ML Image Classifier API")

# Load model and weights at startup
weights = MobileNet_V3_Small_Weights.IMAGENET1K_V1
model = mobilenet_v3_small(weights=weights)
model.eval()

# Get preprocessing transforms and class labels from weights
preprocess = weights.transforms()
categories = weights.meta["categories"]


@app.get("/")
def health_check():
    return {"status": "healthy", "model": "MobileNetV3-Small"}

What does this code do?

  • The imports pull in FastAPI for the web server, Pillow for image handling, PyTorch for the ML model, and torchvision for the pre-trained MobileNetV3-Small model and its weights.
  • MobileNet_V3_Small_Weights.IMAGENET1K_V1 loads a set of weights that were pre-trained on the ImageNet dataset (1,000 categories, from "tench" to "volcano").
  • model.eval() switches the model from training mode to inference mode. This disables training-only behaviors like dropout.
  • weights.transforms() returns the exact preprocessing pipeline the model expects: resize to 256, center crop to 224, and normalize pixel values.
  • The health_check endpoint at / returns a simple JSON response so you can verify the server is running.
  • Save main.py.

You won't see any visible output yet. The next chunk adds the prediction endpoint that makes this API useful.

Add the /predict endpoint

The health check confirms your server is alive, but the real work happens in the prediction endpoint. This endpoint accepts an image upload, runs it through the model, and returns classification results.

  • In main.py, place your cursor at the end of the health_check function.
  • Leave two blank lines after the health_check function, then copy the following code:
@app.post("/predict")
async def predict(file: UploadFile = File(...)):
    # Validate that the uploaded file is actually an image
    if not file.content_type or not file.content_type.startswith("image/"):
        raise HTTPException(status_code=400, detail="File must be an image")

    # Read the image bytes and open with Pillow
    try:
        image_data = await file.read()
        image = Image.open(io.BytesIO(image_data)).convert("RGB")
    except Exception:
        raise HTTPException(status_code=400, detail="Could not process image")

    # Apply the same preprocessing the model was trained with
    input_tensor = preprocess(image).unsqueeze(0)

    # Run inference with gradients disabled (faster, less memory)
    with torch.no_grad():
        outputs = model(input_tensor)
        probabilities = torch.nn.functional.softmax(outputs[0], dim=0)

    # Get the top 5 predictions
    top5_prob, top5_idx = torch.topk(probabilities, 5)

    predictions = [
        {"label": categories[idx.item()], "confidence": round(prob.item(), 4)}
        for prob, idx in zip(top5_prob, top5_idx)
    ]

    return JSONResponse(content={
        "predictions": predictions,
        "top_prediction": predictions[0]["label"],
    })

What does this endpoint do?

This endpoint handles the full image classification pipeline in five stages:

  • Validation checks that the uploaded file has an image content type. Non-image files get a 400 error.
  • Image loading reads the raw bytes, opens them with Pillow, and converts to RGB format.
  • Preprocessing applies the transforms the model was trained with (resize, crop, normalize). The .unsqueeze(0) call adds a batch dimension that PyTorch expects.
  • Inference runs the image through the model inside torch.no_grad(), which disables gradient tracking for faster execution. The raw outputs are converted to probabilities using softmax.
  • Results picks the top 5 highest-confidence predictions and returns them as JSON with both labels and scores.
  • Save main.py.

You'll test this endpoint with a real image in the next step. For now, confirm your code matches the complete file below.

Code looking different from the reference?

  • Make sure the predict function starts two blank lines below the health_check function. Python convention uses two blank lines between top-level functions.
  • Check that every line inside the function is indented with 4 spaces. Python is whitespace-sensitive, so incorrect indentation causes errors.
  • help me find the differences between my main.py and the reference code

✔️ Awesome, I've got everything!

Double check you've saved main.py before moving on.

ⓧ I'd like to double check the full code

Here is the complete main.py file as it should look right now.

import io

from fastapi import FastAPI, File, UploadFile, HTTPException
from fastapi.responses import JSONResponse
from PIL import Image
import torch
from torchvision.models import mobilenet_v3_small, MobileNet_V3_Small_Weights

app = FastAPI(title="ML Image Classifier API")

# Load model and weights at startup
weights = MobileNet_V3_Small_Weights.IMAGENET1K_V1
model = mobilenet_v3_small(weights=weights)
model.eval()

# Get preprocessing transforms and class labels from weights
preprocess = weights.transforms()
categories = weights.meta["categories"]


@app.get("/")
def health_check():
    return {"status": "healthy", "model": "MobileNetV3-Small"}


@app.post("/predict")
async def predict(file: UploadFile = File(...)):
    if not file.content_type or not file.content_type.startswith("image/"):
        raise HTTPException(status_code=400, detail="File must be an image")

    try:
        image_data = await file.read()
        image = Image.open(io.BytesIO(image_data)).convert("RGB")
    except Exception:
        raise HTTPException(status_code=400, detail="Could not process image")

    input_tensor = preprocess(image).unsqueeze(0)

    with torch.no_grad():
        outputs = model(input_tensor)
        probabilities = torch.nn.functional.softmax(outputs[0], dim=0)

    top5_prob, top5_idx = torch.topk(probabilities, 5)

    predictions = [
        {"label": categories[idx.item()], "confidence": round(prob.item(), 4)}
        for prob, idx in zip(top5_prob, top5_idx)
    ]

    return JSONResponse(content={
        "predictions": predictions,
        "top_prediction": predictions[0]["label"],
    })
Create requirements.txt

Your API needs a requirements.txt file so Docker knows which Python packages to install inside the container. This file will only list four packages.

  • Create a new file called requirements.txt in the Explorer sidebar (the same way you created main.py).
  • Add the following package list to requirements.txt:
fastapi
uvicorn
python-multipart
Pillow

Where are torch and torchvision?

You might notice torch and torchvision are missing from this file. That is intentional.

The default PyTorch package includes CUDA GPU support and weighs over 3 GB. In the Dockerfile, you'll install PyTorch from a special CPU-only index that keeps the image under 1.5 GB. Listing torch in requirements.txt would pull the wrong (much larger) version.

  • Save requirements.txt.
  • Confirm both files exist by running this in your terminal:
ls

You should see main.py and requirements.txt in the output (alongside the .venv folder from Step 1).

Don't see both files?

  • Make sure your terminal is inside the ml-classifier-api folder. Run pwd to check your current directory.
  • Confirm you saved both files in VS Code. Unsaved files show a dot next to their name in the editor tab.
  • help me figure out why my project files are missing

Your API code is complete. Next up, you'll start the server and test it with a real image to see MobileNetV3-Small in action.

Test the API Locally

Your FastAPI application is built and your dependencies are pinned. Before you package everything into a container, you need to prove the API actually works.

In this step, you will start the server, send it a real image, and check that the predictions come back correctly.

In this step, get ready to:
  • Start the API server and load the ML model.
  • Send a test image to the prediction endpoint and examine the results.
  • Verify the health check endpoint and shut down the server.
Start the API server

The MobileNetV3-Small model loads when the server starts. On the first run, PyTorch downloads the model weights (about 9.8 MB) before accepting requests.

  • Make sure your virtual environment is active in your terminal (you should see (.venv) in your prompt).
  • Start the API server by running this command:
uvicorn main:app --port 8080

What happens on the first run?

PyTorch downloads the MobileNetV3-Small weights (~9.8 MB) to a local cache. You will see a progress bar in the terminal while this happens.

Once the download finishes, the model loads into memory and the server starts. On future runs, PyTorch uses the cached weights and starts immediately.

You should see output ending with Uvicorn running on http://127.0.0.1:8080. The terminal will stay occupied because the server is now running and waiting for requests.

Server won't start?

  • If you see ModuleNotFoundError, your virtual environment might not be activated. Run source .venv/bin/activate (macOS/Linux) or .venv\Scripts\activate (Windows) and try again.
  • If you see Address already in use, another process is using port 8080. Stop that process first, or check if you already have a uvicorn instance running.
  • Still stuck? Help me debug why uvicorn won't start on port 8080.
Test the prediction endpoint

The server is occupying your terminal, so you need a second terminal to send requests. You also need a test image for the model to classify.

  • Open a new terminal window or tab. In VS Code, click the + icon in the terminal panel.
  • Navigate to your project folder in the new terminal by running:
cd ~/Desktop/ml-classifier-api
  • Find any .jpg or .png image on your computer (a photo, screenshot, or any image file).
  • Copy or move the image into your ml-classifier-api folder and rename it to test-image.jpg.

Any image works

The model recognizes 1,000 different object categories. A photo of a pet, a coffee mug, or even a selfie will all produce interesting results.

  • Send your test image to the prediction endpoint by running this command:
curl -X POST http://localhost:8080/predict -F "file=@test-image.jpg"

What does this command do?

  • curl -X POST sends an HTTP POST request to your local server.
  • -F "file=@test-image.jpg" uploads the file as form data. The @ symbol tells curl to read the file from disk. The field name file matches the parameter name in your predict() function.

You should see a JSON response containing a predictions array with five objects. Each object has a label (the category name) and a confidence score between 0 and 1.

What do the results mean?

The confidence value is a probability. A confidence of 0.85 means the model is 85% confident in that label.

The top_prediction field gives you the model's single best guess. The exact labels depend on what image you used. If you sent a photo of a cat, you might see labels like tabby or tiger_cat.

Not seeing predictions?

  • If you see Could not resolve host, make sure the uvicorn server is still running in your first terminal.
  • If you see No such file or directory, the file test-image.jpg is not in the current directory. Run ls to check, and make sure your terminal is inside ml-classifier-api.
  • If you see File must be an image, the file might not be a valid image format. Try a different .jpg or .png file.
  • Still stuck? Help me debug my curl request to the /predict endpoint.
Test the health check endpoint

The / endpoint is a health check. Monitoring tools and load balancers use this to confirm the service is alive and the model is loaded.

  • Test the health check by running this command in the same second terminal:
curl http://localhost:8080/

You should see {"status":"healthy","model":"MobileNetV3-Small"} in the response.

  • Switch back to your first terminal (where uvicorn is running).
  • Press Ctrl+C to stop the server.

Seeing a different response?

  • If you see Connection refused, the uvicorn server may have crashed or been stopped. Switch to your first terminal and restart it with uvicorn main:app --port 8080.
  • If the response is missing the model field, double check the health_check() function in main.py returns both status and model keys.
  • Need help? Help me debug my health check endpoint.

Both endpoints work. Your API loads a real ML model, processes images, and returns predictions. Next, you will package this into a Docker container so it runs the same way on any machine.

Containerize with Docker

Your API is tested and producing real predictions. The next challenge is packaging it so it can run anywhere, not just your machine.

Docker solves this by bundling your code, PyTorch model, and all dependencies into a single container image. In this step, you'll write a Dockerfile, build the image, and verify the containerized API produces identical results.

In this step, get ready to:
  • Create a Dockerfile and .dockerignore file.
  • Build a Docker image for your ML API.
  • Run and test the containerized API.
Create the Dockerfile and .dockerignore

Your Dockerfile is a blueprint that defines the exact environment your API needs to run. Every dependency, file, and startup command gets declared here.

  • Create a new file called Dockerfile (no file extension) in your ml-classifier-api folder.
  • Copy the following code into it:
FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir torch torchvision --index-url https://download.pytorch.org/whl/cpu
RUN pip install --no-cache-dir -r requirements.txt

COPY main.py .

EXPOSE 8080

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080"]

What does this Dockerfile do?

  • FROM python:3.11-slim starts from a lightweight Python base image.
  • The first RUN pip install grabs PyTorch and torchvision from the CPU-only wheel index. This keeps the image around 1.5 GB instead of 3+ GB.
  • The second RUN pip install installs the remaining packages from requirements.txt.
  • COPY main.py . adds your application code to the container.
  • CMD starts uvicorn when the container launches, listening on all interfaces (0.0.0.0) at port 8080.

Why install PyTorch separately from the CPU-only index?

PyTorch's default package includes CUDA GPU support, which adds gigabytes to the download. MobileNetV3-Small runs fast enough on CPU alone, so the GPU libraries are dead weight.

Using the CPU-only index (https://download.pytorch.org/whl/cpu) gives you a much smaller package. This is a standard optimization for containerized ML workloads that don't need GPU acceleration.

  • Save Dockerfile.
  • Create a .dockerignore file in the same ml-classifier-api folder.
  • Copy the following into it:
__pycache__
*.pyc
.venv
.git

What does .dockerignore do?

This file tells Docker to skip these files and directories when copying your project into the image. Your virtual environment, cached bytecode, and Git history aren't needed inside the container.

  • Save .dockerignore.
  • Confirm both files exist by running:
ls -a

You should see Dockerfile, .dockerignore, main.py, and requirements.txt in the output.

Don't see both files?

  • Check that Dockerfile has no file extension. It should be Dockerfile, not Dockerfile.txt.
  • Make sure .dockerignore starts with a dot. The -a flag in ls shows hidden files that start with a dot.
  • Why aren't my Dockerfile and .dockerignore showing up?

✔️ Awesome, I've got everything!

Double check you've saved both files before moving on.

ⓧ I'd like to double check the full code

Here is the complete Dockerfile:

FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir torch torchvision --index-url https://download.pytorch.org/whl/cpu
RUN pip install --no-cache-dir -r requirements.txt

COPY main.py .

EXPOSE 8080

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080"]

And the complete .dockerignore:

__pycache__
*.pyc
.venv
.git
Build the Docker image

Docker Desktop must be running before you can build. If it's not already open, start it now.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Docker Desktop and press Enter to open it.
  • Wait until Docker Desktop shows a green Running indicator before continuing.
  • Build your Docker image by running this command:
docker build -t ml-classifier-api .

This takes a few minutes. Docker downloads the Python base image and installs PyTorch and all your other dependencies.

  • Verify the image was created by running:
docker images ml-classifier-api

You should see your ml-classifier-api image listed with the latest tag.

Build failing?

  • If you see Cannot connect to the Docker daemon, Docker Desktop is not running. Open it and wait for the engine to start.
  • If the build fails during pip install, check your internet connection. Docker needs to download packages from the PyTorch wheel index.
  • Why is my Docker build failing?
Run and test the container

Time to start a container from your image and confirm it produces the same predictions as your local API.

  • Start a container from your image by running:
docker run -p 8080:8080 ml-classifier-api

What does -p 8080:8080 mean?

The -p flag maps a port on your machine to a port inside the container. -p 8080:8080 means requests to localhost:8080 get forwarded into the container's port 8080.

You should see uvicorn startup output in your terminal. The first container startup downloads the model weights, which adds a few seconds.

  • Open a new terminal tab or window. Your current terminal is occupied by the running container.
  • Test the health check by running:
curl http://localhost:8080/

You should see {"status":"healthy","model":"MobileNetV3-Small"}.

  • Test the prediction endpoint with your sample image by running:
curl -X POST http://localhost:8080/predict -F "file=@[[TEST_IMAGE=\"your-test-image.jpg\"]]"

You should see the same top 5 predictions and confidence scores you got when testing locally in Step 3.

  • Switch back to the terminal running the container.
  • Press Ctrl+C to stop the container.

Not getting predictions from the container?

  • If you see Connection refused, the container may still be starting up. Wait for the uvicorn startup message before sending requests.
  • If you see port is already allocated, another process is using port 8080. Make sure you stopped the local uvicorn server from Step 3.
  • Make sure you're running the curl command in your new terminal tab, not in the terminal running the container.
  • Why can't I connect to my Docker container?

Your ML API is now packaged in a portable container. Next up, you'll deploy this container to Cloud Run and get a live public URL.

Deploy to Google Cloud Run

Your ML classifier is containerized and proven to work locally. The final step is getting it onto a live URL so anyone can send an image and get predictions back.

Cloud Run is Google Cloud's serverless container platform. You give it a Docker image, and it gives you a public HTTPS endpoint that scales to zero when idle. That means zero cost when nobody is sending requests.

In this step, get ready to:
  • Configure your GCP project and enable the required APIs.
  • Deploy your container to Cloud Run with a single command.
  • Test your live ML inference endpoint with real images.
Configure your GCP project

Before deploying, gcloud needs to know which Google Cloud project to use. You also need to enable two APIs that power the deployment.

  • Sign in to Google Cloud by running this command:
gcloud auth login

What happens next?

This command opens your browser and asks you to sign in with your Google account. After you authorize access, the terminal confirms you are authenticated with a message: You are now authenticated with the gcloud CLI!

  • Set your active GCP project by running this command:
gcloud config set project [[GCP_PROJECT_ID="your-gcp-project-id"]]

You should see Updated property [core/project]. in your terminal.

Find your project ID

Your project ID is visible in the Google Cloud Console at the top of the page, next to the project name. It looks something like my-project-123456. This is different from the project name.

  • Enable the Cloud Run Admin API and Cloud Build API by running this command:
gcloud services enable run.googleapis.com cloudbuild.googleapis.com

What are these APIs?

run.googleapis.com lets you create and manage Cloud Run services. cloudbuild.googleapis.com lets Google build your Docker image in the cloud from your source code.

This command takes a few seconds. You should see Operation finished successfully when both APIs are enabled.

Seeing a permission error?

Make sure you are signed in with the correct Google account by running gcloud auth login again. You also need to be a project Owner or Editor to enable APIs.

Help me fix a permission error when enabling GCP APIs.

Deploy the container to Cloud Run

The gcloud run deploy --source . command handles the entire pipeline in one shot. It uploads your source code, builds the Docker image using Cloud Build, pushes the image to Artifact Registry, and deploys it to Cloud Run.

  • Deploy your application to Cloud Run by running this command:
gcloud run deploy ml-classifier --source . --region us-central1 --memory 1Gi --max-instances 1 --allow-unauthenticated --timeout 120

What do these flags mean?

  • ml-classifier is the name of your Cloud Run service.
  • --source . tells Cloud Build to use the Dockerfile in your current directory.
  • --memory 1Gi gives the container 1 GB of RAM. PyTorch needs more than the default 512 MB.
  • --max-instances 1 caps scaling at one container to stay within free tier limits.
  • --allow-unauthenticated makes the endpoint publicly accessible without authentication.
  • --timeout 120 gives each request up to 120 seconds. The first request needs extra time because the model loads on cold start.

Expect interactive prompts during this deploy

Cloud Build may ask you to create an Artifact Registry repository or enable additional APIs. Enter y for each prompt and press Enter to continue.

The build takes 3-5 minutes. Cloud Build is downloading and installing PyTorch inside the container image, which takes time. Wait for the command to finish completely.

When the deploy finishes, you should see output that includes your service URL. It looks something like https://ml-classifier-xxxxx-uc.a.run.app.

  • Copy the Service URL from the deployment output. You will use this URL in the next substep to test your live endpoint.

Build failed or timed out?

Make sure you are running the command from inside the ml-classifier-api folder, where your Dockerfile lives. The --source . flag points to the current directory.

If you see a billing error, make sure billing is enabled on your GCP project. Cloud Build and Cloud Run both require an active billing account, even when using the free tier.

Help me debug a failed Cloud Run deployment.

Test your live endpoint

Your service is live. Time to send real requests to it and confirm the predictions match what you saw locally.

  • Test the health check endpoint by running this command:
curl [[SERVICE_URL="https://ml-classifier-xxxxx-uc.a.run.app"]]

You should see {"status":"healthy","model":"MobileNetV3-Small"} in your terminal.

Why is the first request slow?

Cloud Run scales to zero when idle. The first request after a period of inactivity triggers a "cold start" where Cloud Run spins up a new container and loads the PyTorch model into memory. This can take 5-15 seconds. Subsequent requests are much faster.

  • Test the prediction endpoint by running this command with the same test image you used in Step 3:
curl -X POST -F "file=@[[TEST_IMAGE="your-test-image.jpg"]]" [[SERVICE_URL="https://ml-classifier-xxxxx-uc.a.run.app"]]/predict

You should see a JSON response with the top 5 predictions and confidence scores. These predictions should match the results you got when testing locally in Step 3.

Getting a connection error or 403?

Double-check that your URL matches the Service URL from the deploy output exactly. A missing https:// or a trailing slash on the /predict path can cause failures.

If you see a 403 Forbidden error, the service may not have unauthenticated access enabled. Redeploy with the --allow-unauthenticated flag included.

Help me troubleshoot my Cloud Run endpoint.

Secret mission

Add Performance Monitoring

Your ML API works, but how fast does it actually run? Add timing instrumentation that tracks inference latency and create a model metadata endpoint. This is exactly the kind of observability an ML infrastructure engineer adds before handing a service to a team.

Before you go

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project uses Google Cloud Run, which scales to zero when idle. Your free tier covers 240,000 vCPU-seconds and 2 million requests per month, so costs stay at $0 under normal usage.

Cost warning

Your Cloud Run service scales to zero automatically, so you will not be charged while it is idle. However, if your service receives a large volume of requests that exceed the free tier limits, charges will apply. If you are unsure, delete the Cloud Run service to be safe.

Resources you used:

  • Cloud Run service ml-classifier. Scales to zero when not receiving requests. Consumes from the free tier (240,000 vCPU-seconds/month) while running.
  • Artifact Registry repository cloud-run-source-deploy . Stores the built container image. Artifact Registry provides 500 MB of free storage.
  • Cloud Build. Used to build the container image during deployment. 120 free build-minutes per day.
  • Local project directory ml-classifier-api with Python virtual environment and project files.

Keep everything running

No action needed. Choose this if you are still actively building or plan to continue with the next project in this series.

  • Your Cloud Run service scales to zero automatically. You will not be charged unless someone sends requests to your endpoint.
  • Your Artifact Registry repository stores the container image. It stays within the free 500 MB storage tier.
  • Your local ml-classifier-api folder and virtual environment remain on your Desktop.

Pause - I'll come back to this later

Delete the Cloud Run service to ensure no requests can reach it. Keep your local files and Artifact Registry image so you can redeploy quickly later.

  • Delete the Cloud Run service by running this command:
gcloud run services delete ml-classifier --region us-central1
  • Confirm the deletion when prompted by entering Y.
  • Keep your local ml-classifier-api folder. You can redeploy from it anytime with a single gcloud run deploy command.

Delete - I don't want to use this again

Remove all cloud resources and optionally delete your local project files.

  • Delete the Cloud Run service and the Artifact Registry repository by running these commands:
gcloud run services delete ml-classifier --region us-central1
gcloud artifacts repositories delete cloud-run-source-deploy --location us-central1
  • Confirm each deletion when prompted by entering Y.
  • To remove local project files, delete the ml-classifier-api folder from your Desktop.
  • To remove the Docker image from your machine, run:
docker rmi ml-classifier-api

Mission accomplished

That's a Wrap!

Nice work! You just deployed your first ML model to production infrastructure.

You've learned how to:

  • Built a FastAPI application that loads a pre-trained MobileNetV3-Small model and serves image classification predictions through a REST API.
  • Containerized the application with Docker using CPU-optimized PyTorch to keep the image lean and deployment-ready.
  • Deployed to Google Cloud Run with a single command, getting a live HTTPS endpoint with automatic scaling to zero.
  • Added performance monitoring with inference timing and a model metadata endpoint (Secret Mission).

This is the deployment pattern used for real ML services at scale. The model changes and the infrastructure gets more complex, but the core loop stays the same: load a model, wrap it in an API, containerize it, deploy it. You now have that pattern in your toolkit.

Ready to quiz yourself?