Scale a Python API with Docker and Redis

Build a load-balanced, cached API system using Docker, Nginx, and Redis.

Introduction

30 Second Summary

When a popular app handles a traffic surge without slowing down, it's because the system runs multiple copies of itself and spreads traffic across them. Frequently requested data gets cached so the database isn't overwhelmed.

In this project, you will build a product catalog API that starts as a single Python process and evolves into a scaled, multi-container system. You'll add Docker containers, an Nginx load balancer, and Redis caching, running load tests at every stage to prove each layer's impact with real numbers.

What You'll Build

A FastAPI product catalog API that evolves from a single Python process into a scaled, load-balanced, cached system orchestrated by Docker Compose.

By the end of this project, you'll have:

  • A FastAPI product catalog API with simulated database latency, packaged into a Docker container.
  • An Nginx reverse proxy distributing traffic across 3 scaled API instances using Docker Compose.
  • Redis caching that eliminates redundant database queries, with load test results proving the performance improvement at each stage.
  • Secret Mission: A real-time /stats endpoint tracking cache hit ratios and per-instance request counts.

Are there any prerequisites?

This project assumes basic Python knowledge and comfort using the terminal. You don't need any prior experience with Docker, Nginx, or Redis.

Before We Start

Before We Start...

Set Up Your Development Environment

Before you start building, you need the tools that make each scaling layer possible. Docker packages your API into containers. Nginx distributes traffic across those containers. Redis caches responses to speed things up.

In this step, you'll install Docker Desktop and a load testing tool called hey. You'll also create the project folder and install the Python libraries your API needs.

In this step, get ready to:
  • Install Docker Desktop and the hey load testing tool.
  • Create the project directory and a Python virtual environment.
  • Install FastAPI, Redis, and uvicorn as project dependencies.
Install Docker Desktop and hey

Docker Desktop is the app that makes Docker available on your Mac. It lets you build and run containers locally.

The hey tool is a command-line load tester that fires hundreds of requests at your API in a single command. You'll use it at every stage of this project to measure the real impact of each scaling layer.

  • Check if Docker is already installed by running this command:
docker --version

✔️ I see a version number

Docker is already installed. Make sure Docker Desktop is running by checking for the whale icon in your menu bar at the top of your screen. The whale icon stops animating once Docker Desktop has finished starting.

  • Verify Docker Compose is also available by running this command:
docker compose version

You should see a version number starting with v2. Docker Compose comes bundled with Docker Desktop.

ⓧ Command not found

  • Open your browser and go to the Docker Desktop for Mac download page.
  • Click the download button for your Mac (Apple Silicon or Intel).
  • Open the downloaded Docker.dmg file.
  • Drag the Docker icon into your Applications folder.
  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Docker and press Enter to launch Docker Desktop.
  • Accept the Docker Subscription Service Agreement when prompted. Docker Desktop is free for personal use and education.
  • Wait for Docker Desktop to finish starting. The whale icon in your menu bar stops animating when it is ready.
  • Confirm Docker is installed by running this command:
docker --version

You should see a version number like Docker version 28.x.x.

  • Verify Docker Compose is available by running this command:
docker compose version

You should see a version number starting with v2.

Docker command still not found?

Make sure Docker Desktop is fully started. The whale icon in your menu bar should be static, not animating.

Try closing and reopening your terminal after installing Docker Desktop. The Docker CLI is added to your system path during installation, but your current terminal session may not have picked it up yet.

Help me troubleshoot Docker installation on my Mac

  • Check if hey is already installed by running this command:
which hey

✔️ I see a file path

The hey load testing tool is already installed. You're ready to move on.

ⓧ hey not found

  • Install hey by running this command:
brew install hey

You should see Homebrew download and install the package.

brew command not found?

Homebrew needs to be installed first. Visit brew.sh and follow the installation instructions for your Mac.

Help me install Homebrew on my Mac

  • Verify hey works by sending a single test request:
hey -n 1 https://example.com

You should see a short summary with response time statistics. This confirms hey can send HTTP requests.

Create the project and install Python dependencies

Your project needs a specific folder structure. The app/ directory holds your API code and Dockerfile. The nginx/ directory holds the Nginx configuration.

  • Open your terminal.

How do I open my terminal?

Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar. Type Terminal and press Enter.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder structure and move into it by running these commands:
mkdir -p scaling-api/app scaling-api/nginx
cd scaling-api
  • Confirm the folders were created by running:
ls

You should see app and nginx listed in the output.

Don't see both folders?

Make sure you ran the mkdir command with the -p flag. Without it, the nested directories won't be created.

Check that you are inside the scaling-api folder by running pwd. You should see a path ending in /Desktop/scaling-api.

Help me troubleshoot my project folder structure

A Python virtual environment isolates your project's packages from the rest of your system. This prevents version conflicts between different projects on your machine.

  • Create and activate a virtual environment by running these commands:
python3 -m venv .venv
source .venv/bin/activate

Your terminal prompt now shows (.venv) at the beginning. This confirms the virtual environment is active.

Don't see (.venv) in your prompt?

Make sure you ran both commands. The first command creates the virtual environment. The second command activates it.

If you see an error about python3 not being found, check that Python 3 is installed on your system by running python3 --version.

Help me activate my Python virtual environment

Your API depends on three Python packages. FastAPI is the web framework. Redis is the caching client. Uvicorn is the server that runs your API.

  • Create the app/requirements.txt file by running this command:
cat > app/requirements.txt << 'EOF'
fastapi[standard]==0.136.1
redis==7.4.0
uvicorn==0.34.0
EOF

What are these packages?

  • fastapi[standard] installs FastAPI along with its recommended extras, including the development server.
  • redis is the Python client for Redis. You won't use it until Step 5, but installing it now keeps your container builds consistent.
  • uvicorn is an ASGI server that runs FastAPI in production. Pinning it here ensures your Docker containers use the same version.
  • Confirm the file was created correctly by running:
cat app/requirements.txt

You should see the three package lines printed in your terminal.

✔️ Awesome, I've got everything!

Great. Double check you've saved your file before moving on.

ⓧ I'd like to double check the full code

Here is the complete app/requirements.txt file. Make sure yours matches exactly.

fastapi[standard]==0.136.1
redis==7.4.0
uvicorn==0.34.0
  • Install the dependencies by running this command:
pip install -r app/requirements.txt

You should see pip download and install all three packages along with their dependencies. The output ends with Successfully installed followed by a list of package names.

Seeing a permission error?

Make sure your virtual environment is active. You should see (.venv) at the beginning of your terminal prompt. If not, run source .venv/bin/activate and try the install again.

Help me fix pip install errors

Verify your tools are ready

Before moving on, confirm all three tools are working.

  • Verify FastAPI is installed by running this command:
python3 -c "import fastapi; print(fastapi.__version__)"

You should see 0.136.1 printed in your terminal.

  • Verify Docker is running by running this command:
docker info

You should see detailed information about your Docker installation, including the number of containers and images.

  • Verify hey is ready by running this command:
hey --help

You should see hey's usage information and a list of available flags.

One of the commands failed?

  • If python3 -c "import fastapi..." fails, make sure your virtual environment is active. Look for (.venv) in your prompt.
  • If docker info fails, make sure Docker Desktop is running. Look for the whale icon in your menu bar.
  • If hey --help fails, try reinstalling with brew reinstall hey.

Help me get my development environment working

Your development environment is ready. Next up, you'll build the product catalog API that you'll spend the rest of this project scaling.

Build the Product Catalog API

Your environment is ready. Now it's time to build the thing you're actually going to scale.

In this step, you'll create a product catalog API using FastAPI, a Python web framework built for speed. The API will simulate a slow database query on every request. That deliberate slowness is your baseline. Every scaling improvement you add later will be measured against it.

In this step, get ready to:
  • Create a FastAPI application with product endpoints and simulated database latency.
  • Run the API locally and explore its auto-generated documentation.
  • Run a load test to establish your performance baseline.
Create the FastAPI application

Your API needs a few moving parts: a fake product database, a connection to Redis for caching, and a simulated delay on every "database" read. You'll build these foundations first, then add the endpoints on top.

  • Create a new file called main.py inside the app/ folder of your scaling-api project.
  • Add the app setup, product data, Redis connection, and helper function by pasting this code into main.py:
import asyncio
import json
import os
import socket

from fastapi import FastAPI

import redis

app = FastAPI(title="Product Catalog API")

# Simulated product "database"
PRODUCTS = [
    {"id": 1, "name": "Mechanical Keyboard", "price": 129.99, "category": "Electronics"},
    {"id": 2, "name": "Standing Desk", "price": 499.99, "category": "Furniture"},
    {"id": 3, "name": "Noise-Canceling Headphones", "price": 249.99, "category": "Electronics"},
    {"id": 4, "name": "Ergonomic Chair", "price": 399.99, "category": "Furniture"},
    {"id": 5, "name": "Ultrawide Monitor", "price": 599.99, "category": "Electronics"},
]

HOSTNAME = socket.gethostname()

# Redis connection
REDIS_URL = os.getenv("REDIS_URL", "redis://localhost:6379")
try:
    redis_client = redis.from_url(REDIS_URL, decode_responses=True)
    redis_client.ping()
    redis_available = True
except Exception:
    redis_client = None
    redis_available = False

CACHE_TTL = 30  # seconds


async def simulate_db_delay():
    """Simulate a slow database query."""
    await asyncio.sleep(0.05)


@app.get("/")
async def root():
    return {"hostname": HOSTNAME, "status": "running"}

What does this code do?

There's a lot happening here. Let's break it down section by section.

  • PRODUCTS is a list of dictionaries acting as a fake database. In a real app, this would be a SQL or NoSQL database.
  • HOSTNAME captures the machine's hostname. When the API runs inside a Docker container, this returns the container ID. That's how you'll verify load balancing is working later.
  • The Redis connection block tries to connect, and if Redis isn't running, it sets redis_available to False. The rest of the code checks this flag before touching Redis, so the app works with or without it.
  • simulate_db_delay() pauses for 50 milliseconds. This represents the time a real database query would take. Caching will skip this delay entirely.
  • The GET / root endpoint returns the hostname. You'll use this to prove that different containers are handling different requests.

Why write caching code before Redis is running?

The try/except block makes the app resilient. Right now, Redis isn't running on your machine, so redis_available will be False and all caching code is simply skipped.

When you add Redis to the stack in Step 5, the caching will activate automatically. Writing it now means you can focus purely on the API logic in this step.

You won't see a change yet. The next chunk adds the API endpoints that bring this foundation to life.

  • Still in app/main.py, add the product endpoints below the root() function by pasting this code at the bottom of the file:
@app.get("/products")
async def get_products():
    # Check cache first
    if redis_available:
        cached = redis_client.get("products:all")
        if cached:
            return json.loads(cached)

    # Simulate slow database query
    await simulate_db_delay()

    result = {"products": PRODUCTS, "source": "database", "served_by": HOSTNAME}

    # Store in cache
    if redis_available:
        redis_client.setex("products:all", CACHE_TTL, json.dumps(result))

    return result


@app.get("/products/{product_id}")
async def get_product(product_id: int):
    cache_key = f"products:{product_id}"

    # Check cache first
    if redis_available:
        cached = redis_client.get(cache_key)
        if cached:
            return json.loads(cached)

    # Simulate slow database query
    await simulate_db_delay()

    product = next((p for p in PRODUCTS if p["id"] == product_id), None)
    if not product:
        return {"error": "Product not found", "served_by": HOSTNAME}

    result = {"product": product, "source": "database", "served_by": HOSTNAME}

    # Store in cache
    if redis_available:
        redis_client.setex(cache_key, CACHE_TTL, json.dumps(result))

    return result


@app.get("/cache-status")
async def cache_status():
    return {
        "redis_connected": redis_available,
        "redis_url": REDIS_URL,
        "served_by": HOSTNAME,
    }

What do these endpoints do?

  • GET /products returns all five products. It checks Redis for a cached copy first. On a cache miss, it waits 50ms (the simulated DB delay), builds the result, stores it in Redis with a 30-second TTL, and returns it.
  • GET /products/{product_id} returns a single product by ID. It follows the same cache-check-then-query pattern, with a unique cache key per product.
  • GET /cache-status is a diagnostic endpoint. It tells you whether Redis is connected and which container served the response.
  • Save app/main.py.

Seeing a syntax error when you save?

  • Make sure there's a blank line between the end of the first code chunk (the root() function) and the start of the second chunk (@app.get("/products")). Python needs blank lines between top-level function definitions.
  • Check that the f-string f"products:{product_id}" uses curly braces around product_id, not parentheses.
  • Double check the full code using the tab below to make sure everything lines up.

My main.py has a syntax error and I can't figure out where.

✔️ Awesome, I've got everything!

Great. Double check you've saved app/main.py before moving on.

ⓧ I'd like to double check the full code

Here's the complete app/main.py file. Make sure your code matches this exactly.

import asyncio
import json
import os
import socket

from fastapi import FastAPI

import redis

app = FastAPI(title="Product Catalog API")

# Simulated product "database"
PRODUCTS = [
    {"id": 1, "name": "Mechanical Keyboard", "price": 129.99, "category": "Electronics"},
    {"id": 2, "name": "Standing Desk", "price": 499.99, "category": "Furniture"},
    {"id": 3, "name": "Noise-Canceling Headphones", "price": 249.99, "category": "Electronics"},
    {"id": 4, "name": "Ergonomic Chair", "price": 399.99, "category": "Furniture"},
    {"id": 5, "name": "Ultrawide Monitor", "price": 599.99, "category": "Electronics"},
]

HOSTNAME = socket.gethostname()

# Redis connection
REDIS_URL = os.getenv("REDIS_URL", "redis://localhost:6379")
try:
    redis_client = redis.from_url(REDIS_URL, decode_responses=True)
    redis_client.ping()
    redis_available = True
except Exception:
    redis_client = None
    redis_available = False

CACHE_TTL = 30  # seconds


async def simulate_db_delay():
    """Simulate a slow database query."""
    await asyncio.sleep(0.05)


@app.get("/")
async def root():
    return {"hostname": HOSTNAME, "status": "running"}


@app.get("/products")
async def get_products():
    # Check cache first
    if redis_available:
        cached = redis_client.get("products:all")
        if cached:
            return json.loads(cached)

    # Simulate slow database query
    await simulate_db_delay()

    result = {"products": PRODUCTS, "source": "database", "served_by": HOSTNAME}

    # Store in cache
    if redis_available:
        redis_client.setex("products:all", CACHE_TTL, json.dumps(result))

    return result


@app.get("/products/{product_id}")
async def get_product(product_id: int):
    cache_key = f"products:{product_id}"

    # Check cache first
    if redis_available:
        cached = redis_client.get(cache_key)
        if cached:
            return json.loads(cached)

    # Simulate slow database query
    await simulate_db_delay()

    product = next((p for p in PRODUCTS if p["id"] == product_id), None)
    if not product:
        return {"error": "Product not found", "served_by": HOSTNAME}

    result = {"product": product, "source": "database", "served_by": HOSTNAME}

    # Store in cache
    if redis_available:
        redis_client.setex(cache_key, CACHE_TTL, json.dumps(result))

    return result


@app.get("/cache-status")
async def cache_status():
    return {
        "redis_connected": redis_available,
        "redis_url": REDIS_URL,
        "served_by": HOSTNAME,
    }
Run the API locally and test it

FastAPI comes with a built-in development server and auto-generated interactive documentation. You can test every endpoint directly in your browser without writing any client code.

  • Make sure your virtual environment is still active. You should see (.venv) at the start of your terminal prompt.
  • Start the FastAPI development server by running this command:
fastapi dev app/main.py

What does this command do?

fastapi dev starts a development server with auto-reload. Every time you save a code change, the server restarts automatically. It runs on http://127.0.0.1:8000 by default.

You should see output that includes Uvicorn running on http://127.0.0.1:8000. The server is now listening for requests.

Seeing a ModuleNotFoundError?

  • If the error mentions fastapi or redis, your virtual environment may not be active. Run source .venv/bin/activate and try again.
  • If the error mentions a different module, re-run pip install -r app/requirements.txt to reinstall the dependencies.

Help me fix a ModuleNotFoundError when starting my FastAPI server.

  • Open your browser and go to http://127.0.0.1:8000/docs.

You should see the Swagger UI documentation page with all four of your endpoints listed: GET /, GET /products, GET /products/{product_id}, and GET /cache-status. FastAPI generates this documentation automatically from your code.

  • Click on the GET /products row to expand it.
  • Click Try it out, then click Execute.

The response body shows all five products. Notice the "source": "database" field. That tells you the data came from the simulated database, not from cache.

  • Now test the single product endpoint. Click on GET /products/{product_id} to expand it.
  • Click Try it out.
  • Enter 1 in the product_id field, then click Execute.

You should see the Mechanical Keyboard product returned with "source": "database". Every response includes that 50ms simulated delay, even though you can't feel it on a single request.

Check cache-status too

Try the GET /cache-status endpoint. It should show "redis_connected": false. That confirms Redis isn't running yet and all caching is being skipped.

Run your first load test

Testing a single request in the browser tells you the API works. It doesn't tell you how the API performs under pressure. A load test sends hundreds of requests at once to measure throughput and latency.

  • Open a new terminal window. The FastAPI server needs to keep running in the first terminal.
  • Run a load test against your products endpoint by running this command:
hey -n 200 -c 10 http://127.0.0.1:8000/products

What does this command do?

  • -n 200 sends 200 total requests to the endpoint.
  • -c 10 runs 10 requests at a time (concurrently). This simulates 10 users hitting your API at the same moment.
  • The output shows you requests per second, average latency, and latency distribution. These are the numbers you'll compare against as you scale.

After a few seconds, hey prints a summary. Look for the Requests/sec and Average latency values near the top of the output.

What should I see?

Your exact numbers will vary depending on your machine, but expect something in the ballpark of 100-180 requests/sec with an average latency around 55-70ms. The 50ms simulated delay is the main bottleneck.

Write down your Requests/sec and Average latency. This is your baseline. Every improvement from here will be measured against these numbers.

  • Go back to the terminal running your FastAPI server.
  • Press Ctrl+C to stop the development server.

Getting connection refused from hey?

  • Make sure the FastAPI server is still running in your other terminal window. If you accidentally closed it, run fastapi dev app/main.py again from the scaling-api directory.
  • Check that the URL uses port 8000 and 127.0.0.1, not localhost. Some systems resolve localhost to IPv6, which FastAPI's dev server doesn't bind to by default.

Help me troubleshoot a connection refused error with hey and FastAPI.

You've got your API working and your baseline numbers recorded. Next up, you'll package this API into a Docker container so it's ready to be scaled.

Containerize the API with Docker

In the last step, you ran your API directly on your machine and recorded baseline performance numbers. Those numbers give you a reference point for measuring every improvement from here.

The next step toward a scalable system is packaging your API into a Docker container. Containers guarantee your app runs identically regardless of the host machine.

In this step, get ready to:
  • Create a Dockerfile that packages your API into a container image.
  • Build and run the containerized API.
  • Load test the container and compare results to your baseline.
Create the Dockerfile

A Dockerfile is a set of instructions that tells Docker how to package your application into an image. Each instruction adds a layer to the image, building up from a base operating system to your running app.

  • In your code editor, create a new file called Dockerfile inside your app/ folder, then paste in the following code:
# Use a lightweight Python image as the base
FROM python:3.12-slim

# Set the working directory inside the container
WORKDIR /app

# Copy and install dependencies first (for Docker layer caching)
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy the application code
COPY main.py .

# Document which port the container listens on
EXPOSE 8000

# Start the API server
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

What does this Dockerfile do?

  • FROM python:3.12-slim starts from a minimal Python 3.12 image, keeping the container small.
  • COPY requirements.txt and RUN pip install install your Python dependencies. These are copied separately from the app code so Docker can cache this layer when only your code changes.
  • COPY main.py . copies your application code into the container.
  • CMD starts uvicorn bound to 0.0.0.0 so the container accepts connections from the outside world.
  • Save app/Dockerfile.
  • Confirm the file exists by running:
ls app/Dockerfile

You should see app/Dockerfile printed in the output.

Don't see the file?

  • Make sure you created the file inside the app/ folder, not in the scaling-api/ root.
  • The file must be named exactly Dockerfile with a capital D and no file extension.
  • Help me figure out why my Dockerfile isn't being found.

✔️ Awesome, I've got everything!

Great. Make sure you've saved app/Dockerfile before moving on.

ⓧ I'd like to double check the full code

FROM python:3.12-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY main.py .

EXPOSE 8000

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
Build and run the container

Your Dockerfile is the recipe. Now you need to bake it into a Docker image and then run a container from that image.

  • Build your Docker image by running this command:
docker build -t scaling-api ./app

What does this command do?

  • docker build reads your Dockerfile and creates a Docker image from it.
  • -t scaling-api tags (names) the image scaling-api so you can reference it later.
  • ./app tells Docker to use the app/ directory as the build context, which is where your Dockerfile lives.
  • You should see output ending with a line like Successfully tagged scaling-api:latest.

Build failed?

  • Check that Docker Desktop is running. The whale icon in your menu bar should be steady and not animating.
  • Make sure your terminal is in the scaling-api/ directory, not inside app/.
  • Verify that app/Dockerfile exists and is spelled correctly (capital D, no extension).
  • Help me debug my Docker build failure.
  • Run a container from your new image by running this command:
docker run -d -p 8000:8000 --name api-single scaling-api

What do these flags mean?

  • -d runs the container in the background (detached mode).
  • -p 8000:8000 maps port 8000 on your machine to port 8000 inside the container.
  • --name api-single gives the container a memorable name so you can reference it easily.
  • scaling-api is the image you just built.
  • Open http://localhost:8000/ in your browser.

You should see a JSON response with a hostname field containing a short hex string like a1b2c3d4e5f6. This is the container's ID, proving the API is running inside Docker instead of directly on your machine.

  • Test the products endpoint by running this command:
curl http://localhost:8000/products

You should see the list of 5 products in JSON format. Notice the "source": "database" field in the response. There is no Redis server running yet, so every request hits the simulated database.

Can't connect to localhost:8000?

  • Run docker ps to check if the container is running. If it's not listed, run docker logs api-single to see what went wrong.
  • Make sure no other process is using port 8000. If you left the FastAPI dev server running from Step 2, stop it first with Ctrl+C.
  • Help me debug why my Docker container isn't responding on port 8000.
Load test the containerized API

You now have the same API running inside a container instead of directly on your machine. Time to measure whether containerization adds any performance overhead.

  • Run a load test against the containerized API by running this command:
hey -n 200 -c 10 http://localhost:8000/products

What should I see?

Look at the Requests/sec and Average latency values in the output. Compare these to the numbers you recorded in Step 2.

The results should be very similar, confirming that Docker adds minimal overhead to your API's performance. The bottleneck is the 50ms simulated database delay, not the container itself.

You're done with the single container. Time to clean it up before moving on to multi-instance scaling.

  • Stop and remove the container by running this command:
docker stop api-single && docker rm api-single
  • Confirm the container is gone by running:
docker ps

You should see an empty table with just the column headers. No containers are running.

Container won't stop?

  • If docker stop hangs, try docker kill api-single to force-stop it.
  • If you see No such container: api-single, the container was already stopped and removed. You can move on.
  • Help me figure out why my Docker container won't stop.

Your API runs just as fast inside a container as it does on bare metal. Next up, you'll scale to multiple container instances behind an Nginx load balancer.

Scale with Nginx Load Balancing

Your API is containerized and running in Docker. A single container handles requests just fine for now, but in the real world, one instance can only handle so much traffic before it starts dropping requests or slowing down.

The solution is horizontal scaling. Instead of making one container bigger, you run multiple copies and spread the traffic across them. In this step, you'll put Nginx in front of 3 API instances and use Docker Compose to orchestrate the whole system.

In this step, get ready to:
  • Configure Nginx as a reverse proxy for your API.
  • Orchestrate multiple containers with Docker Compose.
  • Load test the scaled system and compare throughput to your single-container baseline.
Create the Nginx configuration

Right now, if you wanted to run 3 copies of your API, each on a different port, you'd need something to sit in front and decide which copy handles each incoming request. That's exactly what a reverse proxy does.

What is a reverse proxy?

A reverse proxy is a server that sits between clients and your backend services. Clients send all requests to the proxy, and the proxy forwards each request to one of the backend instances.

Nginx is one of the most popular reverse proxies. When combined with Docker's internal DNS, it automatically distributes traffic across all copies of a service using round-robin rotation. Each new request goes to the next container in line.

  • Create the Nginx config file by opening nginx/nginx.conf in your editor and pasting the following:
events {
    worker_connections 1024;
}

http {
    upstream api_backend {
        server api:8000;
    }

    server {
        listen 80;

        location / {
            proxy_pass http://api_backend;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }
    }
}

What does this configuration do?

  • upstream api_backend defines a group of backend servers. The name api:8000 refers to Docker Compose's internal DNS. Docker resolves api to the IP addresses of all running api containers automatically.
  • listen 80 tells Nginx to accept traffic on port 80, the standard HTTP port.
  • proxy_pass http://api_backend forwards every incoming request to the upstream group. Nginx picks a different container for each request.
  • The proxy_set_header lines pass the original client's IP and hostname through to the backend, so your API knows who actually made the request.
  • Save nginx/nginx.conf.

You won't see a change yet. Nginx needs to run inside Docker Compose to use this config, which you'll set up next.

✔️ Awesome, I've got everything!

Double check you've saved your file before moving on.

ⓧ I'd like to double check the full code

events {
    worker_connections 1024;
}

http {
    upstream api_backend {
        server api:8000;
    }

    server {
        listen 80;

        location / {
            proxy_pass http://api_backend;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }
    }
}
Create the Docker Compose file

You need a way to start your API, Nginx, and a Redis server together as a single system. Docker Compose lets you define all your services in one file and start them with a single command.

  • Create docker-compose.yml in the scaling-api root folder (not inside app/) and paste the following:
services:
  api:
    build: ./app
    expose:
      - "8000"
    environment:
      - REDIS_URL=redis://redis:6379
    depends_on:
      - redis

  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
    depends_on:
      - api

  redis:
    image: redis:7-alpine

What does this file define?

  • api builds your FastAPI app from the Dockerfile in ./app. It uses expose instead of ports, which means port 8000 is only accessible to other containers in the stack, not from your Mac.
  • nginx pulls the lightweight nginx:alpine image, maps port 80 to your Mac's port 80, and mounts the config file you just wrote. The :ro suffix means read-only.
  • redis pulls the redis:7-alpine image. Your API will connect to it using the REDIS_URL environment variable. You'll explore caching in detail in the next step.
  • depends_on controls startup order. Redis starts first, then the API containers, then Nginx.
  • Save docker-compose.yml.

Why is only Nginx exposed on port 80?

All traffic enters through Nginx on port 80. Nginx then forwards it to one of the API containers on port 8000 internally. The API containers never need to be directly reachable from your Mac. This is the standard pattern for load-balanced systems.

Now you'll start the entire stack and tell Docker Compose to create 3 copies of the API service.

  • Start the stack with 3 API instances by running this command:
docker compose up -d --build --scale api=3

What does this command do?

  • up creates and starts all the services defined in docker-compose.yml.
  • -d runs everything in the background so you get your terminal back.
  • --build rebuilds the Docker image from your Dockerfile before starting.
  • --scale api=3 tells Docker Compose to run 3 separate containers of the api service instead of just 1.
  • Verify that all containers are running by running:
docker compose ps

You should see 5 containers listed: 3 api containers, 1 nginx container, and 1 redis container. All should show a status of Up.

Containers not starting?

  • If you see a port conflict error mentioning port 80, another process is already using it. Run lsof -i :80 to find it, stop that process, then try again.
  • If you see a port conflict on port 8000, make sure you stopped the standalone container from Step 3. Run docker ps to check for leftover containers.
  • If you see a build error, check that docker-compose.yml is in the scaling-api root folder, not inside app/.

Help me debug my Docker Compose setup

✔️ Awesome, I've got everything!

Double check you've saved both files before moving on.

ⓧ I'd like to double check the full code

events {
    worker_connections 1024;
}

http {
    upstream api_backend {
        server api:8000;
    }

    server {
        listen 80;

        location / {
            proxy_pass http://api_backend;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }
    }
}
services:
  api:
    build: ./app
    expose:
      - "8000"
    environment:
      - REDIS_URL=redis://redis:6379
    depends_on:
      - redis

  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
    depends_on:
      - api

  redis:
    image: redis:7-alpine
Test load balancing and measure performance

Your 3 API instances are running behind Nginx, but how do you know traffic is actually being spread across all of them? Each API container returns its own container hostname in the response. If load balancing is working, hitting the same URL repeatedly should return different hostnames.

  • Verify that Nginx is distributing requests across all 3 instances by running:
for i in {1..6}; do curl -s http://localhost/; echo; done

You should see 6 JSON responses, each with a hostname field. The hostnames should cycle through 2-3 different values. These are the container IDs of your 3 API instances.

Why different hostnames?

Each API container has its own unique hostname (a short hex string like a3f2b1c9e8d7). When Nginx receives a request, Docker's internal DNS resolves the service name api to the IP of the next container in the rotation.

This is round-robin load balancing. Each request goes to the next instance in line, spreading the work evenly across all 3 containers.

Seeing the same hostname every time?

  • Check that you're hitting http://localhost/ (port 80, through Nginx), not http://localhost:8000/ (which would bypass Nginx).
  • Verify you have 3 API instances running with docker compose ps. If you only see 1, re-run the docker compose up command with --scale api=3.

Why am I seeing the same hostname in every response?

Now it's time to measure the real performance impact of running 3 instances instead of 1.

  • Run a load test against the scaled system by running:
hey -n 500 -c 20 http://localhost/products

What does this command do?

This sends 500 total requests with 20 hitting the server at the same time. Compared to the Step 3 test (200 requests, 10 concurrent), this is a heavier workload. With 3 instances sharing the load, your system should handle it comfortably.

When the test finishes, look at the Requests/sec and Average latency lines in the output. Compare these to the numbers you recorded in Step 3.

What should I expect?

With 3 instances behind Nginx, throughput (requests per second) should be noticeably higher than a single container. The improvement comes from three containers handling requests in parallel instead of one container handling them sequentially.

Your exact numbers will vary depending on your machine, but the trend should be clear. Write down your Requests/sec number. You'll compare it again after adding caching in the next step.

Leave the stack running. You'll build on top of it in the next step.

Your API is running across 3 containers with Nginx spreading traffic evenly. Next up, you'll take a closer look at Redis caching and see what happens when repeated queries skip the simulated database entirely.

Add Redis Caching

In Step 4, you scaled your API to 3 instances behind an Nginx load balancer. Traffic now gets distributed across all three containers, and your throughput improved.

But there's still a bottleneck. Every single request hits the 50ms "database" delay, no matter how many instances you add. Redis caching solves this by storing the results of expensive queries in memory. Once a product list is fetched once, every subsequent request skips the database entirely and gets an instant response.

In this step, get ready to:
  • Review the Redis configuration in your Docker Compose stack.
  • Walk through the cache-aside pattern in your API code.
  • Rebuild the stack and measure the performance improvement with load tests.
Review the Redis configuration in Docker Compose

Your docker-compose.yml already has Redis wired into the stack from Step 4. Before you test it, take a moment to understand how each piece fits together.

  • Open docker-compose.yml in your editor and find the redis service at the bottom of the file:
  redis:
    image: redis:7-alpine
  • Now find the environment and depends_on sections inside the api service:
  api:
    build: ./app
    expose:
      - "8000"
    environment:
      - REDIS_URL=redis://redis:6379
    depends_on:
      - redis

What do these three pieces do?

  • redis: image: redis:7-alpine adds a Redis container to your stack using the lightweight Alpine-based image.
  • REDIS_URL=redis://redis:6379 tells each API container where to find Redis. The hostname redis is automatically resolved by Docker's internal DNS to the Redis container's IP address.
  • depends_on: - redis ensures Redis starts before the API containers boot up.

All three API instances share a single Redis container. This means a product cached by one instance is instantly available to the others.

  • Stop the current stack by running this command:
docker compose down
  • You should see messages confirming each container has been stopped and removed.

Seeing a "no configuration file" error?

Make sure your terminal is in the scaling-api folder, where your docker-compose.yml file lives.

Help me troubleshoot this Docker Compose error.

Walk through the caching code in your API

Your app/main.py already includes the caching logic. It uses a pattern called cache-aside, which is the most common caching strategy in production systems.

What is cache-aside?

With cache-aside, the application checks the cache before querying the database. If the data is already cached (a cache hit), it returns immediately. If the data is not cached (a cache miss), it queries the database, stores the result in the cache for next time, and then returns.

Each cached entry has a TTL (time to live) so stale data eventually expires. In your API, the TTL is 30 seconds.

  • Open app/main.py and find the Redis connection block near the top of the file:
# Redis connection
REDIS_URL = os.getenv("REDIS_URL", "redis://localhost:6379")
try:
    redis_client = redis.from_url(REDIS_URL, decode_responses=True)
    redis_client.ping()
    redis_available = True
except Exception:
    redis_client = None
    redis_available = False

CACHE_TTL = 30  # seconds

What does this connection code do?

  • os.getenv("REDIS_URL") reads the Redis URL from the environment variable that Docker Compose sets. The fallback redis://localhost:6379 is used when running outside Docker.
  • redis_client.ping() tests the connection immediately. If Redis is not available, the API falls back gracefully with redis_available = False and skips all caching.
  • CACHE_TTL = 30 means cached data expires after 30 seconds, ensuring the API eventually returns fresh data.
  • Now scroll down to the get_products() function. This is where the cache-aside pattern lives:
@app.get("/products")
async def get_products():
    # Check cache first
    if redis_available:
        cached = redis_client.get("products:all")
        if cached:
            return json.loads(cached)

    # Simulate slow database query
    await simulate_db_delay()

    result = {"products": PRODUCTS, "source": "database", "served_by": HOSTNAME}

    # Store in cache
    if redis_available:
        redis_client.setex("products:all", CACHE_TTL, json.dumps(result))

    return result

How does the caching logic work?

  • redis_client.get("products:all") checks if a cached result exists under the key products:all. If it does, the function returns immediately without touching the database.
  • If the cache is empty, the function runs the full 50ms database simulation. Then redis_client.setex() stores the result with a 30-second TTL so future requests skip the delay.
  • The get_product() function below uses the same pattern, but with a unique key per product (e.g., products:1, products:2).

Your API also has a GET /cache-status endpoint that reports whether Redis is connected. You will use this to confirm caching is active after you rebuild the stack.

✔️ Awesome, I've got everything!

Double check you have both files saved before moving on.

ⓧ I'd like to double check the full code

Here is the complete docker-compose.yml:

services:
  api:
    build: ./app
    expose:
      - "8000"
    environment:
      - REDIS_URL=redis://redis:6379
    depends_on:
      - redis

  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
    depends_on:
      - api

  redis:
    image: redis:7-alpine

And here is the complete app/main.py:

import asyncio
import json
import os
import socket

from fastapi import FastAPI

import redis

app = FastAPI(title="Product Catalog API")

# Simulated product "database"
PRODUCTS = [
    {"id": 1, "name": "Mechanical Keyboard", "price": 129.99, "category": "Electronics"},
    {"id": 2, "name": "Standing Desk", "price": 499.99, "category": "Furniture"},
    {"id": 3, "name": "Noise-Canceling Headphones", "price": 249.99, "category": "Electronics"},
    {"id": 4, "name": "Ergonomic Chair", "price": 399.99, "category": "Furniture"},
    {"id": 5, "name": "Ultrawide Monitor", "price": 599.99, "category": "Electronics"},
]

HOSTNAME = socket.gethostname()

# Redis connection
REDIS_URL = os.getenv("REDIS_URL", "redis://localhost:6379")
try:
    redis_client = redis.from_url(REDIS_URL, decode_responses=True)
    redis_client.ping()
    redis_available = True
except Exception:
    redis_client = None
    redis_available = False

CACHE_TTL = 30  # seconds


async def simulate_db_delay():
    """Simulate a slow database query."""
    await asyncio.sleep(0.05)


@app.get("/")
async def root():
    return {"hostname": HOSTNAME, "status": "running"}


@app.get("/products")
async def get_products():
    # Check cache first
    if redis_available:
        cached = redis_client.get("products:all")
        if cached:
            return json.loads(cached)

    # Simulate slow database query
    await simulate_db_delay()

    result = {"products": PRODUCTS, "source": "database", "served_by": HOSTNAME}

    # Store in cache
    if redis_available:
        redis_client.setex("products:all", CACHE_TTL, json.dumps(result))

    return result


@app.get("/products/{product_id}")
async def get_product(product_id: int):
    cache_key = f"products:{product_id}"

    # Check cache first
    if redis_available:
        cached = redis_client.get(cache_key)
        if cached:
            return json.loads(cached)

    # Simulate slow database query
    await simulate_db_delay()

    product = next((p for p in PRODUCTS if p["id"] == product_id), None)
    if not product:
        return {"error": "Product not found", "served_by": HOSTNAME}

    result = {"product": product, "source": "database", "served_by": HOSTNAME}

    # Store in cache
    if redis_available:
        redis_client.setex(cache_key, CACHE_TTL, json.dumps(result))

    return result


@app.get("/cache-status")
async def cache_status():
    return {
        "redis_connected": redis_available,
        "redis_url": REDIS_URL,
        "served_by": HOSTNAME,
    }
Rebuild, test, and measure the improvement

Time to see caching in action. You will rebuild the entire stack, test that Redis is connected, observe the difference between cache hits and misses, and then run a load test to measure the improvement.

  • Rebuild and start the full stack with 3 API instances by running this command:
docker compose up -d --build --scale api=3
  • Verify all 5 containers are running by running:
docker compose ps

You should see 5 containers listed: 3 api instances, 1 nginx instance, and 1 redis instance. All should show a status of Up.

Containers exiting or not starting?

Check the logs for the failing container by running docker compose logs api or docker compose logs redis. A common cause is a syntax error in main.py. Use the double-check tab above to compare your code.

Help me debug why my Docker Compose containers are not starting.

  • Confirm Redis is connected to the API by running:
curl http://localhost/cache-status

You should see "redis_connected": true in the response. This confirms your API containers can reach the Redis container through Docker's internal network.

Seeing redis_connected: false?

Make sure the REDIS_URL environment variable in your docker-compose.yml is set to redis://redis:6379. The hostname redis must match the service name exactly. Also confirm the redis container is running with docker compose ps.

Help me figure out why my API cannot connect to Redis.

Now test the cache-aside pattern by hitting the same endpoint twice.

  • Fetch the product list for the first time by running:
curl http://localhost/products

This is a cache miss. The response includes "source": "database", meaning the API ran the full 50ms database simulation and then stored the result in Redis.

  • Run the exact same command again:
curl http://localhost/products

This time the response comes back near-instantly. The data was served straight from Redis, skipping the 50ms delay entirely. You just saw the cache-aside pattern in action.

Why does the second request still show "source": "database"?

The source field was set when the data was first cached. Redis returns the exact JSON that was stored, including that original field. The key proof is the speed difference, not the source field.

Time for the real test. Let's run a load test and compare the results to your Step 4 numbers.

  • Run a load test against the cached endpoint by running:
hey -n 500 -c 20 http://localhost/products

What do these flags mean?

  • -n 500 sends 500 total requests.
  • -c 20 sends 20 requests at a time (concurrently).

The first request triggers a cache miss, but the remaining 499 all hit the Redis cache. This means almost every request skips the 50ms database delay.

  • Compare the Requests/sec and Average latency in the output to your Step 4 results.

With caching, the throughput should jump dramatically and the average latency should drop. Most requests now skip the 50ms database simulation, so the bottleneck you had before is gone.

  • Verify that Redis has stored the cached data by running:
docker compose exec redis redis-cli KEYS '*'

You should see products:all in the output. If you also tested the single-product endpoint, you will see keys like products:1 as well.

No keys showing up?

Redis keys expire after 30 seconds (the TTL you set in the code). If you waited too long between running curl and checking the keys, they may have already expired. Hit http://localhost/products again, then immediately check the keys.

Help me figure out why Redis has no cached keys.

Your system now has all three scaling layers working together: containerization, load balancing, and caching. Next up, you will put your skills to the test with a Secret Mission that adds real-time monitoring across all of your instances.

Secret mission

Build a Real-Time Stats Dashboard

Ready for more? Add a /stats endpoint that tracks cache hit/miss ratios and per-instance request counts in real time. See exactly how your load balancer distributes traffic across all 3 containers.

Before you go

Clean Up Your Resources

All resources in this project run locally via Docker. There are no cloud costs, but running containers do use system resources like RAM and CPU.

Decide whether to keep your containers running, pause them to come back later, or delete them entirely.

Resources you used:

  • Docker containers: 3 API instances, 1 Nginx container, 1 Redis container.
  • Docker images: scaling-api (custom-built), nginx:alpine, redis:7-alpine, python:3.12-slim.
  • Project files in the scaling-api folder on your Desktop.
  • Python virtual environment (.venv) inside the project folder.
  • The hey load testing tool installed via Homebrew.

Keep everything running

No action needed. Choose this if you're still actively experimenting with your scaled system or want to keep tweaking the configuration.

  • Your containers will keep running until you stop them or restart your machine.
  • Docker Desktop uses some system RAM and CPU while containers are active.
  • You can check which containers are running at any time with docker compose ps.

Pause - I'll come back to this later

Stop your containers without removing them. This frees up system resources while keeping your setup intact so you can resume later.

  • Stop all containers by running:
docker compose stop
  • When you're ready to resume, navigate back to your scaling-api folder and run:
docker compose start
  • Your containers will start up with the same configuration as before.

Delete - I don't want to use this again

Remove all project containers, images, and volumes to start completely fresh.

  • Stop and remove all containers by running:
docker compose down
  • Remove the custom Docker image by running:
docker rmi scaling-api
  • Clean up any unused Docker volumes by running:
docker volume prune
  • Docker will ask you to confirm the volume prune. Type y and press Enter.
  • To remove the project files, delete the scaling-api folder from your Desktop.
  • To uninstall hey, run:
brew uninstall hey

Mission accomplished

That's a Wrap!

Nice work! You just built a scaled system from scratch and proved why each layer matters with real numbers.

You started with a single FastAPI process, containerized it with Docker, scaled it behind Nginx, and added Redis caching. At every stage, load tests gave you measurable proof that each architectural layer made a real difference.

You've learned how to:

  • Build and containerize a FastAPI product catalog API with a Dockerfile and Docker Compose.
  • Scale to multiple instances behind an Nginx load balancer using round-robin DNS and measure the throughput increase with load tests.
  • Integrate Redis caching using the cache-aside pattern to eliminate redundant database queries and prove the latency reduction with real numbers.
  • Secret Mission: Build a real-time /stats endpoint that tracks cache hit/miss ratios and per-instance request distribution across your scaled containers.

Ready to quiz yourself?