Load-Balanced, Cached API with Nginx

Build a multi-container API with Nginx load balancing and Redis caching.

Introduction

30 Second Summary

Every time you visit a popular website, your request is silently routed to one of many identical servers. A shared cache ensures you get fast responses regardless of which server handles your request.

In this project, you will build a multi-container system that demonstrates load balancing and caching. You'll deploy three identical API instances behind Nginx, add Redis as a shared cache, and measure the performance difference yourself.

What You'll Build

A product catalog API with 3 replicas running behind Nginx as a reverse proxy and load balancer, with Redis providing a shared cache layer.

By the end of this project, you'll have:

  • A FastAPI service deployed as 3 identical instances inside Docker containers, each identifiable by a unique instance ID.
  • An Nginx reverse proxy distributing traffic across all three instances using round-robin load balancing, with automatic failover when an instance goes down.
  • A Redis caching layer that reduces response times from ~500ms to ~1ms, shared across all instances.
  • Secret Mission: Experiment with least-connections and weighted load balancing strategies to compare their behavior under uneven load.

Are there any prerequisites?

You'll need Docker (with Docker Compose) installed on your machine, basic Python knowledge, and familiarity with REST APIs. Everything runs in containers, so no other local setup is required.

Set Up the Project and Build the API

Every scalable web service starts the same way: a single API that works correctly on its own, before you worry about distributing traffic or optimizing performance.

In this step, you will create the FastAPI application that serves as the product catalog service. You will also containerize it with Docker so it can be replicated and run behind a load balancer in the next steps.

In this step, get ready to:
  • Create the project directory structure with subdirectories for the app and Nginx configuration.
  • Write the FastAPI application with Redis-backed caching and instance identification.
  • Set up the Dockerfile and dependencies to containerize the service.
Create the project directory structure

You need a root project folder with two subdirectories: one for the Python application code and one for the Nginx configuration you will write in the next step.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create the project folder with its subdirectories and move into it by running these commands:
mkdir -p load-balanced-api/app load-balanced-api/nginx
cd load-balanced-api
  • Confirm the directories exist by running:
ls

You should see app and nginx listed in the output.

Don't see both directories?

Make sure you ran the mkdir -p command exactly as shown. The -p flag creates parent directories as needed.

Help me troubleshoot my directory creation.

Now you will write the FastAPI application. This is the service that will eventually run as three replicas behind the load balancer.

  • Create app/main.py and add the following imports and configuration code:
import os
import time
import json

from fastapi import FastAPI
import redis

# Create the FastAPI application
app = FastAPI(title="Product Catalog API")

# Each container instance gets a unique ID from an environment variable
INSTANCE_ID = os.environ.get("INSTANCE_ID", "unknown")

# Connect to the shared Redis cache (hostname resolves via Docker networking)
redis_client = redis.Redis(
    host=os.environ.get("REDIS_HOST", "redis"),
    port=int(os.environ.get("REDIS_PORT", "6379")),
    decode_responses=True,
)

# Simulated product database (in production, this would be a real database)
PRODUCTS = {
    "1": {"id": "1", "name": "Laptop", "price": 999.99, "category": "electronics"},
    "2": {"id": "2", "name": "Headphones", "price": 149.99, "category": "electronics"},
    "3": {"id": "3", "name": "Coffee Maker", "price": 79.99, "category": "kitchen"},
    "4": {"id": "4", "name": "Running Shoes", "price": 129.99, "category": "sports"},
    "5": {"id": "5", "name": "Backpack", "price": 59.99, "category": "travel"},
}

# Cache entries expire after 30 seconds
CACHE_TTL = 30

What does this code do?

  • INSTANCE_ID reads an environment variable so each container can identify itself. When you see which instance handled a request, you know load balancing is working.
  • redis_client connects to a Redis server by hostname. Docker networking resolves redis to the Redis container automatically.
  • PRODUCTS simulates a database. The simulated delay you will add next makes cache hits dramatically visible.
  • CACHE_TTL sets a 30-second time-to-live so cached data expires and forces a fresh lookup periodically.

You won't see a change yet. The next chunk adds the API endpoints that use this setup.

  • Below the CACHE_TTL = 30 line, add the health check and product listing endpoints:
@app.get("/")
def health_check():
    # Returns which instance handled this request
    return {"status": "healthy", "instance": INSTANCE_ID}


@app.get("/products")
def list_products():
    cache_key = "products:all"
    # Check Redis cache first (cache-aside pattern)
    cached = redis_client.get(cache_key)

    if cached:
        # Cache hit - return immediately without database delay
        return {
            "source": "cache",
            "instance": INSTANCE_ID,
            "products": json.loads(cached),
        }

    # Cache miss - simulate slow database query (500ms)
    time.sleep(0.5)
    products = list(PRODUCTS.values())

    # Store result in cache for subsequent requests
    redis_client.set(cache_key, json.dumps(products), ex=CACHE_TTL)

    return {
        "source": "database",
        "instance": INSTANCE_ID,
        "products": products,
    }

What is the cache-aside pattern?

The cache-aside pattern (also called lazy-loading) follows three steps: check the cache first, fall back to the database on a miss, then populate the cache before returning.

The time.sleep(0.5) simulates a slow database query. When you test later, cache hits will return in under 5ms while misses take ~500ms. That dramatic difference makes caching performance unmistakable.

You won't see a change yet. One more set of endpoints to add.

  • Below the list_products function, add the single-product lookup and cache-clearing endpoints:
@app.get("/products/{product_id}")
def get_product(product_id: str):
    cache_key = f"products:{product_id}"
    cached = redis_client.get(cache_key)

    if cached:
        return {
            "source": "cache",
            "instance": INSTANCE_ID,
            "product": json.loads(cached),
        }

    # Simulate database lookup delay (300ms)
    time.sleep(0.3)
    product = PRODUCTS.get(product_id)

    if not product:
        return {"error": "Product not found", "instance": INSTANCE_ID}

    redis_client.set(cache_key, json.dumps(product), ex=CACHE_TTL)

    return {
        "source": "database",
        "instance": INSTANCE_ID,
        "product": product,
    }


@app.delete("/cache")
def clear_cache():
    # Flush the entire Redis database to reset all cached data
    redis_client.flushdb()
    return {"message": "Cache cleared", "instance": INSTANCE_ID}

What do these endpoints do?

  • get_product follows the same cache-aside pattern but for a single item. The 300ms delay is shorter because single lookups are faster than listing all products.
  • clear_cache flushes the entire Redis database. You will use this endpoint to reset state between test runs so you can observe cache misses on demand.
  • Every response includes the instance field. This is how you will prove load balancing is working when requests hit different containers.
  • Save app/main.py.

Seeing a syntax error when you save?

Check that each function decorator (@app.get, @app.delete) sits on the line directly above its function definition with no blank line in between.

Also confirm that the return statements inside if cached: blocks are indented one level deeper than the if keyword.

Help me debug my main.py syntax error.

✔️ Awesome, I've got everything!

Great. Double check you have saved app/main.py before moving on.

ⓧ I'd like to double check the full code

import os
import time
import json

from fastapi import FastAPI
import redis

app = FastAPI(title="Product Catalog API")

INSTANCE_ID = os.environ.get("INSTANCE_ID", "unknown")

redis_client = redis.Redis(
    host=os.environ.get("REDIS_HOST", "redis"),
    port=int(os.environ.get("REDIS_PORT", "6379")),
    decode_responses=True,
)

PRODUCTS = {
    "1": {"id": "1", "name": "Laptop", "price": 999.99, "category": "electronics"},
    "2": {"id": "2", "name": "Headphones", "price": 149.99, "category": "electronics"},
    "3": {"id": "3", "name": "Coffee Maker", "price": 79.99, "category": "kitchen"},
    "4": {"id": "4", "name": "Running Shoes", "price": 129.99, "category": "sports"},
    "5": {"id": "5", "name": "Backpack", "price": 59.99, "category": "travel"},
}

CACHE_TTL = 30


@app.get("/")
def health_check():
    return {"status": "healthy", "instance": INSTANCE_ID}


@app.get("/products")
def list_products():
    cache_key = "products:all"
    cached = redis_client.get(cache_key)

    if cached:
        return {
            "source": "cache",
            "instance": INSTANCE_ID,
            "products": json.loads(cached),
        }

    time.sleep(0.5)
    products = list(PRODUCTS.values())

    redis_client.set(cache_key, json.dumps(products), ex=CACHE_TTL)

    return {
        "source": "database",
        "instance": INSTANCE_ID,
        "products": products,
    }


@app.get("/products/{product_id}")
def get_product(product_id: str):
    cache_key = f"products:{product_id}"
    cached = redis_client.get(cache_key)

    if cached:
        return {
            "source": "cache",
            "instance": INSTANCE_ID,
            "product": json.loads(cached),
        }

    time.sleep(0.3)
    product = PRODUCTS.get(product_id)

    if not product:
        return {"error": "Product not found", "instance": INSTANCE_ID}

    redis_client.set(cache_key, json.dumps(product), ex=CACHE_TTL)

    return {
        "source": "database",
        "instance": INSTANCE_ID,
        "product": product,
    }


@app.delete("/cache")
def clear_cache():
    redis_client.flushdb()
    return {"message": "Cache cleared", "instance": INSTANCE_ID}
Create the Python dependencies and Dockerfile

The application needs two Python packages: FastAPI (with its built-in Uvicorn server) and the Redis client library. You will pin exact versions so the build is reproducible.

  • Create app/requirements.txt with the following content:
fastapi[standard]==0.136.3
redis==8.0.0

What do these dependencies do?

  • fastapi[standard]==0.136.3 installs FastAPI plus Uvicorn (the ASGI server) as a bundled standard extra.
  • redis==8.0.0 installs the Python Redis client for connecting to the Redis cache container.
  • Save app/requirements.txt.
  • Create app/Dockerfile with the following content:
# Use a lightweight Python image
FROM python:3.12-slim

WORKDIR /app

# Install dependencies first (leverages Docker layer caching)
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy the application code
COPY main.py .

# Start the FastAPI server on port 8000
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Why copy requirements.txt before main.py?

Docker builds each instruction as a cached layer. By copying and installing dependencies first, Docker only re-runs pip install when requirements.txt changes.

Code changes to main.py rebuild only the final layer, making rebuilds much faster.

  • Save app/Dockerfile.

Not sure if the files are in the right place?

All three files (main.py, requirements.txt, Dockerfile) must be inside the app/ directory, not in the project root.

The Dockerfile references requirements.txt and main.py as relative paths from its own location.

Help me verify my project structure.

✔️ Awesome, I've got everything!

Great. Make sure all files inside app/ are saved before continuing.

ⓧ I'd like to double check the full code

Here is app/requirements.txt:

fastapi[standard]==0.136.3
redis==8.0.0

Here is app/Dockerfile:

FROM python:3.12-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY main.py .

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
Verify the project structure
  • Confirm the app directory contents by running:
ls app/

You should see Dockerfile, main.py, and requirements.txt listed.

  • Confirm the nginx directory exists (it should be empty for now) by running:
ls nginx/

This should return no output, confirming the directory is empty and ready for the Nginx configuration in the next step.

Your API code and container configuration are ready. Next up, you will write the Nginx configuration that distributes incoming traffic across multiple instances of this service.

Configure Nginx as a Load Balancer

Your API is ready to run inside containers, but right now there is no way to route incoming traffic across multiple instances.

In this step, you will configure Nginx as a reverse proxy that distributes requests evenly across your three app instances using round-robin load balancing.

In this step, get ready to:
  • Create the Nginx configuration file with an upstream server group.
  • Understand the key directives that control traffic routing.
Create the Nginx configuration

Nginx needs a configuration file that tells it where to find backend servers and how to forward requests to them.

  • Create a file called nginx.conf inside your nginx/ directory by running this command:
touch nginx/nginx.conf
  • Open nginx/nginx.conf in your code editor and paste the following configuration:
events {
    worker_connections 1024;
}

http {
    # Define a group of backend servers for load balancing
    upstream api_servers {
        server app1:8000;
        server app2:8000;
        server app3:8000;
    }

    # Listen for incoming requests and forward them to the upstream group
    server {
        listen 80;

        location / {
            proxy_pass http://api_servers;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }
    }
}

What does this configuration do?

  • upstream api_servers defines a group of three backend servers. Nginx distributes incoming requests across them.
  • proxy_pass http://api_servers forwards every request that hits port 80 to one of the servers in the upstream group.
  • proxy_set_header lines pass the original client's IP and hostname to the backend so your app knows who made the request.
  • Save nginx/nginx.conf.
  • Confirm the file exists by running:
ls nginx/

You should see nginx.conf listed in the output.

Don't see nginx.conf?

  • Make sure your terminal is in the project root directory (the folder containing both app/ and nginx/).
  • Check that the nginx/ directory was created in Step 1. If not, create it with mkdir nginx before retrying.
  • Help me troubleshoot why my nginx.conf file wasn't created in the nginx/ directory.

✔️ Awesome, I've got everything!

Great. Double check you have saved nginx/nginx.conf before continuing.

ⓧ I'd like to double check the full code

Here is the complete nginx/nginx.conf file for comparison:

events {
    worker_connections 1024;
}

http {
    upstream api_servers {
        server app1:8000;
        server app2:8000;
        server app3:8000;
    }

    server {
        listen 80;

        location / {
            proxy_pass http://api_servers;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }
    }
}
Understand the key Nginx directives

The configuration you just wrote uses a small set of directives that form the core of any Nginx load balancer setup.

What is an upstream block?

The upstream directive defines a named group of backend servers. When no load balancing method is specified, Nginx defaults to round-robin. This means each new request goes to the next server in the list, cycling through all three in order.

The server names app1, app2, and app3 are Docker Compose service names. Docker's internal DNS resolves them to the correct container IP addresses automatically.

Why pass proxy headers?

Without proxy_set_header, your backend only sees Nginx's IP address as the request source. The three headers preserve the original client information:

  • Host passes the original hostname the client used.
  • X-Real-IP passes the client's actual IP address.
  • X-Forwarded-For maintains the chain of proxies the request has passed through.

Your load balancer configuration is ready. Next up, you will wire everything together with Docker Compose and launch all five containers.

Orchestrate with Docker Compose

You have all the individual pieces ready: a FastAPI app with its Dockerfile, and an Nginx configuration that knows how to route traffic to three backends.

The missing piece is something to bring all these containers to life together. Docker Compose lets you define your entire multi-container system in a single YAML file and launch everything with one command.

In this step, get ready to:
  • Create a Docker Compose file defining all 5 services.
  • Build and launch the full container stack.
  • Verify the system is reachable through Nginx.
Create the Docker Compose file

Docker Compose needs to know about every container in your system: what image to use, which ports to expose, and how the services depend on each other.

  • Create a file called docker-compose.yml in your project root (the same level as the app/ and nginx/ folders).
  • Paste the following configuration into the file:
services:
  app1:
    build: ./app
    environment:
      - INSTANCE_ID=instance-1
      - REDIS_HOST=redis
    depends_on:
      - redis

  app2:
    build: ./app
    environment:
      - INSTANCE_ID=instance-2
      - REDIS_HOST=redis
    depends_on:
      - redis

  app3:
    build: ./app
    environment:
      - INSTANCE_ID=instance-3
      - REDIS_HOST=redis
    depends_on:
      - redis

  nginx:
    image: nginx:1.31.1-alpine
    ports:
      - "8080:80"
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
    depends_on:
      - app1
      - app2
      - app3

  redis:
    image: redis:8.8.0-alpine
    ports:
      - "6379:6379"

What does this configuration do?

This file defines 5 services that Docker Compose will manage together:

  • app1, app2, app3 each build from the same Dockerfile in ./app, but receive a unique INSTANCE_ID environment variable so you can tell them apart.
  • nginx uses the official nginx:1.31.1-alpine image, maps port 8080 on your machine to port 80 inside the container, and mounts your config file as read-only.
  • redis uses the official redis:8.8.0-alpine image and exposes port 6379 for debugging access.

All services share a default Docker Compose network, which means they can find each other by service name (e.g. app1 resolves to app1's container IP automatically).

Why three separate services instead of replicas?

Docker Compose supports a deploy: replicas: 3 option, but all replicas share a single service name. That makes it impossible to list them individually in Nginx's upstream block.

By defining app1, app2, app3 explicitly, each gets its own hostname and its own INSTANCE_ID. This lets you observe exactly which backend handled each request.

  • Save docker-compose.yml.

✔️ Awesome, I've got everything!

Great. Double check you've saved the file in your project root, at the same level as the app/ and nginx/ folders.

ⓧ I'd like to double check the full code

services:
  app1:
    build: ./app
    environment:
      - INSTANCE_ID=instance-1
      - REDIS_HOST=redis
    depends_on:
      - redis

  app2:
    build: ./app
    environment:
      - INSTANCE_ID=instance-2
      - REDIS_HOST=redis
    depends_on:
      - redis

  app3:
    build: ./app
    environment:
      - INSTANCE_ID=instance-3
      - REDIS_HOST=redis
    depends_on:
      - redis

  nginx:
    image: nginx:1.31.1-alpine
    ports:
      - "8080:80"
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
    depends_on:
      - app1
      - app2
      - app3

  redis:
    image: redis:8.8.0-alpine
    ports:
      - "6379:6379"
Build and launch the stack

With the Compose file in place, a single command builds all three app images and starts every container in the background.

  • Build and start all 5 containers by running this command:
docker compose up --build -d

What does this command do?

  • docker compose up creates and starts all services defined in your Compose file.
  • --build forces Docker to rebuild the images from your Dockerfile (instead of using a cached version).
  • -d runs everything in detached mode so your terminal stays free.

This takes a minute or two on the first run as Docker downloads base images and installs Python packages.

  • Verify all 5 containers are running by running:
docker compose ps

You should see 5 rows in the output, each with a STATUS showing "Up".

Containers not showing as Up?

  • If a container exited immediately, check its logs with docker compose logs app1 (replace app1 with the failing service name). A common cause is a typo in requirements.txt that breaks the pip install.
  • If you see a port conflict error, another process is already using port 8080 or 6379. Stop that process or change the host port in docker-compose.yml.
  • Make sure Docker Desktop is running. help me troubleshoot my Docker containers not starting.
Check initial connectivity

The entire system is now running. Nginx is listening on port 8080 on your machine and forwarding requests to the three app instances.

  • Hit the health endpoint through Nginx by running:
curl http://localhost:8080/

You should see a JSON response like this:

{"status": "healthy", "instance": "instance-1"}

The instance field tells you which backend handled this particular request. It could show instance-1, instance-2, or instance-3.

Getting "connection refused"?

  • Run docker compose ps and confirm the nginx container is Up and shows 0.0.0.0:8080->80/tcp in its PORTS column.
  • If the nginx container restarted, check its logs with docker compose logs nginx. A typo in nginx.conf (like a missing semicolon) prevents Nginx from starting.
  • help me debug why curl to localhost:8080 is being refused.

Your distributed system is live. Five containers are working together: three app instances, one load balancer, and one cache. Next, you will send traffic through the system and watch Nginx distribute requests across all three backends in real time.

Test Load Balancing in Action

Your 5-container stack is running and responding to requests. Now it is time to see load balancing in action.

In this step, you will send repeated requests through Nginx and watch it distribute traffic evenly across your three API instances. You will also simulate a failure to see how the load balancer handles it automatically.

In this step, get ready to:
  • Observe round-robin distribution across all three instances.
  • Run a batch test confirming even traffic split.
  • Test automatic failover when an instance goes down.
Observe round-robin distribution

Round-robin is the simplest load balancing algorithm. It sends each new request to the next server in the list, cycling back to the first after reaching the end.

  • Send a few individual requests to see the instance field change by running this command three times:
curl -s http://localhost:8080/ | python3 -m json.tool

Each time you run this, the instance field in the response should cycle through instance-1, instance-2, and instance-3.

  • Run a batch test of 9 requests to confirm the even distribution:
for i in $(seq 1 9); do curl -s http://localhost:8080/ | python3 -m json.tool | grep instance; done

You should see each instance ID appear exactly 3 times in the output. That is round-robin in action.

Why exactly 3 times each?

Nginx cycles through the upstream server list in order. With 9 requests and 3 servers, each server gets exactly 9 / 3 = 3 requests. This predictable distribution is one reason round-robin is the default algorithm.

Seeing the same instance every time?

Make sure all three app containers are running. Check with docker compose ps and confirm app1, app2, and app3 all show status "Up".

If a container exited, restart the full stack with docker compose up -d.

help me debug why my curl requests are all going to the same instance

Understand what you are observing

What is happening behind the scenes?

Nginx distributes requests sequentially across the servers listed in the upstream block of your nginx.conf. Each app instance is stateless and identical. The only difference is the INSTANCE_ID environment variable, which lets you see which one handled the request.

In a production system, these instances could be separate machines, VMs, or Kubernetes pods. The principle is identical: a reverse proxy sits in front and distributes incoming traffic so no single instance becomes a bottleneck.

Test failover behavior

A good load balancer does not just distribute traffic. It also detects when a backend is unavailable and stops sending requests to it.

  • Simulate a failure by stopping one instance:
docker compose stop app2

This shuts down app2 while keeping the other four containers running.

  • Run the batch test again to see how Nginx responds:
for i in $(seq 1 9); do curl -s http://localhost:8080/ | python3 -m json.tool | grep instance; done

You should see only instance-1 and instance-3 in the output. Nginx automatically detected that app2 is unreachable and stopped routing traffic to it.

How does Nginx detect the failure?

By default, Nginx marks an upstream server as unavailable after 1 failed connection attempt. It then skips that server for a timeout period (default 10 seconds) before trying again. This happens transparently to the client.

  • Bring the instance back online:
docker compose start app2
  • Confirm all three instances are receiving traffic again by running the batch test one more time:
for i in $(seq 1 9); do curl -s http://localhost:8080/ | python3 -m json.tool | grep instance; done

All three instance IDs should appear in the output again, each appearing 3 times.

Still only seeing two instances after restart?

Nginx may still consider app2 unavailable if it has not retried yet. Wait a few seconds and run the loop again. The default recovery timeout is 10 seconds.

You can also verify the container is running with docker compose ps and confirm app2 shows status "Up".

help me troubleshoot why my restarted container is not receiving traffic

You have now seen load balancing and automatic failover working in real time. Next up, you will add a Redis caching layer and measure how dramatically it improves response times.

Observe Caching Performance

Your load balancer is distributing traffic evenly and handling failover automatically. Now it's time to tackle the second pillar of scalable systems: caching.

Every request to your product catalog currently hits the "database" (simulated with a 500ms delay). In production, this kind of latency adds up fast under heavy traffic. Redis sits between your app instances and the database as a shared cache, letting any instance serve data in milliseconds instead of waiting for a full database round-trip.

In this step, get ready to:
  • Observe a cache miss with ~500ms response time.
  • Observe a cache hit with sub-5ms response time.
  • Confirm that Redis shares cached data across all three instances.
Trigger a cache miss (first request)

When Redis doesn't have the requested data, the app falls back to the database. Your code simulates this with a 500ms time.sleep() delay, making the performance difference between a cache miss and hit dramatically visible.

  • Clear any existing cached data by running this command:
curl -X DELETE http://localhost:8080/cache

You should see a response like {"message":"Cache cleared","instance":"instance-1"} confirming Redis has been flushed.

  • Now time a request to the products endpoint by running:
time curl -s http://localhost:8080/products | python3 -m json.tool | grep source

You should see "source": "database" in the output, and the real time reported by time should be around 0.5 seconds. That's the simulated database delay in action.

What does this command do?

  • time measures how long the entire command takes to complete.
  • curl -s sends the HTTP request silently (no progress bar).
  • python3 -m json.tool pretty-prints the JSON response so it's readable.
  • grep source filters the output to only show the line containing the data source (cache or database).

Seeing "Connection refused"?

Make sure all 5 containers are still running. Run docker compose ps and confirm each service shows a status of "Up". If any container has exited, restart the stack with docker compose up -d.

Help me debug why my containers aren't running

Trigger a cache hit (second request)

That first request stored the product data in Redis with a 30-second TTL. Any subsequent request for the same data will find it already waiting in the cache.

  • Run the exact same command again:
time curl -s http://localhost:8080/products | python3 -m json.tool | grep source

This time you should see "source": "cache" and the real time should drop to roughly 1-5ms. That's a 100x improvement.

  • Run the command a few more times and pay attention to the instance field in the full response. Confirm that different instances return the cached data by running:
for i in $(seq 1 6); do curl -s http://localhost:8080/products | python3 -m json.tool | grep -E "source|instance"; echo "---"; done

You should see "source": "cache" on every request, but the instance cycling between instance-1, instance-2, and instance-3. This proves that all three instances share the same Redis cache.

Why can any instance serve the cached data?

All three app instances connect to the same Redis container. When instance-1 writes product data into Redis on the first request, that data is immediately available to instance-2 and instance-3 too.

This is what makes horizontally scaled services work. Without a shared cache, each instance would have to query the database independently, wasting resources on duplicate work.

Understand the cache-aside pattern

What you just observed is the cache-aside pattern (also called lazy-loading). It's the most common caching strategy in production systems and a frequent topic in system design interviews.

How does cache-aside work?

  • The app checks Redis first. If the key exists, it returns the cached data immediately (a "cache hit").
  • If the key doesn't exist (a "cache miss"), the app queries the database, stores the result in Redis with a TTL, then returns the data.
  • After the TTL expires (30 seconds in your project), the key is automatically deleted by Redis. The next request triggers a fresh database lookup.
  • Test the TTL expiration by waiting 30 seconds after your last request, then running:
time curl -s http://localhost:8080/products | python3 -m json.tool | grep source

You should see "source": "database" again with the ~500ms delay. The cache expired, and the app populated it fresh.

  • Run it one more time immediately after:
time curl -s http://localhost:8080/products | python3 -m json.tool | grep source

Back to "source": "cache" with sub-5ms timing. The cycle continues: miss, populate, hit, hit, hit, expire, miss again.

Why use a TTL instead of caching forever?

TTL ensures data freshness. If a product's price changes in the database, the stale cached version will expire within 30 seconds and the next request will fetch the updated data.

In production, you balance TTL length against your freshness requirements. A 30-second TTL works for a product catalog. A stock ticker might need a 1-second TTL. A user profile might tolerate 5 minutes.

Still seeing "source": "cache" after 30 seconds?

Make sure you waited a full 30 seconds from the last request that populated the cache (not from your first command). You can force a fresh start by running curl -X DELETE http://localhost:8080/cache to clear Redis manually.

Help me understand why my cache TTL doesn't seem to be working

Secret mission

Experiment with Load Balancing Strategies

Round-robin works when all servers are equal, but real systems rarely stay balanced. What happens when one instance is overloaded, or when your servers have different hardware specs? In this mission, you'll configure Nginx to use least-connections and weighted load balancing, then observe how traffic distribution changes under real conditions.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally with Docker, so there are no ongoing costs.

Resources you used:

  • 5 Docker containers (app1, app2, app3, nginx, redis).
  • Docker images built during the project (Python app image, pulled Nginx and Redis Alpine images).
  • Project folder on your Desktop with source files.

Keep everything running

No action needed. Choose this if you are still actively building or experimenting with the load balancing configurations.

  • Your 5 containers will continue running in the background.
  • Docker Desktop may use some memory while containers are active.
  • You can revisit the API at http://localhost:8080/ anytime.

Pause - I'll come back to this later

Shut down running containers to free up memory, but keep your project files and Docker images for a quick restart later.

  • Stop all containers by running this command from your project folder:
docker compose stop
  • When you want to resume, run:
docker compose start

Your containers, images, and project files all remain intact for next time.

Delete - I don't want to use this again

Remove all project resources and start fresh. This removes containers, networks, and images created by this project.

  • Stop and remove all containers, networks, and built images by running:
docker compose down --rmi all
  • Move up one directory, then delete the project folder:

macOS

cd ..
rm -rf load-balanced-api

Windows

cd ..
rmdir /s /q load-balanced-api
  • Confirm the folder is gone by running:
ls

You should no longer see the project folder in the output.

That's a Wrap!

That's a Wrap!

Nice work! You just built a production-style distributed system that demonstrates the two most common system design interview topics: load balancing and caching.

You've learned how to:

  • Configure Nginx as a reverse proxy that distributes traffic across 3 identical FastAPI instances using round-robin load balancing, with automatic failover when an instance goes down.
  • Implement the cache-aside pattern with Redis as a shared cache layer, reducing response times from ~500ms to sub-5ms.
  • Orchestrate a 5-container system with Docker Compose where services discover each other by name over the default bridge network.
  • Secret Mission: Experimented with least-connections and weighted load balancing strategies and compared their behavior under uneven load.

Ready to quiz yourself?