Chaos Experiments on a Dockerized Web App

Break and harden a Flask + Redis app using chaos engineering methods.

Introduction

30 Second Summary

Every app you rely on depends on services running behind the scenes. When one of those services fails, users see errors, lost data, or a completely broken experience.

In this project, you will practice chaos engineering by deploying a Flask web app backed by Redis using Docker Compose. You will intentionally break the app by killing and freezing containers, then engineer resilience so it keeps serving users during outages.

What You'll Build

You will stop a running container mid-experiment and watch your app respond with a friendly fallback message instead of crashing.

By the end of this project, you'll have:

  • A two-service stack you spin up with one command. Your web app counts page visits through a cache and serves responses at http://localhost:8000.
  • Hands-on chaos experiments following the steady-state hypothesis method that Netflix and Google SRE teams use to test production systems.
  • A before-and-after demo where the same failure goes from crashing the app to triggering a graceful degradation response, with health checks and restart policies enabling automatic recovery.
  • Secret Mission: Inject artificial latency into your Redis connection to simulate a slow network and monitor container resource usage with docker stats.

Do I need any prior experience?

No prior experience with Docker or chaos engineering is needed. This project walks you through installing Docker Desktop and setting up every file from scratch.

Before We Start

Before you start spinning up containers and breaking things, let's lock in what you're about to build and why it matters.

Set Up Docker Desktop and Project Files

Before you can run chaos experiments, you need something to break. This step gives you a working baseline: a Flask web app backed by Redis, all running inside Docker containers.

You will install Docker Desktop, create the app files, and launch everything with Docker Compose. By the end of this step, you will have a running app you can visit in your browser.

In this step, get ready to:
  • Install Docker Desktop with the WSL 2 backend.
  • Create a Flask web app backed by Redis and containerize it with Docker.
  • Build and launch the multi-container stack with Docker Compose.
Install Docker Desktop

Docker Desktop is the application that lets you build and run containers on your local machine. It includes everything you need: the Docker engine, the docker compose command, and a graphical dashboard.

  • Open your browser and go to the Docker Desktop for Windows install page.
  • Click the download button at the top of the page to download Docker Desktop Installer.exe.
  • Double-click Docker Desktop Installer.exe to run the installer.
  • When prompted, choose per-user installation mode. This does not require administrator privileges.
  • Make sure the WSL 2 backend is enabled. The installer may prompt you to turn this on.
  • Follow the remaining prompts to complete the installation. Restart your computer if prompted.

What is WSL 2?

WSL 2 (Windows Subsystem for Linux 2) is a feature built into Windows that runs a real Linux kernel. Docker Desktop uses it to run Linux containers on your Windows machine without needing a full virtual machine.

Once the install finishes and Docker Desktop is running, open PowerShell so you can verify that Docker is ready.

  • Press the Windows key to open your search bar.
  • Type PowerShell and press Enter to open it.
  • Check that Docker is installed by running this command:
docker --version

✔️ I see a version number

Docker is installed and ready. You should see something like Docker version 27.x.x or similar. The exact version number does not matter for this project.

ⓧ Command not found or Docker Desktop is not running

Docker Desktop may not be running yet, or the installation did not complete.

  • Press the Windows key, type Docker Desktop, and press Enter to launch it.
  • Wait for Docker Desktop to finish starting. The whale icon in your system tray will stop animating when it is ready.
  • Close and reopen PowerShell, then run docker --version again.

If you still see an error, revisit the Docker Desktop install page and re-run the installer.

Create the project files

Your project needs four files: the Flask app, its dependencies list, a Dockerfile to build the app image, and a Compose file to orchestrate both services.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir chaos-app
cd chaos-app
  • Confirm the folder was created by running:
ls

The folder is empty for now. That is expected.

Now open the chaos-app folder in your code editor so you can create files inside it.

  • Create a new file called app.py inside your chaos-app folder. This is the Flask web app that counts page visits using Redis. Paste this code into it:
import os
import redis
from flask import Flask, jsonify

# Create the Flask application
app = Flask(__name__)

# Read Redis connection details from environment variables
REDIS_HOST = os.getenv("REDIS_HOST", "redis")
REDIS_PORT = int(os.getenv("REDIS_PORT", "6379"))

def get_redis_client():
    # Create a new Redis connection
    return redis.Redis(
        host=REDIS_HOST,
        port=REDIS_PORT,
    )

@app.route("/")
def hello():
    # Connect to Redis and increment the hit counter
    cache = get_redis_client()
    count = cache.incr("hits")
    # Return a JSON response with the visit count
    return jsonify({
        "message": f"Hello from Docker! I have been seen {count} time(s).",
        "redis_status": "connected",
    })

What does this code do?

This Flask app has one route at /. Every time someone visits it, the app connects to Redis and increments a counter called hits. It returns a JSON response showing how many times the page has been visited.

The get_redis_client() function creates a fresh Redis connection each time. The host and port come from environment variables, which Docker Compose will set for you.

  • Save app.py.

The app needs Flask and the Redis Python client. These go in a separate file so Docker can install them during the image build.

  • Create a new file called requirements.txt in the same chaos-app folder and paste these two dependencies:
flask==3.1.3
redis==8.1.0
  • Save requirements.txt. Now create a file called Dockerfile (no file extension) in the same folder and paste this build configuration:
# Use a lightweight Python image
FROM python:3.12-slim
# Set the working directory inside the container
WORKDIR /app
# Copy and install dependencies first (for Docker layer caching)
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Copy the rest of the application code
COPY . .
# Document the port Flask runs on
EXPOSE 5000
# Start the Flask development server
CMD ["flask", "run", "--host=0.0.0.0"]

What does this Dockerfile do?

A Dockerfile is a recipe for building a container image. This one starts from a slim Python 3.12 image, copies your dependencies, installs them with pip, then copies your app code.

The CMD line tells Docker to start the Flask development server when the container launches. The --host=0.0.0.0 flag makes the server listen on all network interfaces, which is required for traffic to reach it from outside the container.

  • Save Dockerfile.
  • Back in PowerShell, confirm your files exist by running:
ls

You should see app.py, requirements.txt, and Dockerfile listed in the output.

One more file to go. The Compose file defines how Docker should run your app and Redis together as a multi-container stack.

  • Create a new file called compose.yaml in your chaos-app folder and paste this configuration:
services:
  # The Flask web app, built from the local Dockerfile
  web:
    build: .
    ports:
      - "8000:5000"
    environment:
      - REDIS_HOST=redis
      - REDIS_PORT=6379

  # Redis cache used by the web app to store hit counts
  redis:
    image: redis:alpine

What does this Compose file do?

This file defines two services. The web service builds your Flask app from the Dockerfile in the current directory. The ports setting maps port 8000 on your machine to port 5000 inside the container (where Flask listens).

The redis service pulls a pre-built Redis image from Docker Hub. Docker Compose automatically puts both services on the same network, so the web app can reach Redis using the hostname redis.

  • Save compose.yaml.

✔️ Awesome, I've got everything!

Make sure you have saved all four files. Time to build and launch the stack.

ⓧ I'd like to double check the full code

Scroll up and confirm you have all four files in your chaos-app folder, in this order:

  • app.py with the Flask app and hello() route.
  • requirements.txt with flask==3.1.3 and redis==8.1.0.
  • Dockerfile with the python:3.12-slim base image and the Flask run command.
  • compose.yaml with web and redis services.
Build and launch the stack

With all four files saved, you can now build the Docker image for your Flask app and start both containers.

  • Build and launch both containers in the background by running this command:
docker compose up --build -d

What does this command do?

  • docker compose up reads compose.yaml and starts all the services defined in it.
  • --build tells Docker to build the image from your Dockerfile before starting the containers.
  • -d runs the containers in detached mode (in the background), so you get your terminal back.

The first run takes a minute or two because Docker needs to download the Python and Redis base images. You will see build progress in your terminal as Docker executes each step from your Dockerfile.

Seeing an error about Docker daemon?

If you see an error about connecting to the Docker daemon, Docker Desktop is not running.

  • Press the Windows key, type Docker Desktop, and press Enter to launch it.
  • Wait for the whale icon in your system tray to stop animating, then try the command again.

Why won't my Docker containers start?

Before you open the app, what do you think you will see at http://localhost:8000? Think about what your hello() function returns.

  • Open your browser and go to http://localhost:8000.

You should see a JSON response that says Hello from Docker! I have been seen 1 time(s).

  • Refresh the page a few times. The counter should increment with each refresh: 2, 3, 4, and so on.

Page not loading?

  • Make sure you are using http:// and not https:// in the URL.
  • Check that the port is 8000, not 5000. Your compose.yaml maps external port 8000 to internal port 5000.
  • Run docker compose ps in PowerShell to check that both containers are running. If a container exited, run docker compose logs web to see the error.

Why can't I reach my Docker container in the browser?

Your app is up and running. Next up, you will measure exactly how this app behaves when everything is healthy, so you have a baseline to compare against when you start breaking things.

Establish Your Steady State

Your Flask + Redis app is live and every page refresh bumps the counter. That is a working app, but "it works right now" tells you nothing about what happens when things go wrong.

Chaos engineering starts by defining what "normal" looks like. Without a documented baseline, you have nothing to compare against when things break. In this step, you will measure your app's steady state and write the hypothesis you will test in the next step.

In this step, get ready to:
  • Confirm both containers are running and check the Flask startup logs.
  • Record your steady-state baseline by hitting the app endpoint multiple times.
  • Write your chaos hypothesis for the next experiment.
Confirm your containers are running

Before any chaos experiment, you need to verify that your starting point is clean. Both the web and Redis containers need to be up and healthy.

  • Confirm both containers are running by running this command in PowerShell:
docker compose ps

You should see two rows: one for the web service and one for the redis service. Both should show a status of Up.

One or both containers not showing Up?

If either container is missing or stopped, restart the stack by running docker compose up --build -d in your project folder. Make sure Docker Desktop is running before you try again.

help me debug why my Docker Compose containers are not starting

  • Check the Flask app's startup logs by running:
docker compose logs web

What do the logs tell you?

The output shows Flask starting up and listening on 0.0.0.0:5000. This confirms the web server inside the container is ready to accept requests.

Port 5000 is the container's internal port. Docker Compose maps it to port 8000 on your machine, which is why you access the app at http://localhost:8000.

Record your steady-state baseline

In chaos engineering, the steady-state hypothesis defines how your system behaves when everything is healthy. You measure it before injecting any failure so you have a clear "before" picture.

Before you run the next command, think about what you expect the response to contain. What fields should be in the JSON? What value should the counter show?

  • Hit the app endpoint by running this command in PowerShell:
curl http://localhost:8000
  • Run this same command two more times (three times total) and observe how the response changes each time.

Each response returns JSON with a "message" that includes an incrementing counter and a "redis_status" of "connected". The counter might start higher than 1 if you refreshed the browser earlier.

What is your steady-state baseline?

Here is what "normal" looks like for your app. Record these four things:

  • Every request returns a successful response.
  • The hit counter increments by 1 on each request.
  • The redis_status field shows "connected".
  • Responses come back in under a second.

Any change to these behaviors during a chaos experiment means something is broken. This baseline is what you will compare against.

Seeing unexpected output instead of JSON?

In Windows PowerShell 5.1, curl is an alias for a different command. If you see a large block of output with headers and status codes instead of clean JSON, use curl.exe instead:

curl.exe http://localhost:8000

help me get curl working in PowerShell to hit my local Docker app

Write your chaos hypothesis

Every chaos experiment starts with a hypothesis. Just like a science experiment, you predict what should happen, then test whether it actually does.

Your hypothesis for this project is:

Your chaos hypothesis

"If Redis becomes unavailable, the web app should still respond to users."

This sets a clear pass/fail bar. If the app keeps responding when Redis is down, the hypothesis holds. If the app crashes, it fails.

Why write a hypothesis first?

The hypothesis forces you to think about failure modes before you inject them. Without one, you are just randomly breaking things. With one, you are running a structured experiment that produces a clear conclusion.

This is the same methodology used by SRE teams at companies running chaos engineering in production. Define normal, hypothesize, inject failure, observe, then fix.

Your steady state is documented and your hypothesis is written. Next up, you will inject your first real failure by killing the Redis container and finding out whether your app survives.

Run Chaos Experiment 1 - Kill a Dependency

You have a documented steady state and a written hypothesis. Your app works, the counter increments, and every request succeeds.

But what happens when Redis goes down? Most apps crash hard when a dependency disappears. In this step, you will inject a real failure to discover whether your hypothesis holds.

In this step, get ready to:
  • Find and stop the Redis container to simulate a dependency outage.
  • Test your chaos hypothesis by hitting the app while Redis is down.
  • Bring Redis back and verify the app recovers.
Find and stop the Redis container

Docker Compose names containers using a pattern: <project-folder>-<service>-<number>. The project folder name defaults to the directory your compose.yaml lives in. You need the exact Redis container name so you can target it with a stop command.

  • List your running containers by running this command in PowerShell:
docker compose ps

You should see two containers listed: one for web and one for redis. Both should show a status of Up.

  • Find the name of the Redis container in the NAME column. It follows the pattern <project-folder>-redis-1. Record it here: your Redis container name, e.g. chaos-app-redis-1

Time to simulate an outage. You are going to pull the plug on Redis and see what your app does without it.

  • Stop the Redis container by running this command:
docker stop [[REDIS_CONTAINER="your Redis container name, e.g. chaos-app-redis-1"]]

Docker echoes back the container name when the stop succeeds.

  • Confirm Redis is down by running:
docker compose ps

The Redis container should no longer appear in the list, or show an Exited status. The web container is still running.

Container not stopping?

Make sure you copied the exact container name from the NAME column of docker compose ps. The name is case-sensitive and includes the project folder name.

Help me figure out why docker stop is not working.

Test your chaos hypothesis

Redis is down. Your Flask web app is still running. Your hypothesis from the last step says: "If Redis becomes unavailable, the web app should still respond to users."

Before you run the next command, think about what you expect to happen. Do you think the app will still respond with a message, or will something go wrong?

  • Hit the app by running this command:
curl http://localhost:8000

The app does not respond gracefully. Instead of a friendly message, you see a 500 Internal Server Error. The app is completely down even though only Redis is gone.

What just happened?

Your app.py calls cache.incr("hits") on every request. When Redis is not reachable, that call throws a redis.exceptions.ConnectionError. The app has zero error handling, so Flask catches the unhandled exception and returns a generic 500 error page.

Your hypothesis failed. One missing dependency took down the entire application. This is exactly the kind of fragility that chaos engineering is designed to expose.

Not seeing an error?

If the app responds normally, Redis might not have actually stopped. Run docker compose ps to confirm the Redis container is no longer running.

Help me troubleshoot why my app still responds after I stopped Redis.

Bring Redis back and verify recovery

You have proven that the app crashes without Redis. The experiment is complete. Now bring Redis back so the app is functional again before the next step.

  • Restart the Redis container by running this command:
docker start [[REDIS_CONTAINER="your Redis container name, e.g. chaos-app-redis-1"]]
  • Confirm Redis is back by running:
docker compose ps

Both the web and redis containers should show a status of Up again.

Before you test the app, think about the hit counter. It was at several hits before you stopped Redis. What do you think it will show now?

  • Test the app by running:
curl http://localhost:8000

The app responds again. But the hit counter shows 1, not the number you had before.

Why did the counter reset?

Redis stores data in memory by default. When the container stopped, all the data in memory was lost. The compose.yaml does not define a persistent volume for Redis, so there is no data on disk to recover from.

This is another chaos engineering insight. Data loss during restarts is a real-world side effect that stateless services handle better than stateful ones.

  • Refresh the app a few more times by running curl http://localhost:8000 to confirm the counter increments normally from 1.

App still returning an error after restarting Redis?

Give Redis a few seconds to fully start before retrying curl. If the error persists, check that docker compose ps shows both containers as Up.

Help me figure out why my app is not recovering after restarting Redis.

You have proven that your app is fragile. A single missing dependency takes down the entire service. In the next step, you will fix this by adding error handling and fallback responses so the app degrades gracefully instead of crashing.

Build Resilience and Re-test

In the last step you proved something important: your app crashes completely when Redis goes down. A single missing dependency takes down the entire service, with no helpful message for the user.

Now you fix it. In this step, you will add error handling, connection timeouts, and a fallback response so the app degrades gracefully instead of dying. Then you will re-run the exact same chaos experiment to prove the fix works.

In this step, get ready to:
  • Add connection timeouts and try/except error handling to your Flask app.
  • Add a health check endpoint that reports Redis reachability.
  • Re-run the chaos experiment to prove the app now survives Redis going down.
Add timeouts and error handling to your app

The crash you saw in Step 3 happened for two reasons. First, the app has no timeout on its Redis connection, so a frozen or unresponsive Redis could make it hang forever. Second, there is no error handling, so any Redis failure turns into an unhandled exception.

  • In your code editor, open app.py.
  • Select all the code in the file and replace it with this updated top section, which adds timeouts to the Redis client:
import os
import time
import redis
from flask import Flask, jsonify

app = Flask(__name__)

# Read config from environment variables (set in compose.yaml)
REDIS_HOST = os.getenv("REDIS_HOST", "redis")
REDIS_PORT = int(os.getenv("REDIS_PORT", "6379"))
CHAOS_LATENCY_MS = int(os.getenv("CHAOS_LATENCY_MS", "0"))


def get_redis_client():
    # Timeouts prevent the app from hanging when Redis is frozen or unreachable
    return redis.Redis(
        host=REDIS_HOST,
        port=REDIS_PORT,
        socket_timeout=3,
        socket_connect_timeout=3,
    )

What does this code do?

socket_timeout=3 tells the Redis client to give up after 3 seconds if a request to Redis gets no response. socket_connect_timeout=3 does the same for the initial connection attempt.

CHAOS_LATENCY_MS is a hook for injecting artificial latency later. It defaults to 0, so it has no effect right now.

Timeouts alone are not enough. The app still needs to handle the error that fires when a timeout is hit. Next you will replace the hello() route with a version that catches Redis errors and returns a friendly fallback.

  • Directly below the get_redis_client() function you just pasted, add this new hello() route:
@app.route("/")
def hello():
    # Inject artificial latency if CHAOS_LATENCY_MS is set (used in Secret Mission)
    if CHAOS_LATENCY_MS > 0:
        time.sleep(CHAOS_LATENCY_MS / 1000)

    try:
        cache = get_redis_client()
        count = cache.incr("hits")
        return jsonify({
            "message": f"Hello from Docker! I have been seen {count} time(s).",
            "redis_status": "connected",
        })
    except (redis.exceptions.ConnectionError, redis.exceptions.TimeoutError):
        # Return a friendly response instead of crashing
        return jsonify({
            "message": "Hello from Docker! (Redis is unavailable, running in degraded mode)",
            "redis_status": "disconnected",
        }), 200

What does this code do?

  • The try block attempts to connect to Redis and increment the hit counter, exactly like before.
  • The except block catches ConnectionError and TimeoutError from Redis. Instead of crashing, it returns a JSON response telling the user Redis is down.
  • The fallback still returns HTTP 200 so the user sees a response, not an error page. This is graceful degradation in action.
  • Save app.py.
  • Rebuild and restart the stack by running this command in your PowerShell terminal:
docker compose up --build -d
  • Verify the app still works normally by running:
curl http://localhost:8000

You should see the familiar JSON response with "redis_status": "connected" and an incrementing counter. The app works normally when Redis is healthy.

Seeing a syntax error or the container won't start?

  • Run docker compose logs web to see the Flask error output. Python syntax errors (missing colons, mismatched parentheses) will show up here.
  • Make sure your except line catches both redis.exceptions.ConnectionError and redis.exceptions.TimeoutError inside parentheses, separated by a comma.
  • Check that the hello() function sits directly below get_redis_client() with no duplicate route decorator.

Help me debug why my Flask app won't start after adding try/except error handling.

Add a health check endpoint

A health check endpoint gives you a quick, machine-readable way to ask the app "are your dependencies working?" instead of guessing from user-facing responses.

  • In app.py, add this /health route at the very bottom of the file, below the hello() function:
@app.route("/health")
def health():
    # Check if Redis is reachable and report status
    try:
        cache = get_redis_client()
        cache.ping()
        return jsonify({"status": "healthy", "redis": "connected"}), 200
    except (redis.exceptions.ConnectionError, redis.exceptions.TimeoutError):
        return jsonify({"status": "degraded", "redis": "disconnected"}), 503

What does this endpoint do?

The /health endpoint pings Redis and reports its status. If Redis is reachable it returns 200 (healthy). If not, it returns 503 (degraded). This is separate from the main route so monitoring tools can check it without affecting the hit counter.

✔️ Awesome, I've got everything!

Make sure you have saved app.py before moving on.

ⓧ I'd like to double check the full code

Your app.py is longer than a single view now. Scroll through and confirm these three sections appear in order:

  • Top section: imports (os, time, redis, Flask), three environment variable reads, and the get_redis_client() function with socket_timeout=3 and socket_connect_timeout=3.
  • hello() route: the @app.route("/") handler with the latency injection check, a try/except block around Redis calls, and a fallback JSON response.
  • health() route: the @app.route("/health") handler that pings Redis and returns a status JSON.

You also need to tell Docker Compose about the new environment variable your app reads. This is a small one-line addition.

  • Open compose.yaml and find the environment section under the web service.
  • Find this line:
      - REDIS_PORT=6379
  • Add a new line directly below it so the environment section now looks like this:
      - REDIS_PORT=6379
      - CHAOS_LATENCY_MS=0
  • Save compose.yaml.
  • Rebuild the stack to pick up all changes by running:
docker compose up --build -d
  • Test the new health endpoint by running:
curl http://localhost:8000/health

You should see {"redis": "connected", "status": "healthy"}. Both the main route and the health endpoint are working with Redis healthy.

Getting a 404 on /health?

  • Make sure the health() function is at the bottom of app.py, not nested inside another function.
  • Confirm you ran docker compose up --build -d after saving. Without the --build flag, Docker reuses the old image and your changes are not included.

Help me figure out why my /health endpoint returns a 404.

Re-run the chaos experiment

The real test is whether the same failure that crashed your app in Step 3 is now survivable. You will run the exact same experiment: stop Redis, then hit the app.

  • First, confirm both containers are running by running:
docker compose ps
  • Stop the Redis container to simulate the same outage from Step 3 by running:
docker stop [[REDIS_CONTAINER="your Redis container name"]]

Before you run the next command, think: what do you expect to happen when you hit the app now? Will it crash like before, or will the error handling you just added catch the failure?

  • Hit the app with Redis down by running:
curl http://localhost:8000

Instead of crashing, the app returns a JSON response with "redis_status": "disconnected" and a friendly fallback message. The hypothesis from Step 2 now holds: the web app responds to users even when Redis is unavailable.

  • Check the health endpoint with Redis still down by running:
curl http://localhost:8000/health

You should see "redis": "disconnected" and "status": "degraded". The health endpoint tells you exactly which dependency is down.

  • Bring Redis back and confirm full recovery by running:
docker start [[REDIS_CONTAINER="your Redis container name"]]
  • Verify the app is fully recovered by running:
curl http://localhost:8000

The response is back to "redis_status": "connected" with the hit counter running again. The counter resets to 1 because Redis lost its data when it was stopped.

Why does the counter reset?

Redis stores data in memory by default. When the container stops, that data disappears. This project intentionally does not add persistent storage so you can see data loss as a real side effect of an outage. In production, you would configure Redis persistence or use a managed Redis service.

Your app now survives its dependency disappearing. But killing a service is the easy test. In the next step, you will try something trickier: freezing Redis so it looks alive but is completely unresponsive.

Run Chaos Experiment 2 - Freeze a Service

In Step 4, you proved your app can survive Redis being stopped. Instead of crashing, it returns a graceful fallback response.

Real-world outages are often subtler than a clean shutdown. A database freezes under load. A cache stops responding but never actually dies. docker pause simulates this by suspending a container's processes without stopping it. In this step, you'll test your timeouts against a frozen Redis, then add health checks and restart policies so Docker recovers crashed containers automatically.

In this step, get ready to:
  • Freeze Redis with docker pause and confirm the app responds within 3 seconds instead of hanging.
  • Add health checks, restart policies, and startup dependencies to compose.yaml.
  • Hard-kill Redis and watch Docker restart it without any manual intervention.
Freeze Redis and test your timeouts

Killing a service is dramatic, but docker pause is sneakier. It suspends a container's processes so the container still appears in docker compose ps with a Paused status, but nothing gets in or out.

  • Confirm both containers are running by running this command:
docker compose ps

You should see both the web and redis services listed with a status of Up.

  • Freeze your Redis container by running this command:
docker pause [[REDIS_CONTAINER="your-redis-container-name"]]

How do I find my container name?

The container name comes from the docker compose ps output you just ran. It follows the pattern <project-folder>-redis-1, where <project-folder> is the name of the directory your project lives in.

Before you hit the app, what do you think will happen? Will the request respond immediately, wait a few seconds, or hang forever?

  • Hit the app with this command:
curl http://localhost:8000

The request hangs for about 3 seconds, then returns your degraded-mode response with "redis_status": "disconnected". That 3-second wait is your socket_timeout=3 from Step 4 doing its job.

Why is a frozen service harder than a dead one?

When you docker stop a container, the TCP connection is refused immediately. Your app gets a fast error and can handle it right away.

A paused container is different. The TCP connection is accepted by the OS but never processed. Without a timeout, your app would wait forever for a response that will never come. That is why socket_timeout matters. It turns an infinite hang into a 3-second wait.

App responded instantly instead of waiting 3 seconds?

If the response came back immediately, docker pause may not have taken effect yet. Run docker compose ps and check that the Redis container shows a Paused status, then try the curl command again.

Why is my request responding instantly instead of timing out?

Restore Redis and add auto-recovery
  • Unpause Redis by running this command:
docker unpause [[REDIS_CONTAINER="your-redis-container-name"]]
  • Confirm the app recovered by running this command:
curl http://localhost:8000

The response comes back instantly with "redis_status": "connected" and the hit counter picks up where it left off. Redis is alive again.

Your app handles both dead and frozen services now. But you still had to restart Redis manually every time. Let's make Docker handle recovery automatically.

  • Switch back to your code editor and open compose.yaml.
  • Replace the entire contents of compose.yaml with this updated version:
services:
  web:
    build: .
    ports:
      - "8000:5000"
    environment:
      - REDIS_HOST=redis
      - REDIS_PORT=6379
      - CHAOS_LATENCY_MS=0
    # Wait for Redis to pass its health check before starting
    depends_on:
      redis:
        condition: service_healthy
    # Automatically restart if the container crashes
    restart: unless-stopped

  redis:
    image: redis:alpine
    # Ping Redis every 10 seconds to verify it is responsive
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 5s
      retries: 3
      start_period: 5s
    restart: unless-stopped

What does this new config do?

  • healthcheck runs redis-cli ping every 10 seconds. If Redis does not respond within 5 seconds, or fails 3 times in a row, Docker marks it as unhealthy.
  • depends_on with condition: service_healthy tells Docker Compose to wait until Redis passes its health check before starting the web service. No more connecting to a Redis that is not ready yet.
  • restart: unless-stopped on both services tells Docker to automatically restart any container that crashes. The only way to stop it is a manual docker stop.
  • Save compose.yaml.

✔️ Awesome, I've got everything!

Great. Double check you have saved compose.yaml before moving on.

ⓧ I'd like to double check the full code

Here is the complete compose.yaml you should have at this point:

services:
  web:
    build: .
    ports:
      - "8000:5000"
    environment:
      - REDIS_HOST=redis
      - REDIS_PORT=6379
      - CHAOS_LATENCY_MS=0
    depends_on:
      redis:
        condition: service_healthy
    restart: unless-stopped

  redis:
    image: redis:alpine
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 5s
      retries: 3
      start_period: 5s
    restart: unless-stopped
Rebuild and test auto-restart
  • Rebuild and restart the stack with the new config by running this command:
docker compose up --build -d
  • Wait about 15 seconds for the health check to pass, then verify both containers are healthy by running:
docker compose ps

You should see both containers listed as Up with the Redis container showing (healthy) in its status. That is your new health check confirming Redis is responsive.

Now for the real test. What do you expect to happen when you hard-kill the Redis container, now that the restart policy is in place?

  • Hard-kill the Redis container by running this command:
docker kill [[REDIS_CONTAINER="your-redis-container-name"]]

What is the difference between docker stop and docker kill?

docker stop sends a SIGTERM signal and gives the container time to shut down gracefully. docker kill sends a SIGKILL and terminates it immediately with no cleanup. This simulates a sudden crash.

  • Wait 5-10 seconds, then check whether Docker restarted Redis automatically by running:
docker compose ps

You should see the Redis container is back with a fresh Up status and a short uptime (a few seconds). Docker detected the crash and restarted it automatically thanks to restart: unless-stopped.

  • Confirm the app recovered without any manual intervention by running:
curl http://localhost:8000

The response shows "redis_status": "connected" and the hit counter is back (starting from 1 since Redis lost its in-memory data). Your app survived a hard kill and recovered completely on its own.

Redis not restarting automatically?

Make sure you saved compose.yaml and rebuilt with docker compose up --build -d after adding the restart policy. If you only saved without rebuilding, the old config is still running.

If docker compose ps shows Redis as exited instead of restarting, check that restart: unless-stopped appears under the redis service in your file. A missing or misspelled key will prevent auto-restart.

Why isn't my Redis container restarting automatically?

Your app now handles stopped, frozen, and hard-killed dependencies gracefully. Next up, the Secret Mission challenges you to simulate a slow network that degrades performance instead of breaking things outright.

Secret mission

Simulate Slow Network and Monitor Resources

Your chaos experiments have been binary. Redis is either alive or dead. In this mission, you inject artificial latency to simulate a slow network, monitor container resources with docker stats, and discover the point where slowness becomes just as bad as downtime.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely on your local machine using Docker Desktop. There are no cloud costs or ongoing charges.

Resources you used:

  • Docker containers (web and redis services) running via Docker Compose.
  • Docker images built from the project Dockerfile and pulled from Docker Hub (redis:alpine).
  • Project files (app.py, compose.yaml, Dockerfile, requirements.txt) on your Desktop.

Keep everything running

No action needed. Choose this if you are still actively experimenting with chaos experiments or want to keep practicing.

  • Your containers keep running as long as Docker Desktop is active. The app stays accessible at http://localhost:8000.
  • Docker Desktop remains installed on your machine and runs in the background.

Pause - I'll come back to this later

Stop the running containers to free up memory, but keep everything in place so you can resume later.

  • In your PowerShell terminal, make sure you are in the project folder (the folder containing compose.yaml).
  • Stop the containers without removing them by running:
docker compose stop
  • When you are ready to resume, start the containers again by running:
docker compose start

Delete - I don't want to use this again

Remove all project containers, images, and files to start fresh.

  • In your PowerShell terminal, make sure you are in the project folder (the folder containing compose.yaml).
  • Remove all containers and the default network by running:
docker compose down
  • Reclaim disk space from Docker images by running:
docker image prune -a

What does this command do?

This command removes all Docker images on your system that are not currently used by a running container. It will prompt you to confirm. Type y and press Enter to proceed.

If you have other Docker images you want to keep, skip this command. It only affects images with no running container, but that includes images from other projects.

  • Delete the project folder from your Desktop. Right-click the folder in File Explorer and select Delete.

Nice Work!

Nice Work!

Nice work! You just broke a running app on purpose, watched it crash, and then engineered it to survive the same failures gracefully. You now have hands-on experience with the chaos engineering methodology that SRE teams at Netflix, Google, and Amazon use every day.

You've learned how to:

  • Run chaos experiments using the steady-state hypothesis methodology: define normal, hypothesize, inject failure with docker stop, docker pause, and docker kill, observe, and fix.
  • Build resilience patterns into a multi-container Flask + Redis app, including try/except error handling, connection timeouts, and graceful fallback responses that keep the app alive when a dependency goes down.
  • Add health checks and restart policies to a Docker Compose stack so that Docker automatically restarts crashed containers without any manual intervention.
  • Secret Mission: Inject artificial latency via CHAOS_LATENCY_MS to simulate slow network conditions and monitor container resource usage in real time with docker stats.

Ready to quiz yourself?