DNS and Load Balancing with Docker

Build a local Docker lab comparing DNS round-robin vs health-checked load balancing.

Introduction

30 Second Summary

When you type a website address, your request doesn't magically reach the right server. Behind the scenes, DNS translates names into IP addresses, and load balancers decide which server handles your request.

In this project, you will build a local Docker-based network that simulates production traffic management. You'll configure DNS, deploy load balancers, and run chaos experiments to see what happens when servers fail.

What You'll Build

A 6-container Docker infrastructure with an authoritative BIND9 DNS server, two HAProxy load balancers, and three Nginx web servers on a custom bridge network.

By the end of this project, you'll have:

  • A Docker Compose stack with custom networking and static IP addressing across 6 containers.
  • An authoritative DNS zone with round-robin A records that distributes traffic across your infrastructure.
  • A working HAProxy load balancer with active health checks that automatically detects and routes around failed servers.
  • Secret Mission: Add cookie-based sticky sessions to maintain client affinity across requests.

Are there any prerequisites?

You should have basic terminal command familiarity and some Docker experience. You don't need to be an expert, but you should know how to run commands in a terminal and understand what containers are at a high level.

Before We Start

Before you can experiment with DNS and load balancing, you need to confirm your tools are in place. This project relies on Docker to run containers, Docker Compose to orchestrate them, and dig to query DNS records.

In this step, you'll verify everything is ready and set up the folder structure that organizes your configuration files.

In this step, get ready to:
  • Verify Docker, Docker Compose, and dig are installed.
  • Create the project directory structure with all required subdirectories.
Verify your tools are installed

Docker runs your containers, Docker Compose lets you define and launch all six services in one command, and dig is a DNS lookup utility you'll use to query your DNS server directly.

  • Confirm Docker, Docker Compose, and dig are available by running these commands:
docker --version
docker compose version
dig -v

What do these commands check?

  • docker --version confirms the Docker engine is installed.
  • docker compose version confirms Docker Compose V2 is available as a Docker subcommand.
  • dig -v prints the version of the DNS lookup utility from the dnsutils package.

You should see version numbers printed for each command.

Command not found?

  • If docker or docker compose is not found, install Docker Engine following the official instructions for your Linux distribution.
  • If dig is not found, install it by running:
sudo apt-get install dnsutils

help me troubleshoot a missing tool in my Docker and DNS project

Create the project directory structure

Your project needs separate folders for the DNS configuration, the HAProxy load balancer config, and each of the three web servers.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder with all required subdirectories and move into it by running these commands:
mkdir -p dns-lb-project/{dns,haproxy,web1,web2,web3}
cd dns-lb-project

What does this create?

The -p flag creates parent directories as needed. The brace expansion {dns,haproxy,web1,web2,web3} creates five subdirectories inside dns-lb-project/ in a single command.

  • dns/ holds your BIND9 DNS server configuration and zone files.
  • haproxy/ holds your load balancer configuration.
  • web1/, web2/, web3/ each hold content for one Nginx web server.
  • Confirm the directories exist by running:
ls

You should see dns, haproxy, web1, web2, and web3 listed in the output.

Not seeing the directories?

  • Make sure you ran the mkdir command from ~/Desktop. If you skipped the cd ~/Desktop step, the folder may have been created in a different location.
  • Confirm you are inside the project folder by running pwd. The output should end with dns-lb-project.

help me find my project directory

Your workspace is ready. Next up, you'll configure a DNS server and a load balancer to manage traffic across your web servers.

Set Up Project Files

Your project directories are ready. Now you need to fill them with the content your web servers will serve.

Each web server needs two things: an HTML page to show visitors, and a health check endpoint that tells the load balancer whether the server is alive. You will create both for each of the three web servers.

In this step, get ready to:
  • Create an HTML page for each web server that identifies which server it is.
  • Create a health check file for each web server that the load balancer will poll.
Create the web server content files

Each Nginx container will serve files from its volume-mounted directory. The index.html file is what visitors see, and the health file is what the load balancer checks to decide if the server is alive.

  • Create all six content files (one HTML page and one health check per web server) by running these commands:
echo '<html><body><h1>Web Server 1</h1><p>Served from web1</p></body></html>' > web1/index.html
echo '{"status": "healthy", "server": "web1"}' > web1/health
echo '<html><body><h1>Web Server 2</h1><p>Served from web2</p></body></html>' > web2/index.html
echo '{"status": "healthy", "server": "web2"}' > web2/health
echo '<html><body><h1>Web Server 3</h1><p>Served from web3</p></body></html>' > web3/index.html
echo '{"status": "healthy", "server": "web3"}' > web3/health

What do these files do?

Each index.html contains a heading that identifies its server (e.g. "Web Server 1"). When the load balancer routes traffic, this heading tells you which backend actually responded.

Each health file returns a simple JSON object. HAProxy will send HTTP GET requests to /health on each server every few seconds. If a server stops responding, HAProxy removes it from the rotation.

  • Confirm the files exist by running:
ls web1/ web2/ web3/

You should see health and index.html listed under each of the three directories.

  • Verify the content of one file to make sure it wrote correctly:
cat web1/index.html

You should see the full HTML line: <html><body><h1>Web Server 1</h1>...

Files are empty or missing?

  • Make sure your terminal is inside the dns-lb-project directory. Run pwd to confirm.
  • Check that the subdirectories exist. Run ls and confirm you see web1, web2, and web3 folders.
  • If a file is empty, re-run the echo command for that file. Make sure you use single quotes around the content so the shell does not interpret special characters.

Help me troubleshoot missing or empty web server files in my dns-lb-project.

✔️ Awesome, I've got everything!

Great. Make sure all six files are saved and you are still inside the dns-lb-project directory.

ⓧ I'd like to double check the full code

Here is the exact content each file should contain.

<html><body><h1>Web Server 1</h1><p>Served from web1</p></body></html>
{"status": "healthy", "server": "web1"}
<html><body><h1>Web Server 2</h1><p>Served from web2</p></body></html>
{"status": "healthy", "server": "web2"}
<html><body><h1>Web Server 3</h1><p>Served from web3</p></body></html>
{"status": "healthy", "server": "web3"}

Your web servers have their content ready. Next up, you will configure the DNS server and load balancer that sit in front of them.

Configure DNS and Load Balancer

Your web servers have content ready to serve. Now you need the infrastructure that directs traffic to them.

In this step, you'll configure two critical pieces: a DNS server that translates domain names into IP addresses, and a load balancer that distributes requests across your web servers while monitoring their health.

In this step, get ready to:
  • Configure an authoritative DNS server for the example.com domain.
  • Create a zone file with round-robin A records.
  • Set up HAProxy with health-checked load balancing and a stats dashboard.
Create the BIND9 DNS configuration

An authoritative DNS server is the final source of truth for a domain's records. Your BIND9 server will answer queries for example.com without forwarding them anywhere else.

  • Create dns/named.conf with the following configuration:
options {
    directory "/var/cache/bind";
    recursion no;
    allow-query { any; };
    listen-on { any; };
};

zone "example.com" {
    type primary;
    file "/var/lib/bind/db.example.com";
};

What does this configuration do?

  • recursion no tells BIND to only answer for zones it owns. It will never chase queries to other DNS servers on behalf of a client.
  • allow-query { any; } permits any container on the Docker network to send DNS queries.
  • type primary declares this server as the authoritative primary for the example.com zone.
  • The file directive points to the zone file where actual DNS records live.
  • Confirm the file exists by running:
cat dns/named.conf

You should see the options block and zone declaration printed back in your terminal.

File is empty or missing?

  • Make sure you created the file inside the dns/ directory, not in the project root.
  • Check for typos in the filename. It must be exactly named.conf (not named.cfg or name.conf).
  • help me troubleshoot my dns/named.conf file

✔️ Awesome, I've got everything!

Great. Your DNS server configuration is ready.

ⓧ I'd like to double check the full code

options {
    directory "/var/cache/bind";
    recursion no;
    allow-query { any; };
    listen-on { any; };
};

zone "example.com" {
    type primary;
    file "/var/lib/bind/db.example.com";
};
Create the DNS zone file

A zone file contains the actual DNS records that map domain names to IP addresses. You'll define A records for shop.example.com that point to your two load balancer IPs.

  • Create dns/db.example.com with the following zone definition:
$TTL 60
@   IN  SOA ns1.example.com. admin.example.com. (
    2024060101  ; serial
    3600        ; refresh
    1800        ; retry
    604800      ; expire
    60          ; minimum TTL
)

@       IN  NS  ns1.example.com.
ns1     IN  A   172.20.0.10

shop    IN  A   172.20.0.20
shop    IN  A   172.20.0.21

What does this zone file do?

  • $TTL 60 sets the default TTL (Time to Live) to 60 seconds. Clients will cache DNS answers for one minute before re-querying.
  • The SOA record identifies this as the authoritative start of the zone, with a serial number for tracking changes.
  • The NS record declares ns1.example.com as the nameserver, and the A record maps it to 172.20.0.10 (the DNS container's IP).
  • The two shop A records implement DNS round-robin. When a client queries shop.example.com, it receives both load balancer IPs (172.20.0.20 and 172.20.0.21) and picks one.

This is a key design choice. Traffic enters through the load balancers, not directly to the web servers.

  • Confirm the file exists by running:
cat dns/db.example.com

You should see the SOA record, NS record, and both shop A records printed in your terminal.

Zone file looks wrong?

  • Make sure there are no extra spaces before $TTL on the first line. It must start at column 1.
  • The trailing dots after ns1.example.com. and admin.example.com. are required. Without them, BIND appends the zone name automatically and the records break.
  • help me fix my DNS zone file

✔️ Awesome, I've got everything!

Great. Your zone file is ready to serve DNS records.

ⓧ I'd like to double check the full code

$TTL 60
@   IN  SOA ns1.example.com. admin.example.com. (
    2024060101  ; serial
    3600        ; refresh
    1800        ; retry
    604800      ; expire
    60          ; minimum TTL
)

@       IN  NS  ns1.example.com.
ns1     IN  A   172.20.0.10

shop    IN  A   172.20.0.20
shop    IN  A   172.20.0.21
Create the HAProxy load balancer configuration

HAProxy is an application load balancer that distributes incoming requests across your web servers. Unlike DNS, HAProxy actively monitors server health and stops sending traffic to servers that fail.

  • Create haproxy/haproxy.cfg with the following configuration:
global
    log stdout format raw local0

defaults
    log global
    mode http
    timeout connect 5s
    timeout client 50s
    timeout server 50s

frontend web_frontend
    bind *:80
    default_backend web_servers

backend web_servers
    balance roundrobin
    option httpchk
    http-check send meth GET uri /health
    server web1 172.20.0.30:80 check inter 3s rise 2 fall 2
    server web2 172.20.0.31:80 check inter 3s rise 2 fall 2
    server web3 172.20.0.32:80 check inter 3s rise 2 fall 2

listen stats
    bind *:8404
    stats enable
    stats uri /stats
    stats refresh 2s

What does this configuration do?

The config is split into five sections:

  • global sets logging to stdout so Docker can capture it.
  • defaults defines HTTP mode and connection timeouts applied to all frontends and backends.
  • frontend web_frontend listens on port 80 and forwards all traffic to the web_servers backend.
  • backend web_servers uses round-robin balancing and sends an HTTP GET to /health every 3 seconds. A server is marked down after 2 consecutive failures (fall 2) and restored after 2 successes (rise 2).
  • listen stats enables a live dashboard on port 8404 showing server status and traffic metrics.

Why HAProxy instead of Nginx as a load balancer?

Both can reverse-proxy traffic, but HAProxy is purpose-built for load balancing. Its health checks are active (HAProxy initiates the probe), while Nginx's basic health checks are passive (only triggered by real client requests failing).

HAProxy also provides a built-in stats dashboard with no extra plugins. For this project, that dashboard gives you real-time visibility into failover behaviour.

  • Confirm the file exists by running:
cat haproxy/haproxy.cfg

You should see the global, defaults, frontend, backend, and stats sections printed in your terminal.

Missing sections in the output?

  • HAProxy is indentation-sensitive. Make sure each directive is indented with exactly 4 spaces under its section header.
  • Check that http-check send meth GET uri /health is on a single line. If it wraps onto two lines, HAProxy will reject the config.
  • help me troubleshoot my HAProxy configuration

✔️ Awesome, I've got everything!

Great. Your load balancer configuration is complete.

ⓧ I'd like to double check the full code

global
    log stdout format raw local0

defaults
    log global
    mode http
    timeout connect 5s
    timeout client 50s
    timeout server 50s

frontend web_frontend
    bind *:80
    default_backend web_servers

backend web_servers
    balance roundrobin
    option httpchk
    http-check send meth GET uri /health
    server web1 172.20.0.30:80 check inter 3s rise 2 fall 2
    server web2 172.20.0.31:80 check inter 3s rise 2 fall 2
    server web3 172.20.0.32:80 check inter 3s rise 2 fall 2

listen stats
    bind *:8404
    stats enable
    stats uri /stats
    stats refresh 2s

All your infrastructure configuration is in place. Next up, you'll bring the entire stack online with Docker Compose and watch these components work together.

Launch the Stack and Verify

Your DNS zone file is configured to resolve shop.example.com to two load balancer IPs, and your HAProxy config is set to distribute traffic across three web servers.

Now it is time to wire everything together with Docker Compose. You will define a single file that creates all six containers on a custom network with static IP addresses, then bring the entire infrastructure up with one command.

In this step, get ready to:
  • Create a docker-compose.yml file defining all 6 services on a custom bridge network.
  • Bring up the stack and verify DNS resolution and load balancing work.
  • Confirm all web servers are healthy on the HAProxy stats dashboard.
Create the Docker Compose file

The docker-compose.yml file defines every container, its image, volume mounts, network assignment, and port mappings in one place.

  • Create the network and DNS service by adding the following to a new file called docker-compose.yml in your dns-lb-project folder:
networks:
  lb-net:
    driver: bridge
    ipam:
      config:
        - subnet: 172.20.0.0/24

services:
  dns-server:
    image: internetsystemsconsortium/bind9:9.20
    container_name: dns-server
    volumes:
      - ./dns/named.conf:/etc/bind/named.conf
      - ./dns/db.example.com:/var/lib/bind/db.example.com
    networks:
      lb-net:
        ipv4_address: 172.20.0.10
    ports:
      - "5353:53/udp"
      - "5353:53/tcp"

What does this code do?

The networks block creates a custom bridge network called lb-net with the subnet 172.20.0.0/24. This gives you 254 usable IP addresses that you assign to containers manually.

The dns-server service uses the official ISC BIND9 image. It bind-mounts your named.conf and zone file into the container. Port 5353 on the host maps to port 53 inside the container (both UDP and TCP) to avoid conflicts with any system DNS on port 53.

  • Add the three web server services below the dns-server service in the same docker-compose.yml file:
  web1:
    image: nginx:alpine
    container_name: web1
    volumes:
      - ./web1/index.html:/usr/share/nginx/html/index.html
      - ./web1/health:/usr/share/nginx/html/health
    networks:
      lb-net:
        ipv4_address: 172.20.0.30

  web2:
    image: nginx:alpine
    container_name: web2
    volumes:
      - ./web2/index.html:/usr/share/nginx/html/index.html
      - ./web2/health:/usr/share/nginx/html/health
    networks:
      lb-net:
        ipv4_address: 172.20.0.31

  web3:
    image: nginx:alpine
    container_name: web3
    volumes:
      - ./web3/index.html:/usr/share/nginx/html/index.html
      - ./web3/health:/usr/share/nginx/html/health
    networks:
      lb-net:
        ipv4_address: 172.20.0.32

What does this code do?

Each web server uses the lightweight nginx:alpine image. The volume mounts place your custom index.html and health file into Nginx's default document root so they are served at / and /health respectively.

The static IPs (172.20.0.30, 172.20.0.31, 172.20.0.32) match the addresses in your haproxy.cfg backend. HAProxy uses these IPs to route traffic and run health checks.

  • Add the two load balancer services below the web3 service:
  lb1:
    image: haproxy:2.8-alpine
    container_name: lb1
    volumes:
      - ./haproxy/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
    networks:
      lb-net:
        ipv4_address: 172.20.0.20
    ports:
      - "8080:80"
      - "8404:8404"

  lb2:
    image: haproxy:2.8-alpine
    container_name: lb2
    volumes:
      - ./haproxy/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
    networks:
      lb-net:
        ipv4_address: 172.20.0.21
    ports:
      - "8081:80"
      - "8405:8404"

What does this code do?

Both load balancers share the same haproxy.cfg mounted read-only (:ro). The :ro flag prevents the container from modifying your config file.

The IPs 172.20.0.20 and 172.20.0.21 match the A records in your DNS zone file. Port 8080 on the host reaches lb1's HTTP frontend, and port 8404 exposes its stats dashboard. Port 8081 and 8405 do the same for lb2.

  • Save docker-compose.yml.

✔️ Awesome, I've got everything!

Great. Double check you have saved your file before moving on.

ⓧ I'd like to double check the full code

networks:
  lb-net:
    driver: bridge
    ipam:
      config:
        - subnet: 172.20.0.0/24

services:
  dns-server:
    image: internetsystemsconsortium/bind9:9.20
    container_name: dns-server
    volumes:
      - ./dns/named.conf:/etc/bind/named.conf
      - ./dns/db.example.com:/var/lib/bind/db.example.com
    networks:
      lb-net:
        ipv4_address: 172.20.0.10
    ports:
      - "5353:53/udp"
      - "5353:53/tcp"

  web1:
    image: nginx:alpine
    container_name: web1
    volumes:
      - ./web1/index.html:/usr/share/nginx/html/index.html
      - ./web1/health:/usr/share/nginx/html/health
    networks:
      lb-net:
        ipv4_address: 172.20.0.30

  web2:
    image: nginx:alpine
    container_name: web2
    volumes:
      - ./web2/index.html:/usr/share/nginx/html/index.html
      - ./web2/health:/usr/share/nginx/html/health
    networks:
      lb-net:
        ipv4_address: 172.20.0.31

  web3:
    image: nginx:alpine
    container_name: web3
    volumes:
      - ./web3/index.html:/usr/share/nginx/html/index.html
      - ./web3/health:/usr/share/nginx/html/health
    networks:
      lb-net:
        ipv4_address: 172.20.0.32

  lb1:
    image: haproxy:2.8-alpine
    container_name: lb1
    volumes:
      - ./haproxy/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
    networks:
      lb-net:
        ipv4_address: 172.20.0.20
    ports:
      - "8080:80"
      - "8404:8404"

  lb2:
    image: haproxy:2.8-alpine
    container_name: lb2
    volumes:
      - ./haproxy/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
    networks:
      lb-net:
        ipv4_address: 172.20.0.21
    ports:
      - "8081:80"
      - "8405:8404"
  • Bring up all six containers by running:
docker compose up -d

Docker will pull the three images (BIND9, Nginx, HAProxy) if they are not cached locally. You will see progress lines for each image followed by container creation messages. This may take a minute on the first run.

Seeing an error about port 5353 already in use?

Another process may already be listening on port 5353. Run sudo lsof -i :5353 to identify it, then stop the conflicting process or change the host port mapping in docker-compose.yml.

If you see a YAML parsing error, check that every indentation level uses exactly two spaces and that the file does not contain tabs.

Help me debug my docker compose up error.

Verify containers and test DNS resolution
  • Check that all six containers are running by running:
docker compose ps

You should see six services listed, all with a status of Up.

  • Test DNS resolution by querying your BIND9 server:
dig shop.example.com @127.0.0.1 -p 5353

What should I see?

The ANSWER SECTION should contain two A records: 172.20.0.20 and 172.20.0.21. These are the load balancer IPs you defined in your zone file. The TTL should be 60 seconds.

  • Test that the load balancer routes traffic to a web server by running:
curl http://localhost:8080

You should see the HTML content from one of your web servers, such as <h1>Web Server 1</h1>.

Getting connection refused on port 8080?

HAProxy may still be starting up. Wait 5 seconds and try the curl command again. If it persists, run docker compose logs lb1 to check for configuration errors in your haproxy.cfg file.

Help me troubleshoot why curl to localhost:8080 is failing.

Check the HAProxy stats dashboard

HAProxy includes a built-in statistics page that shows real-time health status of every backend server.

  • Open your browser and navigate to http://localhost:8404/stats.
  • Look at the web_servers backend section.
  • Confirm that web1, web2, and web3 all show a green status indicator marked UP.

Why does the stats page matter?

This dashboard is how you observe HAProxy's health check system in action. Every 3 seconds, HAProxy sends a GET request to each server's /health endpoint. If a server fails two consecutive checks, HAProxy marks it DOWN and stops sending it traffic.

You will use this dashboard in the next steps to watch servers fail and recover in real time.

Stats page not loading?

Make sure you are using port 8404 (not 8080). If the page is blank or unreachable, run docker compose logs lb1 and look for errors related to the listen stats section of your config.

Help me access the HAProxy stats dashboard.

Your entire stack is running and healthy. Next up, you will break things on purpose to see why DNS-only load balancing fails when a server goes down.

Experiment with DNS Round-Robin Failure

Your infrastructure is running smoothly with HAProxy distributing traffic across all three web servers. But what would happen if you skipped the load balancer entirely and let DNS handle traffic distribution on its own?

Many teams assume that DNS round-robin (returning multiple IPs for a single hostname) is a form of load balancing. In this step, you will run a chaos experiment to prove why that assumption breaks down the moment a server goes offline.

In this step, get ready to:
  • Reconfigure DNS to point directly at web servers, bypassing the load balancers.
  • Observe DNS round-robin distributing traffic evenly.
  • Kill a server and watch DNS continue sending traffic to a dead endpoint.
Modify the zone file to bypass load balancers

Right now, your DNS zone file sends clients to the load balancer IPs. To simulate a DNS-only architecture, you need to point shop.example.com directly at the three web server IPs instead.

  • Open dns/db.example.com in your text editor.
  • Find these two lines at the bottom of the file:
shop    IN  A   172.20.0.20
shop    IN  A   172.20.0.21
  • Replace those two lines with three A records pointing directly at the web server IPs:
shop    IN  A   172.20.0.30
shop    IN  A   172.20.0.31
shop    IN  A   172.20.0.32

What does this change do?

Previously, DNS directed clients to the HAProxy load balancers (172.20.0.20 and 172.20.0.21), which then intelligently routed traffic to healthy web servers.

Now, DNS hands out the web server IPs directly. Clients connect straight to web1, web2, or web3 with no intermediary checking server health. This is pure DNS round-robin.

  • Save dns/db.example.com.
  • Restart the DNS container to pick up the new zone file by running:
docker compose restart dns-server
  • Verify the new records loaded by running:
dig shop.example.com @127.0.0.1 -p 5353

You should see three A records in the ANSWER SECTION: 172.20.0.30, 172.20.0.31, and 172.20.0.32.

Still seeing the old load balancer IPs?

Make sure you saved the file before restarting the container. The volume mount reads the file from disk each time BIND starts.

Check that you replaced both original lines (172.20.0.20 and 172.20.0.21) with the three new lines (172.20.0.30, 172.20.0.31, 172.20.0.32).

Help me debug my DNS zone file changes

✔️ Awesome, I've got everything!

Great. Double check you have saved your file before moving on.

ⓧ I'd like to double check the full code

$TTL 60
@   IN  SOA ns1.example.com. admin.example.com. (
    2024060101  ; serial
    3600        ; refresh
    1800        ; retry
    604800      ; expire
    60          ; minimum TTL
)

@       IN  NS  ns1.example.com.
ns1     IN  A   172.20.0.10

shop    IN  A   172.20.0.30
shop    IN  A   172.20.0.31
shop    IN  A   172.20.0.32
Test DNS round-robin with all servers healthy

With DNS now pointing directly at the web servers, you can simulate how a real client behaves. The client queries DNS, picks one of the returned IPs, and connects directly to that server.

  • Run this test loop that queries DNS and curls the first returned IP, simulating client behavior:
for i in {1..12}; do
  IP=$(dig +short shop.example.com @127.0.0.1 -p 5353 | head -1)
  curl -s --max-time 2 http://$IP | grep "Web Server" || echo "FAILED - $IP unreachable"
done

What does this loop do?

  • dig +short queries your DNS server and returns only the IP addresses.
  • head -1 picks the first IP from the response, simulating a client that always uses the top result.
  • curl -s --max-time 2 fetches the page from that IP with a 2-second timeout.
  • BIND rotates the order of IPs in each response, so the "first" IP changes each time. This is DNS round-robin in action.

You should see responses from all three servers: Web Server 1, Web Server 2, and Web Server 3 appearing in a roughly even distribution.

Why DNS round-robin looks fine right now

When all servers are healthy, DNS round-robin works perfectly well. Traffic spreads evenly without any extra infrastructure. This is exactly why teams are tempted to rely on it alone.

Kill a server and observe DNS failure

The real test of any traffic distribution system is what happens when a server dies unexpectedly. DNS has no mechanism to detect server health. It simply returns all the IPs it has on file, regardless of whether those servers are alive.

  • Stop the web2 container to simulate a server crash by running:
docker stop web2
  • Verify that DNS still returns web2's IP by running:
dig +short shop.example.com @127.0.0.1 -p 5353

You should still see all three IPs including 172.20.0.31 (web2's address). DNS has no idea the server is down.

  • Run the same test loop again to observe what happens when clients are sent to a dead server:
for i in {1..12}; do
  IP=$(dig +short shop.example.com @127.0.0.1 -p 5353 | head -1)
  curl -s --max-time 2 http://$IP | grep "Web Server" || echo "FAILED - $IP unreachable"
done

This time, you should see approximately one-third of requests fail with FAILED - 172.20.0.31 unreachable. The other two-thirds succeed normally against web1 and web3.

Why is this a critical problem?

DNS has no health awareness. It keeps returning the dead server's IP because nothing in the DNS protocol detects that the server stopped responding.

In production, this means one-third of your users would see errors or timeouts until someone manually updates the DNS records. Even then, cached DNS responses (governed by the TTL) continue directing traffic to the dead IP until they expire.

Seeing all requests fail instead of one-third?

If every request fails, your DNS server may not have loaded the updated zone file. Run docker compose restart dns-server again and verify with dig that the web server IPs appear.

Help me troubleshoot my DNS round-robin experiment

You have proven the fundamental limitation of DNS-only load balancing: it distributes traffic blindly with zero health awareness. In the next step, you will restore HAProxy in front of the servers and watch it handle the exact same failure scenario gracefully.

Experiment with HAProxy Health-Checked Failover

In the previous step, you proved that DNS round-robin has a fatal flaw. It keeps sending traffic to dead servers because it has no way to check if they are alive.

Now you will restore traffic routing through HAProxy and run the same chaos experiment. This time, the load balancer's health checks will detect the failure and route around it automatically.

In this step, get ready to:
  • Restore DNS to route through the load balancers.
  • Prove HAProxy handles server failure with zero dropped requests.
  • Observe automatic recovery when a failed server returns.
Restore the zone file and bring web2 back online

Right now, your DNS zone file points shop.example.com directly at the web server IPs. You need to revert it so DNS points at the load balancer IPs instead, putting HAProxy back in the traffic path.

  • Open dns/db.example.com in your text editor.
  • Find these three lines at the bottom of the file:
shop    IN  A   172.20.0.30
shop    IN  A   172.20.0.31
shop    IN  A   172.20.0.32
  • Replace all three lines with these two lines that point at the load balancer IPs:
shop    IN  A   172.20.0.20
shop    IN  A   172.20.0.21

What does this change do?

By pointing DNS back at the load balancer IPs (172.20.0.20 and 172.20.0.21), clients that resolve shop.example.com will hit HAProxy first. HAProxy then decides which web server to forward to, using its health check data to skip unhealthy ones.

  • Save dns/db.example.com.

✔️ Awesome, I've got everything!

Great. Double check you have saved your file before continuing.

ⓧ I'd like to double check the full code

$TTL 60
@   IN  SOA ns1.example.com. admin.example.com. (
    2024060101  ; serial
    3600        ; refresh
    1800        ; retry
    604800      ; expire
    60          ; minimum TTL
)

@       IN  NS  ns1.example.com.
ns1     IN  A   172.20.0.10

shop    IN  A   172.20.0.20
shop    IN  A   172.20.0.21
  • Restart the DNS container and bring web2 back online by running these commands:
docker compose restart dns-server
docker start web2

Why restart the DNS container?

BIND9 loads zone files at startup. Restarting the container forces it to re-read your updated db.example.com file so it serves the new A records.

  • Wait about 6 seconds for HAProxy's health checks to pass on web2 (two check intervals of 3 seconds each).
  • Verify DNS is serving the load balancer IPs by running:
dig shop.example.com @127.0.0.1 -p 5353

You should see 172.20.0.20 and 172.20.0.21 in the ANSWER SECTION, confirming traffic now routes through HAProxy.

Still seeing web server IPs in the dig output?

Make sure you saved dns/db.example.com before restarting the DNS container. Check that you replaced all three old lines with the two new ones.

Try restarting the DNS container again with docker compose restart dns-server.

Help me debug my DNS zone file changes.

Test failover through the load balancer

With all three web servers healthy and HAProxy back in the traffic path, you can now run the same chaos experiment from the previous step. This time, requests go through HAProxy instead of directly to web servers.

  • Send 12 requests through the load balancer to confirm even distribution by running:
for i in {1..12}; do curl -s http://localhost:8080 | grep "Web Server"; done

What should I see?

You should see an even round-robin pattern: Web Server 1, Web Server 2, Web Server 3, repeating four times. Each server handles exactly 4 of the 12 requests.

  • Now simulate a server failure by stopping web2:
docker stop web2
  • Wait about 6 seconds for HAProxy to detect the failure (two health check intervals).
  • Run the same test loop again:
for i in {1..12}; do curl -s http://localhost:8080 | grep "Web Server"; done

You should see 12 successful responses split between Web Server 1 and Web Server 3 only. Zero failed requests.

Why zero failures this time?

HAProxy's health checks detected that web2 stopped responding to GET /health requests. After two consecutive failures (fall 2 in your config), HAProxy removed web2 from the active pool. Traffic is automatically redistributed to the remaining healthy servers.

Compare this to DNS round-robin, where one-third of requests failed because DNS had no way to know web2 was down.

Seeing some failed requests?

If you ran the test loop immediately after stopping web2, HAProxy may not have detected the failure yet. Wait 6 seconds and run the loop again.

Help me understand why HAProxy is still routing to the stopped server.

Verify automatic recovery on the stats dashboard

HAProxy does not just remove failed servers. It continuously checks them and brings them back into rotation once they recover. You can observe this in real time on the stats dashboard.

  • Open the HAProxy stats dashboard in your browser at http://localhost:8404/stats.
  • Confirm that web2 is marked as DOWN (shown in red) while web1 and web3 show as UP (green).
  • Bring web2 back online by running:
docker start web2
  • Wait about 6 seconds (two health check intervals of 3 seconds).
  • Refresh the stats page in your browser. Web2 should now show as UP (green) again.

How does HAProxy know web2 is healthy again?

Your config sets rise 2, which means HAProxy needs two consecutive successful health checks before marking a server as UP. With checks every 3 seconds (inter 3s), that takes about 6 seconds.

  • Run the test loop one final time to confirm all three servers are back in rotation:
for i in {1..12}; do curl -s http://localhost:8080 | grep "Web Server"; done

You should see all three servers appearing in the output again, confirming that web2 has been automatically added back to the pool.

Web2 still not appearing in the output?

Make sure you waited at least 6 seconds after running docker start web2. Check the stats page to confirm web2 shows as UP before running the test loop.

If web2 stays DOWN on the stats page, verify the container is running with docker compose ps.

Help me figure out why web2 is not recovering.

You have now proven the full lifecycle: HAProxy detects failures, routes around them with zero dropped requests, and automatically re-adds servers once they recover. This is the pattern that production systems use to stay reliable. Ready to push it further? The Secret Mission adds cookie-based sticky sessions so returning users always hit the same server.

Secret mission

Add Cookie-Based Sticky Sessions

Your load balancer distributes requests evenly, but what about users with shopping carts or login sessions? Add cookie-based sticky sessions so that once a client connects to a server, all subsequent requests from that client go to the same server automatically.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally using Docker containers, so there are no ongoing costs.

Resources you used:

  • 6 Docker containers (dns-server, web1, web2, web3, lb1, lb2).
  • 1 custom Docker bridge network (lb-net).
  • Local project files in the dns-lb-project directory.

Keep everything running

No action needed. Choose this if you're still actively experimenting with DNS zones, load balancer configurations, or chaos testing.

  • Your containers use minimal CPU and memory when idle.
  • The HAProxy stats dashboard remains accessible at http://localhost:8404/stats.
  • You can continue stopping and starting individual containers for further failure testing.

Pause - I'll come back to this later

Stop all containers to free up memory while keeping your configuration files intact. You can restart everything later without rebuilding.

  • Stop all running containers by running:
docker compose stop
  • When you're ready to continue, navigate back to your project folder and start the containers again:
cd ~/Desktop/dns-lb-project
docker compose start

Delete - I don't want to use this again

Remove all containers, the custom network, downloaded images, and project files. This frees up all disk space used by the project.

  • Remove all containers, networks, and images by running:
docker compose down --rmi all
  • Delete the project directory and all configuration files:
cd ~/Desktop
rm -rf dns-lb-project
  • Confirm the folder is gone by running:
ls ~/Desktop | grep dns-lb

If nothing is returned, the project has been fully removed.

Nice Work!

Nice Work!

You just built a production-style traffic management lab from scratch using Docker containers. You now have hands-on proof of why every major web service layers health-checked load balancers on top of DNS.

Here's what you accomplished:

  • Built a 6-container Docker infrastructure with custom bridge networking, static IP addressing, and an authoritative BIND9 DNS server serving round-robin A records.
  • Proved that DNS-only load balancing fails silently when servers go down, with DNS continuing to return dead IPs and causing request failures.
  • Configured HAProxy with active health checks that automatically detect failed servers and reroute traffic with zero downtime, then observed real-time failover and recovery on the stats dashboard.
  • Secret Mission: Added cookie-based sticky sessions to maintain client affinity, ensuring repeat requests from the same client always reach the same backend server.

Ready to quiz yourself?