Build a TCP Load Balancer in Go

Build a Layer-4 TCP load balancer from scratch using only Go's standard library.

Introduction

30 Second Summary

Every time you open a website, your request doesn't just go to one server. Behind the scenes, a load balancer decides which server handles your traffic, and it's one of the most critical pieces of internet infrastructure.

In this project, you will build a TCP load balancer from scratch in Go using only the standard library. You'll accept connections on a single port, distribute them across multiple backend servers, and automatically detect when a server goes down.

What You'll Build

A fully functional Layer-4 TCP load balancer that accepts incoming connections and distributes them across multiple backend servers using round-robin selection, with active health checks to route around failures.

By the end of this project, you'll have:

  • A working TCP load balancer that distributes connections across multiple backend servers using round-robin selection.
  • An active health checking system that detects dead backends and automatically removes them from rotation.
  • A deep understanding of how goroutines, io.Copy, and atomic operations work together to proxy network traffic concurrently.
  • Secret Mission: Add graceful shutdown so your load balancer finishes proxying active connections before exiting.

Are there any prerequisites?

You should be comfortable with basic Go syntax: variables, functions, structs, loops, and error handling. No networking experience is required.

Before We Start

Every time you visit a website or call an API, a load balancer is silently directing your request to one of many servers behind the scenes. Understanding how that works gives you a massive edge when debugging production incidents or designing high-availability architectures.

In this step, you will confirm your Go installation is ready and create the project skeleton for your TCP load balancer.

In this step, get ready to:
  • Verify your Go installation.
  • Create the project directory structure.
  • Initialize a Go module for the load balancer.
Verify your Go installation

Your load balancer will use Go's standard library networking primitives. Let's confirm Go is ready on your machine.

  • Check your Go version by running this command:
go version

You should see output like go version go1.26.5 linux/amd64 (your exact version number may differ). Any version 1.21 or higher works for this project.

Don't see a Go version?

If you see command not found, Go is not installed or not in your PATH. Follow the official install instructions at https://go.dev/doc/install and try again.

help me install Go on my machine

Create the project directory and initialize the module

Go projects live inside a module, which is a directory with a go.mod file that declares the module name and Go version. You will create a directory for your load balancer and a subdirectory for the backend test server you will build later.

  • Create the project directory structure and move into it by running these commands:
mkdir -p ~/tcp-loadbalancer/backend
cd ~/tcp-loadbalancer

What does this command do?

mkdir -p creates a directory and any parent directories that don't exist yet. This single command creates both tcp-loadbalancer/ and tcp-loadbalancer/backend/ inside your home directory.

  • Initialize a Go module by running:
go mod init tcp-loadbalancer

What does this command do?

go mod init creates a go.mod file that declares your project's module name (tcp-loadbalancer) and the Go version it targets. Every Go project needs this file so the compiler knows how to resolve imports.

  • Confirm the module was created by running:
ls

You should see backend and go.mod in the output.

Don't see go.mod?

Make sure you ran cd ~/tcp-loadbalancer before running go mod init. The go mod init command creates go.mod in whatever directory your terminal is currently in.

help me figure out why go.mod was not created

✔️ Awesome, I've got everything!

Great. Make sure you are inside the ~/tcp-loadbalancer directory for the rest of this project.

ⓧ I'd like to double check the full code

Your go.mod file should look like this:

module tcp-loadbalancer

go 1.21

The exact Go version number on the second line may differ depending on your installed Go version. Any value of 1.21 or higher is fine.

Your project skeleton is ready. Next up, you will build a backend echo server that your load balancer can distribute traffic to.

Set Up the Project and Backend Echo Server

Before you can build the load balancer itself, you need something to balance traffic to.

In this step, you will create a simple TCP echo server that acts as a backend. When a client sends text to this server, it echoes the text back with a prefix showing which port it is running on. Later, you will run three copies of this server to prove your load balancer distributes connections across all of them.

In this step, get ready to:
  • Create a backend echo server that identifies itself by port number.
  • Verify the backend responds to connections using netcat.
Create the backend echo server

The echo server is a minimal TCP server that reads lines from a connection and writes them back with a prefix. This gives you visible proof that the load balancer is routing traffic to different backends.

  • In your terminal (already in ~/tcp-loadbalancer), create backend/main.go by running:
touch backend/main.go
  • Confirm the file exists by running:
ls backend/

You should see main.go listed in the output.

  • Open backend/main.go in VS Code and paste the following code to set up the server's main() function:
package main

import (
	"bufio"
	"flag"
	"fmt"
	"log"
	"net"
)

func main() {
	// Define a command-line flag for the port number
	port := flag.Int("port", 5001, "port to listen on")
	flag.Parse()

	// Start listening for TCP connections on the specified port
	addr := fmt.Sprintf(":%d", *port)
	listener, err := net.Listen("tcp", addr)
	if err != nil {
		log.Fatalf("Failed to start backend on %s: %v", addr, err)
	}
	defer listener.Close()

	log.Printf("Backend server listening on %s", addr)

	// Accept connections in an infinite loop
	for {
		conn, err := listener.Accept()
		if err != nil {
			log.Printf("Error accepting connection: %v", err)
			continue
		}
		// Handle each connection in its own goroutine
		go handleConnection(conn, *port)
	}
}

What does this code do?

This is the entry point for your backend echo server.

  • flag.Int("port", 5001, ...) defines a -port flag so you can run multiple copies on different ports.
  • net.Listen("tcp", addr) opens a TCP socket and binds it to the specified port, ready to accept incoming connections.
  • The for loop calls listener.Accept() to wait for the next client connection. Each connection is handed off to a goroutine so the server can handle many clients at once.

You won't see a change yet. The next chunk adds the handleConnection function that main() calls.

  • Below the main() function in backend/main.go, paste the handleConnection function:
func handleConnection(conn net.Conn, port int) {
	defer conn.Close()

	// Read lines from the connection one at a time
	scanner := bufio.NewScanner(conn)
	for scanner.Scan() {
		line := scanner.Text()
		// Echo the line back with a prefix showing which backend responded
		response := fmt.Sprintf("[backend:%d] %s\n", port, line)
		conn.Write([]byte(response))
	}
}

What does handleConnection do?

This function handles a single client connection from start to finish.

  • bufio.NewScanner(conn) wraps the connection in a scanner that reads one line at a time.
  • For each line received, it builds a response like [backend:5001] hello and writes it back to the client.
  • defer conn.Close() ensures the connection is cleaned up when the function returns, even if an error occurs.
  • Save backend/main.go.

Seeing red squiggles in VS Code?

Make sure all your import paths are spelled exactly as shown. The bufio, flag, fmt, log, and net packages are all part of Go's standard library.

Also check that the package main declaration is on line 1. Go requires this to compile an executable.

Help me debug my Go code.

✔️ Awesome, I've got everything!

Great. Double check you have saved backend/main.go.

ⓧ I'd like to double check the full code

package main

import (
	"bufio"
	"flag"
	"fmt"
	"log"
	"net"
)

func main() {
	port := flag.Int("port", 5001, "port to listen on")
	flag.Parse()

	addr := fmt.Sprintf(":%d", *port)
	listener, err := net.Listen("tcp", addr)
	if err != nil {
		log.Fatalf("Failed to start backend on %s: %v", addr, err)
	}
	defer listener.Close()

	log.Printf("Backend server listening on %s", addr)

	for {
		conn, err := listener.Accept()
		if err != nil {
			log.Printf("Error accepting connection: %v", err)
			continue
		}
		go handleConnection(conn, *port)
	}
}

func handleConnection(conn net.Conn, port int) {
	defer conn.Close()

	scanner := bufio.NewScanner(conn)
	for scanner.Scan() {
		line := scanner.Text()
		response := fmt.Sprintf("[backend:%d] %s\n", port, line)
		conn.Write([]byte(response))
	}
}
Verify the backend works

Now you will run the echo server and send it a test message to confirm everything is wired up correctly.

  • Start the backend server on port 5001 by running:
go run backend/main.go -port 5001

You should see a log line containing Backend server listening on :5001.

  • Open a second terminal window.
  • Send a test message to the backend using nc (netcat) by running:
echo "hello" | nc localhost 5001

You should see this response:

[backend:5001] hello

What just happened?

Your echo server received the text hello over a TCP connection, prepended [backend:5001] to identify itself, and sent the response back.

This prefix is how you will confirm which backend handled each request when the load balancer is distributing traffic.

  • Switch back to the first terminal (where the backend is running).
  • Press Ctrl+C to stop the backend server.

Not seeing any response from netcat?

Make sure the backend is still running in the first terminal. If it crashed, check for typos in backend/main.go and re-run it.

If you see Connection refused, the backend is not listening. Confirm you started it with the correct command and that no other process is using port 5001.

Help me debug my netcat connection.

Your test backend is ready. In the next step, you will build the load balancer itself and have it accept incoming TCP connections.

Build the TCP Listener and Connection Acceptor

Your backend echo server is working. Clients can connect and get responses.

Now you need something sitting in front of those backends that accepts client connections and decides which backend gets each one. That something is your load balancer. In this step, you will create the load balancer's entry point, define the data structures that represent your backend servers, and wire up the connection acceptance loop.

In this step, get ready to:
  • Define the Backend and ServerPool structs that model your infrastructure.
  • Start a TCP listener on port 3030.
  • Accept incoming connections and spawn a goroutine for each one.
Create the load balancer skeleton

A load balancer needs to know about its backends and listen for incoming connections. You will define two structs to represent this: one for a single backend server, and one for the pool of all backends.

  • Create a new file called main.go in your tcp-loadbalancer folder (the project root, not the backend subfolder).
  • Add the package declaration, imports, and struct definitions by pasting this code into main.go:
package main

import (
	"log"
	"net"
)

type Backend struct {
	Address string
	Alive   bool
}

type ServerPool struct {
	backends []*Backend
	current  uint64
}

What do these structs represent?

  • Backend holds the address of a single backend server and whether it is currently alive.
  • ServerPool manages the full list of backends. The current field is a counter you will use later for round-robin selection.

You won't be able to run this yet. The next substep adds the main() function that uses these structs.

Implement the accept loop

The accept loop is the heart of your load balancer. It waits for incoming TCP connections and spawns a goroutine to handle each one concurrently.

  • Add the handleConnection function and the main() function below your struct definitions in main.go:
func handleConnection(conn net.Conn, pool *ServerPool) {
	defer conn.Close()
}

func main() {
	backends := []*Backend{
		{Address: "localhost:5001", Alive: true},
		{Address: "localhost:5002", Alive: true},
		{Address: "localhost:5003", Alive: true},
	}

	serverPool := &ServerPool{backends: backends}

	listener, err := net.Listen("tcp", ":3030")
	if err != nil {
		log.Fatalf("Failed to start load balancer: %v", err)
	}
	defer listener.Close()

	log.Println("Load Balancer started on :3030")

	for {
		conn, err := listener.Accept()
		if err != nil {
			log.Printf("Error accepting connection: %v", err)
			continue
		}
		go handleConnection(conn, serverPool)
	}
}

What does this code do?

  • handleConnection is a placeholder. For now, it just closes the connection immediately. You will add proxying logic in the next step.
  • main() creates three backend entries (ports 5001, 5002, 5003) and bundles them into a ServerPool.
  • net.Listen("tcp", ":3030") binds to port 3030 on all network interfaces and waits for connections.
  • The for loop calls listener.Accept() which blocks until a new client connects. Each connection gets its own goroutine via go handleConnection(conn, serverPool).

Why one goroutine per connection?

Each TCP connection might last seconds, minutes, or hours. If you handled them one at a time, the second client would have to wait until the first disconnects. Goroutines let Go handle thousands of connections concurrently without you managing threads manually.

  • Save main.go.

✔️ Awesome, I've got everything!

Great. Double check you have saved main.go in the project root (~/tcp-loadbalancer/main.go), not inside the backend folder.

ⓧ I'd like to double check the full code

package main

import (
	"log"
	"net"
)

type Backend struct {
	Address string
	Alive   bool
}

type ServerPool struct {
	backends []*Backend
	current  uint64
}

func handleConnection(conn net.Conn, pool *ServerPool) {
	defer conn.Close()
}

func main() {
	backends := []*Backend{
		{Address: "localhost:5001", Alive: true},
		{Address: "localhost:5002", Alive: true},
		{Address: "localhost:5003", Alive: true},
	}

	serverPool := &ServerPool{backends: backends}

	listener, err := net.Listen("tcp", ":3030")
	if err != nil {
		log.Fatalf("Failed to start load balancer: %v", err)
	}
	defer listener.Close()

	log.Println("Load Balancer started on :3030")

	for {
		conn, err := listener.Accept()
		if err != nil {
			log.Printf("Error accepting connection: %v", err)
			continue
		}
		go handleConnection(conn, serverPool)
	}
}

Seeing a compile error?

  • If you see declared and not used, make sure both code sections (the structs and the main function) are in the same file. Go requires every declared variable to be used.
  • If you see imported and not used, check that your import block contains only "log" and "net".

Help me fix a compile error in my Go load balancer main.go

Test the listener

Your load balancer should now accept connections on port 3030. Since handleConnection just closes the connection immediately, you will see netcat connect and then disconnect right away.

  • Start the load balancer by running:
go run main.go

You should see Load Balancer started on :3030 in the terminal. The process keeps running, waiting for connections.

  • Open a second terminal window.
  • Test the connection with netcat by running:
nc localhost 3030

Netcat connects and immediately exits. This confirms your load balancer accepted the connection and closed it (the placeholder behavior).

  • Check your load balancer terminal. You should see health check output or an accepted-connection log (depending on timing).
  • Stop the load balancer with Ctrl+C.

Connection refused on port 3030?

  • Make sure go run main.go is still running in the first terminal. If it exited with an error, check the error message.
  • If you see address already in use, another process is already on port 3030. Kill it or choose a different port in your code.
  • Make sure you are running nc localhost 3030 from the second terminal, not the one running the load balancer.

Help me debug a connection refused error on my Go TCP load balancer

Your load balancer skeleton is alive. It accepts connections and hands each one off to a goroutine. Next up, you will make those connections actually useful by implementing round-robin backend selection and proxying real traffic between the client and a backend.

Implement Round-Robin Selection and TCP Proxying

Your load balancer currently accepts connections on port :3030 and immediately drops them. That's like having a receptionist who picks up the phone and hangs up.

In this step, you'll teach it to pick a backend server and shuttle bytes between the client and that backend. This is the core of what every load balancer does: select a destination and proxy traffic to it.

In this step, get ready to:
  • Implement round-robin backend selection using atomic operations.
  • Build a TCP proxy function that shuttles bytes bidirectionally.
  • Test that connections cycle evenly across all three backends.
Implement thread-safe round-robin backend selection

The round-robin algorithm picks backends in a repeating cycle: 1, 2, 3, 1, 2, 3. Multiple goroutines handle connections simultaneously, so the counter that tracks which backend is "next" needs to be thread-safe.

Go's sync/atomic package provides lock-free atomic operations. These are faster than mutexes for simple counters because they avoid locking overhead entirely.

  • In main.go, update your import block at the top to add the new packages you'll need:
import (
	"io"
	"log"
	"net"
	"sync/atomic"
)

What changed in the imports?

You added "io" for io.Copy (used later for proxying bytes) and "sync/atomic" for the lock-free counter that powers round-robin selection.

  • Below the Backend struct (after the closing }), add two helper methods and the round-robin selection logic by copying the following code:
func (b *Backend) SetAlive(alive bool) {
	b.Alive = alive
}

func (b *Backend) IsAlive() bool {
	return b.Alive
}

What do SetAlive and IsAlive do?

These methods wrap access to the Alive field on each backend. Right now they look trivial, but they give you a single place to add thread-safety later (like a mutex or atomic boolean) without changing every caller.

  • Below the ServerPool struct (after its closing }), add the round-robin methods:
func (s *ServerPool) NextIndex() int {
	return int(atomic.AddUint64(&s.current, 1) % uint64(len(s.backends)))
}

func (s *ServerPool) GetNextPeer() *Backend {
	// Get the next index in the round-robin cycle
	next := s.NextIndex()
	// Search through all backends starting from next
	l := len(s.backends) + next

	for i := next; i < l; i++ {
		idx := i % len(s.backends)
		if s.backends[idx].IsAlive() {
			// Update the current pointer if we skipped dead backends
			if idx != next {
				atomic.StoreUint64(&s.current, uint64(idx))
			}
			return s.backends[idx]
		}
	}
	return nil
}

What does this code do?

NextIndex() atomically increments the counter and wraps it around the number of backends. Because it uses atomic.AddUint64, multiple goroutines can call it simultaneously without corrupting the counter.

GetNextPeer() calls NextIndex() to get a starting position, then loops through all backends looking for one that's alive. If it has to skip dead backends, it updates the counter with atomic.StoreUint64 so the next request picks up from the right place. If no backends are alive, it returns nil.

  • Save main.go.

Why atomic instead of a mutex?

A mutex would work, but for a single integer counter it's overkill. Atomic operations complete in a single CPU instruction with no locking. This is the idiomatic Go pattern for shared counters under high concurrency.

Implement the TCP proxy function

With backend selection in place, you now need a function that actually moves data between the client and the chosen backend. This is the "proxy" part of a load balancer.

The pattern is: dial the backend, then launch two goroutines. One copies bytes from client to backend. The other copies bytes from backend to client. When either direction finishes, you clean up both connections.

  • Delete the existing handleConnection function from main.go. Find these lines and remove them:
func handleConnection(conn net.Conn, pool *ServerPool) {
	defer conn.Close()
}
  • In the same spot where handleConnection was, add the proxy function:
func proxy(client net.Conn, backend *Backend) {
	defer client.Close()

	// Connect to the chosen backend
	target, err := net.Dial("tcp", backend.Address)
	if err != nil {
		log.Printf("Failed to connect to backend %s: %v", backend.Address, err)
		return
	}
	defer target.Close()

	log.Printf("Proxying %s -> %s", client.RemoteAddr(), backend.Address)

	// Channel to signal when either direction finishes
	done := make(chan struct{}, 2)

	// Copy bytes from client to backend
	go func() {
		io.Copy(target, client)
		done <- struct{}{}
	}()

	// Copy bytes from backend to client
	go func() {
		io.Copy(client, target)
		done <- struct{}{}
	}()

	// Wait for either direction to finish
	<-done
}

What does this code do?

  • net.Dial("tcp", backend.Address) opens an outbound TCP connection to the selected backend.
  • io.Copy(target, client) reads from the client socket and writes to the backend socket, one chunk at a time, until the client closes the connection.
  • io.Copy(client, target) does the reverse direction, carrying the backend's response back to the client.
  • The buffered channel done signals when either copy finishes. Once one direction closes, the deferred Close() calls clean up both sockets.
  • Now update the accept loop in main(). Find this line inside the for loop:
		go handleConnection(conn, serverPool)
  • Replace that single line with the following block that selects a peer and proxies the connection:
		peer := serverPool.GetNextPeer()
		if peer != nil {
			go proxy(conn, peer)
		} else {
			log.Println("No available backends")
			conn.Close()
		}

What does this code do?

For every new connection, the load balancer calls GetNextPeer() to pick the next healthy backend. If one is available, it spawns a goroutine running proxy() to handle the bidirectional data flow. If all backends are dead, it logs a warning and closes the client connection.

  • Save main.go.

Seeing a compile error?

  • If you see an error about io or sync/atomic being undefined, double check that your import block at the top includes both packages.
  • If you see an error about handleConnection being undefined, make sure you deleted the old function AND replaced the call in the accept loop.

Help me debug my main.go compile error

Test round-robin distribution

Time to see your load balancer in action. You'll start three backends and send requests through the load balancer, watching them cycle through each backend in order.

  • Open three separate terminal windows. Start a backend in each one by running:

Terminal 1:

cd ~/tcp-loadbalancer && go run backend/main.go -port 5001

Terminal 2:

cd ~/tcp-loadbalancer && go run backend/main.go -port 5002

Terminal 3:

cd ~/tcp-loadbalancer && go run backend/main.go -port 5003
  • Open a fourth terminal and start the load balancer by running:
cd ~/tcp-loadbalancer && go run main.go
  • You should see Load Balancer started on :3030 in the output.
  • Open a fifth terminal and send three requests through the load balancer by running:
echo "request 1" | nc localhost 3030
echo "request 2" | nc localhost 3030
echo "request 3" | nc localhost 3030
  • You should see responses cycling through the three backends:
[backend:5001] request 1
[backend:5002] request 2
[backend:5003] request 3

What just happened?

Each request hit your load balancer on port 3030. The load balancer picked the next backend using the atomic round-robin counter, opened a TCP connection to that backend, and shuttled the bytes back and forth. The prefix in each response ([backend:5001], [backend:5002], [backend:5003]) proves that connections were distributed evenly.

Not seeing responses cycle?

  • If you see no output at all, make sure all three backends are running (check each terminal for the Backend server listening on message).
  • If you only see responses from one backend, confirm your GetNextPeer() method uses atomic.AddUint64 (not a plain s.current++).
  • If nc hangs without returning, your proxy might not be closing connections properly. Check that both defer client.Close() and defer target.Close() are present in the proxy() function.

Help me debug my round-robin load balancer

  • Stop the load balancer and all three backends by pressing Ctrl+C in each terminal.

✔️ Awesome, I've got everything!

Great. Double check you've saved main.go and move on to the next step.

ⓧ I'd like to double check the full code

Here is the complete main.go at this point:

package main

import (
	"io"
	"log"
	"net"
	"sync/atomic"
)

type Backend struct {
	Address string
	Alive   bool
}

func (b *Backend) SetAlive(alive bool) {
	b.Alive = alive
}

func (b *Backend) IsAlive() bool {
	return b.Alive
}

type ServerPool struct {
	backends []*Backend
	current  uint64
}

func (s *ServerPool) NextIndex() int {
	return int(atomic.AddUint64(&s.current, 1) % uint64(len(s.backends)))
}

func (s *ServerPool) GetNextPeer() *Backend {
	next := s.NextIndex()
	l := len(s.backends) + next

	for i := next; i < l; i++ {
		idx := i % len(s.backends)
		if s.backends[idx].IsAlive() {
			if idx != next {
				atomic.StoreUint64(&s.current, uint64(idx))
			}
			return s.backends[idx]
		}
	}
	return nil
}

func proxy(client net.Conn, backend *Backend) {
	defer client.Close()

	target, err := net.Dial("tcp", backend.Address)
	if err != nil {
		log.Printf("Failed to connect to backend %s: %v", backend.Address, err)
		return
	}
	defer target.Close()

	log.Printf("Proxying %s -> %s", client.RemoteAddr(), backend.Address)

	done := make(chan struct{}, 2)

	go func() {
		io.Copy(target, client)
		done <- struct{}{}
	}()

	go func() {
		io.Copy(client, target)
		done <- struct{}{}
	}()

	<-done
}

func main() {
	backends := []*Backend{
		{Address: "localhost:5001", Alive: true},
		{Address: "localhost:5002", Alive: true},
		{Address: "localhost:5003", Alive: true},
	}

	serverPool := &ServerPool{backends: backends}

	listener, err := net.Listen("tcp", ":3030")
	if err != nil {
		log.Fatalf("Failed to start load balancer: %v", err)
	}
	defer listener.Close()

	log.Println("Load Balancer started on :3030")

	for {
		conn, err := listener.Accept()
		if err != nil {
			log.Printf("Error accepting connection: %v", err)
			continue
		}

		peer := serverPool.GetNextPeer()
		if peer != nil {
			go proxy(conn, peer)
		} else {
			log.Println("No available backends")
			conn.Close()
		}
	}
}

Your load balancer now distributes connections evenly across healthy backends. But what happens when a backend crashes? Right now, the load balancer would still try to send traffic to it and fail. Next up, you'll add health checks that automatically detect dead backends and remove them from rotation.

Add Active Health Checks

Your load balancer distributes connections across three backends, but right now it blindly sends traffic to all of them even if one has crashed. A dead backend means dropped connections and frustrated users.

Health checks solve this problem. Your load balancer will periodically probe each backend with a TCP connection attempt. If a backend fails to respond, the health checker marks it as dead and the round-robin selector skips it automatically.

In this step, get ready to:
  • Write a HealthCheck() method that probes each backend with a TCP dial.
  • Launch health checks in a background goroutine that runs every 5 seconds.
  • Test failover by killing a backend and confirming traffic reroutes.
Implement the health check function

At Layer 4, a successful TCP handshake (SYN/SYN-ACK/ACK) is enough to confirm a backend is alive and accepting connections. You do not need to send any application data. The standard library's net.DialTimeout function attempts this handshake with a configurable timeout.

  • In main.go, find the GetNextPeer() method. Add the following HealthCheck() method directly below it (before the proxy function):
func (s *ServerPool) HealthCheck() {
	for _, b := range s.backends {
		conn, err := net.DialTimeout("tcp", b.Address, 2*time.Second)
		if err != nil {
			b.SetAlive(false)
			log.Printf("Health check FAILED: %s", b.Address)
		} else {
			b.SetAlive(true)
			conn.Close()
			log.Printf("Health check OK: %s", b.Address)
		}
	}
}

What does this code do?

The method loops over every backend in the pool and attempts a TCP connection with a 2-second timeout.

  • If the dial succeeds, the backend is alive. The method marks it Alive = true and immediately closes the probe connection (it was only opened to test reachability).
  • If the dial fails (backend is down or unreachable within 2 seconds), the method marks it Alive = false and logs a warning.
  • The GetNextPeer() method you wrote earlier already skips backends where IsAlive() returns false, so marking a backend dead is enough to remove it from rotation.
  • Save main.go.

Seeing a compile error?

  • Make sure the method receiver is (s *ServerPool) with a pointer, not (s ServerPool). Without the pointer, changes to Alive would not persist.
  • Check that time is already in your import block at the top of the file. It should be there from Step 3.
  • Confirm the method is placed between the closing brace of GetNextPeer() and the func proxy(...) line. Placing it inside another function causes a nested function error.

Help me debug my HealthCheck method

Run health checks in a background goroutine

The health check method exists, but nothing calls it yet. You need a background loop that runs it on a schedule, independent of the main accept loop that handles incoming connections.

  • In main.go, scroll to the main() function. Find this line:
	serverPool := &ServerPool{backends: backends}
  • Add the following goroutine directly below that line (before the listener, err := net.Listen(...) line):
	go func() {
		for {
			serverPool.HealthCheck()
			time.Sleep(5 * time.Second)
		}
	}()

What does this code do?

  • The go func() { ... }() pattern launches an anonymous function as a separate goroutine. It runs concurrently with the main accept loop.
  • Inside the goroutine, an infinite for loop calls HealthCheck() then sleeps for 5 seconds before checking again.
  • This means every 5 seconds, the load balancer re-evaluates which backends are healthy. A backend that crashed gets marked dead within 5 seconds.
  • Save main.go.
  • Run the load balancer to confirm it compiles and starts:
go run main.go

You should see the startup message followed by health check logs printing every 5 seconds (all three will show FAILED since no backends are running yet).

  • Stop the load balancer with Ctrl+C.

Not seeing health check logs?

  • Make sure the goroutine is placed before the net.Listen call, not after the accept loop. The accept loop runs forever, so any code after it never executes.
  • Check that you have the () at the end of go func() { ... }(). Without the trailing parentheses, the anonymous function is defined but never called.

Help me debug my health check goroutine

✔️ Awesome, I've got everything!

Great. Double check you have saved main.go before moving on.

ⓧ I'd like to double check the full code

Here is the complete main.go at this point:

package main

import (
	"io"
	"log"
	"net"
	"sync/atomic"
	"time"
)

type Backend struct {
	Address string
	Alive   bool
}

func (b *Backend) SetAlive(alive bool) {
	b.Alive = alive
}

func (b *Backend) IsAlive() bool {
	return b.Alive
}

type ServerPool struct {
	backends []*Backend
	current  uint64
}

func (s *ServerPool) NextIndex() int {
	return int(atomic.AddUint64(&s.current, 1) % uint64(len(s.backends)))
}

func (s *ServerPool) GetNextPeer() *Backend {
	next := s.NextIndex()
	l := len(s.backends) + next

	for i := next; i < l; i++ {
		idx := i % len(s.backends)
		if s.backends[idx].IsAlive() {
			if idx != next {
				atomic.StoreUint64(&s.current, uint64(idx))
			}
			return s.backends[idx]
		}
	}
	return nil
}

func (s *ServerPool) HealthCheck() {
	for _, b := range s.backends {
		conn, err := net.DialTimeout("tcp", b.Address, 2*time.Second)
		if err != nil {
			b.SetAlive(false)
			log.Printf("Health check FAILED: %s", b.Address)
		} else {
			b.SetAlive(true)
			conn.Close()
			log.Printf("Health check OK: %s", b.Address)
		}
	}
}

func proxy(client net.Conn, backend *Backend) {
	defer client.Close()

	target, err := net.Dial("tcp", backend.Address)
	if err != nil {
		log.Printf("Failed to connect to backend %s: %v", backend.Address, err)
		return
	}
	defer target.Close()

	log.Printf("Proxying %s -> %s", client.RemoteAddr(), backend.Address)

	done := make(chan struct{}, 2)

	go func() {
		io.Copy(target, client)
		done <- struct{}{}
	}()

	go func() {
		io.Copy(client, target)
		done <- struct{}{}
	}()

	<-done
}

func main() {
	backends := []*Backend{
		{Address: "localhost:5001", Alive: true},
		{Address: "localhost:5002", Alive: true},
		{Address: "localhost:5003", Alive: true},
	}

	serverPool := &ServerPool{backends: backends}

	go func() {
		for {
			serverPool.HealthCheck()
			time.Sleep(5 * time.Second)
		}
	}()

	listener, err := net.Listen("tcp", ":3030")
	if err != nil {
		log.Fatalf("Failed to start load balancer: %v", err)
	}
	defer listener.Close()

	log.Println("Load Balancer started on :3030")

	for {
		conn, err := listener.Accept()
		if err != nil {
			log.Printf("Error accepting connection: %v", err)
			continue
		}

		peer := serverPool.GetNextPeer()
		if peer != nil {
			go proxy(conn, peer)
		} else {
			log.Println("No available backends")
			conn.Close()
		}
	}
}
Test failover behavior

The real test of a health-checking load balancer is what happens when a backend dies. You will kill one backend, watch the health checker detect the failure, and confirm that traffic reroutes to the remaining healthy backends.

  • Start three backends in three separate terminals by running each of these commands in its own terminal window:
go run backend/main.go -port 5001
go run backend/main.go -port 5002
go run backend/main.go -port 5003
  • In a fourth terminal, start the load balancer by running:
go run main.go

You should see the startup message and then health check logs confirming all three backends are OK.

  • In a fifth terminal, send a few requests to confirm round-robin is working:
echo "request 1" | nc localhost 3030
echo "request 2" | nc localhost 3030
echo "request 3" | nc localhost 3030

You should see responses cycling through all three backends.

  • Now kill the backend running on port 5002 by pressing Ctrl+C in that backend's terminal.
  • Wait up to 5 seconds for the next health check cycle. In the load balancer terminal, you should see:
Health check FAILED: localhost:5002
  • Send more requests through the load balancer:
echo "after kill 1" | nc localhost 3030
echo "after kill 2" | nc localhost 3030
echo "after kill 3" | nc localhost 3030
echo "after kill 4" | nc localhost 3030

Traffic now only goes to ports 5001 and 5003. The dead backend on 5002 is completely skipped.

  • Restart the killed backend by running this command in the terminal where you stopped it:
go run backend/main.go -port 5002
  • Wait for the next health check cycle (up to 5 seconds). You should see Health check OK: localhost:5002 in the load balancer logs.
  • Send a few more requests and confirm backend 5002 is back in the rotation.

Why does this matter?

You just implemented the same self-healing pattern that production load balancers like HAProxy and AWS Network Load Balancers use. When a server goes down, traffic automatically reroutes. When it recovers, it rejoins. No manual intervention needed.

Traffic still going to the dead backend?

  • If you send a request within the first 5 seconds after killing a backend, the health check has not run yet. Wait for the Health check FAILED log message before testing.
  • Make sure the goroutine launching HealthCheck() is placed before net.Listen. If it is placed after the accept loop, it never runs.
  • Confirm the SetAlive(false) call uses the pointer receiver. If Backend is not a pointer in your backends slice, changes to Alive will not be seen by GetNextPeer().

Help me debug my health check failover

Your load balancer now detects failures and heals itself automatically. Next up, you will tackle the Secret Mission and add graceful shutdown so active connections are not dropped during deploys.

Secret mission

Add Graceful Shutdown

Your load balancer works, but what happens when you press Ctrl+C? Active connections get dropped instantly, causing data loss for any in-flight requests. Can you add graceful shutdown so the load balancer finishes proxying active connections before the process exits?

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your project files for future reference or delete them entirely.

Resources you used:

  • Project directory: ~/tcp-loadbalancer (contains main.go, backend/main.go, and go.mod).

Keep everything running

No action needed. Choose this if you plan to extend the project further.

  • Your project directory stays at ~/tcp-loadbalancer.
  • You can pick up where you left off any time by running go run main.go from that directory.
  • Consider adding weighted round-robin, connection metrics, or TLS termination as your next challenge.

Pause - I'll come back to this later

Since all resources are local files, there is nothing to pause. Your project sits idle on disk until you return.

  • Make sure no backend or load balancer processes are still running. Press Ctrl+C in any terminal windows that are running them.
  • Your files remain at ~/tcp-loadbalancer ready to pick up later.

Delete - I don't want to use this again

Remove all project files and start fresh.

  • Stop any running backend or load balancer processes by pressing Ctrl+C in their terminal windows.
  • Delete the entire project directory by running:
rm -rf ~/tcp-loadbalancer
  • Confirm deletion by running:
ls ~/tcp-loadbalancer

You should see an error message indicating the directory does not exist. That confirms everything has been removed.

That's a Wrap!

That's a Wrap!

Nice work! You just built a production-style Layer-4 TCP load balancer from scratch using only Go's standard library.

You've learned how to:

  • Build a TCP load balancer that accepts connections with net.Listen, dials backends with net.Dial, and proxies traffic bidirectionally using io.Copy goroutines.
  • Implement thread-safe round-robin backend selection using sync/atomic for lock-free concurrency across multiple goroutines.
  • Add active health checks that probe backends with net.DialTimeout and automatically route around failures.
  • Secret Mission: Implement graceful shutdown with signal handling and connection draining so in-flight requests finish before the process exits.

Ready to quiz yourself?