Build a Rate Limiter Service in Go

Implement the token bucket algorithm with Go concurrency primitives.

Introduction

30 Second Summary

Every time you open an app, your phone sends dozens of requests to a server. Without a traffic controller deciding who gets through and who has to wait, a single user could overwhelm the entire system.

In this project, you will build a rate limiter service in Go that tracks clients by IP address and rejects requests when they exceed their allowed burst.

What You'll Build

A token bucket rate limiter HTTP service with per-client tracking, concurrency-safe design, and standard rate limit response headers.

By the end of this project, you'll have:

  • A design decision document comparing four rate limiting algorithms and justifying your chosen approach.
  • A token bucket implementation with thread-safe concurrency using Go's sync.Mutex.
  • An HTTP middleware layer that returns 429 responses with standard rate limit headers and a concurrency test using Go's built-in race detector.
  • Secret Mission: A background cleanup goroutine that automatically evicts idle client buckets to prevent memory leaks at scale.

Are there any prerequisites?

You should be comfortable using a terminal and have a basic understanding of what an HTTP request is. No prior Go experience is required.

Before We Start

Before We Start...

Install Go and Set Up Your Editor

To build your rate limiter, you need Go installed on your machine. Go is a programming language built at Google for systems and network services.

In this step, you'll install Go, set up VS Code with Go-specific tooling, and create the project folder where your rate limiter will live.

In this step, get ready to:
  • Install Go 1.26.1 on your machine.
  • Set up VS Code with the Go extension for code completion and error checking.
  • Create and initialize the ratelimiter project.
Download and install Go

Go compiles to a single binary and has concurrency primitives baked into the language. These are the features that make it ideal for building a rate limiter service.

  • Check if Go is already installed by running this command in your terminal:
go version

✔️ I see go1.26 or higher

Your output looks something like:

go version go1.26.1 darwin/arm64

Go is ready. Skip ahead to the next substep.

ⓧ I see an older version

You have Go installed, but this project requires version 1.26.1 or higher. The installer will replace your existing version automatically.

macOS

  • Open https://go.dev/dl/ in your browser.
  • Download go1.26.1.darwin-arm64.pkg (Apple Silicon) or go1.26.1.darwin-amd64.pkg (Intel).
  • Double-click the downloaded .pkg file and follow the prompts to install.
  • Close and reopen your terminal so the updated PATH takes effect.
  • Verify the upgrade by running:
go version

You should now see go version go1.26.1 in the output.

Windows

  • Open https://go.dev/dl/ in your browser.
  • Download go1.26.1.windows-amd64.msi.
  • Double-click the downloaded .msi file and follow the installer prompts.
  • Close and reopen your terminal so the updated PATH takes effect.
  • Verify the upgrade by running:
go version

You should now see go version go1.26.1 in the output.

ⓧ Command not found

Go is not installed yet. Follow the instructions below to install it for the first time.

macOS

  • Open https://go.dev/dl/ in your browser.
  • Download go1.26.1.darwin-arm64.pkg (Apple Silicon) or go1.26.1.darwin-amd64.pkg (Intel).
  • Double-click the downloaded .pkg file and follow the prompts to install.
  • Close and reopen your terminal so the PATH takes effect.
  • Verify the installation by running:
go version

You should see go version go1.26.1 followed by your operating system and architecture.

Windows

  • Open https://go.dev/dl/ in your browser.
  • Download go1.26.1.windows-amd64.msi.
  • Double-click the downloaded .msi file and follow the installer prompts.
  • Close and reopen your terminal so the PATH takes effect.
  • Verify the installation by running:
go version

You should see go version go1.26.1 followed by your operating system and architecture.

Why close and reopen the terminal?

The Go installer adds /usr/local/go/bin to your system PATH. Your terminal only reads PATH changes when it starts a new session. Closing and reopening ensures go is available.

Install the Go extension in VS Code

The Go extension gives you code completion, error highlighting, and auto-formatting as you write Go code.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Visual Studio Code and press Enter to open it.
  • Open the Extensions panel by pressing Cmd+Shift+X (macOS) or Ctrl+Shift+X (Windows).
  • Type Go in the search bar at the top of the Extensions panel.
  • Find the extension published by Go Team at Google (extension ID: golang.Go).
  • Click Install.

What is gopls?

When you open your first .go file, the extension will automatically install gopls (the Go language server). This is what powers autocomplete, go-to-definition, and real-time error checking in your editor.

Don't see the extension?

Make sure you're searching for Go (not "Golang"). The correct extension shows Go Team at Google as the publisher and has the ID golang.Go.

help me find and install the Go extension in VS Code

Create the project directory and initialize a Go module

Every Go project needs a module. A module is a collection of Go source files with a go.mod file that declares the module path and Go version.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir ratelimiter
cd ratelimiter
  • Confirm the folder was created by running:
ls

You should see the ratelimiter folder listed in the output. If you already ran cd ratelimiter, the folder will be empty.

  • Open the ratelimiter folder in VS Code by running:
code .

code command not found?

If the code command is not recognized, open VS Code manually, then press Cmd+Shift+P (macOS) or Ctrl+Shift+P (Windows) and type Shell Command: Install 'code' command in PATH. Select it and restart your terminal.

help me open my project folder in VS Code from the terminal

  • Open a terminal inside VS Code by pressing Ctrl+` (backtick key, same shortcut on macOS and Windows).
  • Initialize your Go module by running this command:
go mod init github.com/yourname/ratelimiter

What does this command do?

go mod init creates a go.mod file that declares your module path and the Go version. The module path github.com/yourname/ratelimiter follows Go's naming convention for modules.

For this project, the exact module path does not matter since you will not be publishing it. In a real Go project, you would replace yourname with your actual GitHub username.

  • Verify the module was created by running:
cat go.mod

You should see your module path and the Go version printed in the output.

Seeing 'go: command not found' in the VS Code terminal?

The VS Code terminal may not have picked up the PATH change from your Go installation. Close VS Code completely and reopen it from a fresh terminal session using code . inside the ratelimiter folder.

help me fix 'go: command not found' in my VS Code terminal

✔️ Awesome, I've got everything!

Great. Double check that you have saved your file and your go.mod looks correct.

ⓧ I'd like to double check the full code

Your go.mod file should look exactly like this:

module github.com/yourname/ratelimiter

go 1.26.1

Your environment is ready. Next up, you'll write a design decision document that evaluates four rate limiting algorithms and justifies why you chose the token bucket approach.

Write a Design Decision Document

Your Go environment is ready and your project module is initialized. Before writing any code, you will create a design decision document that evaluates different rate limiting algorithms and justifies your choice.

Design docs are standard practice at Google and most large tech companies. Engineers write them to evaluate technology alternatives and document their reasoning. This maps directly to the Google SWE responsibility: "Participate in, or lead design reviews with peers and stakeholders to decide amongst available technologies."

In this step, get ready to:
  • Write a Problem Statement explaining why rate limiting matters.
  • Document four algorithm alternatives with strengths and weaknesses.
  • Record your decision and the trade-offs you accept.
Write the Problem Statement

Every design doc starts by framing the problem. This section answers: what breaks if we do nothing?

  • In the VS Code Explorer sidebar, click the new file icon at the top of your ratelimiter folder.
  • Name the file DESIGN.md and press Enter.
  • Paste the following content into DESIGN.md:
# Rate Limiter Design Decision

## Problem Statement

Our HTTP service needs to protect backend resources from being overwhelmed by too many requests from any single client. Without rate limiting, a single misbehaving client (or a DDoS attack) can exhaust server resources and degrade service for all users. We need a mechanism that:

- Limits each client to a fair share of requests
- Allows short bursts of traffic (real users are bursty)
- Returns clear feedback to clients about their remaining quota
- Operates with minimal latency overhead per request

What does this section do?

The Problem Statement defines the requirements your solution must satisfy. It frames four concrete needs: fairness between clients, burst tolerance, client feedback, and low latency.

Anyone reading this doc can now evaluate each algorithm against these four criteria. That is the power of a clear problem statement.

  • Save DESIGN.md with Cmd+S (macOS) or Ctrl+S (Windows).
  • Confirm DESIGN.md appears in the Explorer sidebar under your ratelimiter folder.

Don't see DESIGN.md in the sidebar?

Make sure you created the file inside the ratelimiter folder, not in a parent directory. The file should be at the same level as go.mod.

If you accidentally created it elsewhere, drag it into the ratelimiter folder in the Explorer sidebar.

Why isn't my DESIGN.md file showing up in the right place?

Document the algorithm alternatives

Four main rate limiting algorithms exist. Each makes a different trade-off between simplicity, accuracy, and burst handling. Documenting all four shows you understand the design space before committing to one.

  • In DESIGN.md, add a blank line after the last bullet point, then paste the following Alternatives Considered section below:
## Alternatives Considered

### 1. Fixed Window Counter
- **How it works**: Count requests in fixed time windows (e.g., 100 requests per minute). Reset counter at each window boundary.
- **Strength**: Simple to implement. O(1) memory per client (one counter).
- **Weakness**: Boundary burst problem. A client can send 100 requests at 0:59 and 100 more at 1:00, effectively getting 200 requests in 2 seconds.

### 2. Sliding Window Log
- **How it works**: Track timestamps of all requests in a sliding window. Count entries within the window to decide.
- **Strength**: No boundary burst problem. Accurate rate enforcement.
- **Weakness**: O(n) memory per client where n is the number of requests in the window. Expensive cleanup.

### 3. Token Bucket
- **How it works**: A bucket holds tokens up to a maximum capacity. Tokens refill at a steady rate. Each request consumes one token. If the bucket is empty, the request is rejected.
- **Strength**: Two independent tuning knobs (capacity for burst, refill rate for sustained throughput). Handles bursty traffic naturally. O(1) per request.
- **Weakness**: Requires per-client state (one bucket per client). Memory grows with unique clients unless stale buckets are evicted.

### 4. Leaky Bucket
- **How it works**: Requests enter a queue (the bucket). The queue drains at a fixed rate. If the queue is full, new requests are dropped.
- **Strength**: Produces perfectly smooth output rate. Good for protecting downstream services that cannot handle bursts.
- **Weakness**: Cannot accommodate any bursts. Real users experience unnecessary latency even when the system has capacity. More complex to implement (requires a queue and a drain worker).

What does this section do?

Each alternative is structured identically: how it works, one strength, one weakness. This consistent format makes comparison easy for a reviewer.

Notice the trade-offs: Fixed Window is simplest but has a burst exploit. Sliding Window is most accurate but uses the most memory. Leaky Bucket is smoothest but rejects legitimate bursts. Token Bucket balances burst tolerance with efficiency.

  • Save DESIGN.md.
  • Open the VS Code Markdown preview by pressing Cmd+Shift+V (macOS) or Ctrl+Shift+V (Windows). You should see the Problem Statement and all four algorithm sections rendered with bold labels and bullet points.

Preview looks garbled or shows raw markdown?

Check that you have a blank line between the Problem Statement bullets and the ## Alternatives Considered heading. Markdown requires blank lines between sections to render correctly.

Also verify that the bold markers use double asterisks (**How it works**) with no spaces between the asterisks and the text.

My DESIGN.md markdown preview is not rendering correctly in VS Code. How do I fix the formatting?

Record your decision and trade-offs

The final section commits to a choice and honestly states what you give up. Listing trade-offs shows you understand the limits of your design, not just its strengths.

  • In DESIGN.md, add a blank line after the Leaky Bucket weakness, then paste the following Decision and Trade-offs sections below:
## Decision

**Chosen approach: Token Bucket**

Token bucket provides the best balance for a user-facing API:
- Real users send bursty traffic (page loads, batch API calls). Token bucket absorbs these naturally.
- The two parameters (capacity and refill rate) map directly to business requirements: "allow up to 10 requests instantly, sustain 2 per second."
- O(1) time complexity per request with lazy refill (calculate tokens on demand rather than running a background refiller).
- Most commonly discussed algorithm in system design interviews at Google and similar companies.

## Trade-offs Accepted

- **Memory per client**: Each client requires a bucket struct (~64 bytes). At 1M unique clients, this is ~64MB. Acceptable, and we will implement idle client cleanup to bound growth.
- **No smoothing**: Unlike leaky bucket, we allow bursts up to capacity. For our use case (user-facing API), this is desirable, not a drawback.
- **Single-instance only**: This implementation is in-memory. For multi-instance deployments, a distributed approach (e.g., Redis-backed) would be needed. This is out of scope for the initial implementation.

Why token bucket over the other three?

Token bucket decouples burst capacity from sustained rate using two independent parameters. This means you can tune "how many requests can arrive at once" separately from "how many requests per second over time."

It is also the most commonly discussed rate limiting algorithm in system design interviews. Building it from scratch means you can walk an interviewer through the internals, not just name-drop it.

  • Save DESIGN.md.
  • Check the Markdown preview. You should now see five sections rendered: Problem Statement, Alternatives Considered (with four subsections), Decision, and Trade-offs Accepted.

Decision section not showing up in preview?

Make sure there is a blank line between the last line of the Leaky Bucket weakness and the ## Decision heading. Without a blank line, Markdown treats them as one block.

My Decision section is not rendering as a separate heading in the DESIGN.md preview. How do I fix this?

✔️ Awesome, I've got everything!

Great. Double check you have saved DESIGN.md before moving on.

ⓧ I'd like to double check the full code

Here is the complete DESIGN.md file. Make sure yours matches:

# Rate Limiter Design Decision

## Problem Statement

Our HTTP service needs to protect backend resources from being overwhelmed by too many requests from any single client. Without rate limiting, a single misbehaving client (or a DDoS attack) can exhaust server resources and degrade service for all users. We need a mechanism that:

- Limits each client to a fair share of requests
- Allows short bursts of traffic (real users are bursty)
- Returns clear feedback to clients about their remaining quota
- Operates with minimal latency overhead per request

## Alternatives Considered

### 1. Fixed Window Counter
- **How it works**: Count requests in fixed time windows (e.g., 100 requests per minute). Reset counter at each window boundary.
- **Strength**: Simple to implement. O(1) memory per client (one counter).
- **Weakness**: Boundary burst problem. A client can send 100 requests at 0:59 and 100 more at 1:00, effectively getting 200 requests in 2 seconds.

### 2. Sliding Window Log
- **How it works**: Track timestamps of all requests in a sliding window. Count entries within the window to decide.
- **Strength**: No boundary burst problem. Accurate rate enforcement.
- **Weakness**: O(n) memory per client where n is the number of requests in the window. Expensive cleanup.

### 3. Token Bucket
- **How it works**: A bucket holds tokens up to a maximum capacity. Tokens refill at a steady rate. Each request consumes one token. If the bucket is empty, the request is rejected.
- **Strength**: Two independent tuning knobs (capacity for burst, refill rate for sustained throughput). Handles bursty traffic naturally. O(1) per request.
- **Weakness**: Requires per-client state (one bucket per client). Memory grows with unique clients unless stale buckets are evicted.

### 4. Leaky Bucket
- **How it works**: Requests enter a queue (the bucket). The queue drains at a fixed rate. If the queue is full, new requests are dropped.
- **Strength**: Produces perfectly smooth output rate. Good for protecting downstream services that cannot handle bursts.
- **Weakness**: Cannot accommodate any bursts. Real users experience unnecessary latency even when the system has capacity. More complex to implement (requires a queue and a drain worker).

## Decision

**Chosen approach: Token Bucket**

Token bucket provides the best balance for a user-facing API:
- Real users send bursty traffic (page loads, batch API calls). Token bucket absorbs these naturally.
- The two parameters (capacity and refill rate) map directly to business requirements: "allow up to 10 requests instantly, sustain 2 per second."
- O(1) time complexity per request with lazy refill (calculate tokens on demand rather than running a background refiller).
- Most commonly discussed algorithm in system design interviews at Google and similar companies.

## Trade-offs Accepted

- **Memory per client**: Each client requires a bucket struct (~64 bytes). At 1M unique clients, this is ~64MB. Acceptable, and we will implement idle client cleanup to bound growth.
- **No smoothing**: Unlike leaky bucket, we allow bursts up to capacity. For our use case (user-facing API), this is desirable, not a drawback.
- **Single-instance only**: This implementation is in-memory. For multi-instance deployments, a distributed approach (e.g., Redis-backed) would be needed. This is out of scope for the initial implementation.

Your design document is complete. You have evaluated four algorithms, committed to token bucket, and documented the trade-offs. Next up, you will implement the token bucket algorithm in Go code.

Implement the Token Bucket Algorithm

Your design document lays out exactly why the token bucket algorithm is the right choice for this service. Now it is time to turn that decision into working Go code.

You will build the core data structure that holds tokens, refills them over time, and decides whether each incoming request is allowed or rejected. By the end of this step, you will have a running algorithm you can test directly in your terminal.

In this step, get ready to:
  • Define the TokenBucket struct and its constructor.
  • Implement the refill and allow logic with mutex protection.
  • Test the algorithm by running a loop that exhausts the bucket.
Define the TokenBucket struct and constructor

The token bucket needs to track how many tokens it currently holds, its maximum capacity, how fast tokens refill, and when the last refill happened. It also needs a mutex to prevent multiple requests from corrupting the token count at the same time.

  • Create a new file called limiter.go in your ratelimiter folder.
  • Add the following code to define the struct and its constructor:
package main

import (
	"sync"
	"time"
)

// TokenBucket implements the token bucket rate limiting algorithm.
type TokenBucket struct {
	capacity   float64
	tokens     float64
	refillRate float64
	lastRefill time.Time
	mu         sync.Mutex
}

// NewTokenBucket creates a new token bucket starting at full capacity.
func NewTokenBucket(capacity, refillRate float64) *TokenBucket {
	return &TokenBucket{
		capacity:   capacity,
		tokens:     capacity,
		refillRate: refillRate,
		lastRefill: time.Now(),
	}
}

What does this code do?

  • capacity sets the maximum number of tokens (the burst limit).
  • tokens tracks how many tokens are currently available.
  • refillRate controls how many tokens are added per second (the sustained rate).
  • mu sync.Mutex is a lock that ensures only one goroutine can read or modify the bucket at a time.
  • NewTokenBucket initializes a bucket that starts completely full, so the first burst of requests goes through immediately.
  • Save limiter.go.

You will not see any output yet. The next chunk adds the methods that use this struct.

Add the refill and allow methods

The token bucket uses "lazy refill." Instead of a background timer constantly adding tokens, it calculates how many tokens should have been added since the last request. This means the bucket does zero work between requests.

  • In limiter.go, add the following two methods below the NewTokenBucket function:
// refill adds tokens based on elapsed time since the last refill.
func (tb *TokenBucket) refill() {
	now := time.Now()
	elapsed := now.Sub(tb.lastRefill).Seconds()
	tokensToAdd := elapsed * tb.refillRate
	tb.tokens = min(tb.capacity, tb.tokens+tokensToAdd)
	tb.lastRefill = now
}

// Allow checks if a request is allowed and consumes one token if so.
// Returns whether the request was allowed and the remaining token count.
func (tb *TokenBucket) Allow() (bool, int) {
	tb.mu.Lock()
	defer tb.mu.Unlock()

	tb.refill()

	if tb.tokens >= 1 {
		tb.tokens--
		return true, int(tb.tokens)
	}
	return false, 0
}

What does this code do?

  • refill() calculates how many seconds have passed, multiplies by the refill rate, and adds that many tokens back. The min function caps tokens at the bucket's capacity so they never overflow.
  • Allow() locks the mutex so no other goroutine can interfere, calls refill to top up tokens, then checks if at least 1 token is available.
  • If a token is available, it subtracts one and returns true with the remaining count. If the bucket is empty, it returns false with 0.

Why lazy refill instead of a background timer?

A background timer would wake up every second to add tokens to every client's bucket, even when nobody is making requests. Lazy refill does the math on demand, only when a request arrives. This is O(1) per request with zero background cost.

  • Save limiter.go.

You will not see any output yet. The next substep creates a test to run the algorithm.

✔️ Awesome, I've got everything!

Great. Double check you have saved limiter.go before continuing.

ⓧ I'd like to double check the full code

package main

import (
	"sync"
	"time"
)

// TokenBucket implements the token bucket rate limiting algorithm.
type TokenBucket struct {
	capacity   float64
	tokens     float64
	refillRate float64
	lastRefill time.Time
	mu         sync.Mutex
}

// NewTokenBucket creates a new token bucket starting at full capacity.
func NewTokenBucket(capacity, refillRate float64) *TokenBucket {
	return &TokenBucket{
		capacity:   capacity,
		tokens:     capacity,
		refillRate: refillRate,
		lastRefill: time.Now(),
	}
}

// refill adds tokens based on elapsed time since the last refill.
func (tb *TokenBucket) refill() {
	now := time.Now()
	elapsed := now.Sub(tb.lastRefill).Seconds()
	tokensToAdd := elapsed * tb.refillRate
	tb.tokens = min(tb.capacity, tb.tokens+tokensToAdd)
	tb.lastRefill = now
}

// Allow checks if a request is allowed and consumes one token if so.
// Returns whether the request was allowed and the remaining token count.
func (tb *TokenBucket) Allow() (bool, int) {
	tb.mu.Lock()
	defer tb.mu.Unlock()

	tb.refill()

	if tb.tokens >= 1 {
		tb.tokens--
		return true, int(tb.tokens)
	}
	return false, 0
}
Test the algorithm with a temporary main file

Before building the HTTP server, you will test the token bucket in isolation. A simple loop that makes more requests than the bucket can hold proves the algorithm works.

  • Create a new file called main.go in your ratelimiter folder.
  • Add the following code to create a bucket and test it:
package main

import "fmt"

func main() {
	bucket := NewTokenBucket(5, 1)
	for i := 0; i < 7; i++ {
		allowed, remaining := bucket.Allow()
		fmt.Printf("Request %d: allowed=%v, remaining=%d\n", i+1, allowed, remaining)
	}
}

What does this code do?

  • NewTokenBucket(5, 1) creates a bucket that allows a burst of 5 requests and refills at 1 token per second.
  • The loop fires 7 requests instantly. Since the bucket starts full with 5 tokens, the first 5 succeed and the last 2 are rejected.
  • go run . compiles and runs all .go files in the current directory that share the same package.
  • Save main.go.
  • Run the algorithm by running this command in your terminal:
go run .

You should see output like this:

Request 1: allowed=true, remaining=4
Request 2: allowed=true, remaining=3
Request 3: allowed=true, remaining=2
Request 4: allowed=true, remaining=1
Request 5: allowed=true, remaining=0
Request 6: allowed=false, remaining=0
Request 7: allowed=false, remaining=0

The first 5 requests drain the bucket one token at a time. Requests 6 and 7 are rejected because the bucket is empty.

Seeing a compile error?

  • Check that both limiter.go and main.go start with package main. They must share the same package name for go run . to compile them together.
  • Make sure you are running the command from inside the ratelimiter folder where both files live.
  • If you see an error about min being undefined, confirm your Go version is 1.21 or higher by running go version.

Help me debug my Go compile error

✔️ Awesome, I've got everything!

Great. Your token bucket algorithm is working correctly.

ⓧ I'd like to double check the full code

package main

import "fmt"

func main() {
	bucket := NewTokenBucket(5, 1)
	for i := 0; i < 7; i++ {
		allowed, remaining := bucket.Allow()
		fmt.Printf("Request %d: allowed=%v, remaining=%d\n", i+1, allowed, remaining)
	}
}

Your token bucket algorithm is working. You can see it draining tokens and rejecting requests once they run out. Next up, you will add per-client isolation so each user gets their own independent bucket.

Add Per-Client Rate Limiting

Your token bucket algorithm works, but right now it only tracks a single bucket.

In a real service, a single global bucket means one abusive client can exhaust the entire limit and starve everyone else. Per-client rate limiting solves this by giving each user their own independent bucket, isolated from other traffic.

In this step, get ready to:
  • Add a RateLimiter struct that manages individual buckets per client.
  • Implement methods to create, retrieve, and check buckets for each client.
  • Test per-client isolation by running two clients against independent limits.
Add the RateLimiter struct

The RateLimiter struct acts as a registry of client buckets. It holds a Go map that links each client's ID to their personal TokenBucket instance, plus its own mutex to protect concurrent map access.

  • In limiter.go, scroll to the bottom of the file (below the existing Allow() method) and add this struct:
// RateLimiter manages per-client token buckets.
type RateLimiter struct {
	buckets    map[string]*TokenBucket
	mu         sync.Mutex
	capacity   float64
	refillRate float64
}

What does this struct do?

  • buckets is a map from client ID strings to their individual TokenBucket pointers. Each client gets their own isolated bucket.
  • mu is a separate mutex that protects the map itself. Go maps are not safe for concurrent reads and writes, so this lock prevents race conditions when multiple goroutines add or look up clients simultaneously.
  • capacity and refillRate store the configuration so new client buckets are created with the same settings.

You won't see a change yet. The next chunk adds the methods that bring this struct to life.

Implement the constructor and methods

The RateLimiter needs three methods: a constructor to initialize it, a method to get or create a bucket for a client, and an Allow() method that delegates to the correct client's bucket.

  • Add these three functions below the RateLimiter struct in limiter.go:
// NewRateLimiter creates a rate limiter that tracks clients individually.
func NewRateLimiter(capacity, refillRate float64) *RateLimiter {
	return &RateLimiter{
		buckets:    make(map[string]*TokenBucket),
		capacity:   capacity,
		refillRate: refillRate,
	}
}

// GetBucket returns the token bucket for a given client, creating one if needed.
func (rl *RateLimiter) GetBucket(clientID string) *TokenBucket {
	rl.mu.Lock()
	defer rl.mu.Unlock()

	bucket, exists := rl.buckets[clientID]
	if !exists {
		bucket = NewTokenBucket(rl.capacity, rl.refillRate)
		rl.buckets[clientID] = bucket
	}
	return bucket
}

// Allow checks if a request from the given client should be allowed.
func (rl *RateLimiter) Allow(clientID string) (bool, int) {
	bucket := rl.GetBucket(clientID)
	return bucket.Allow()
}

What does this code do?

  • NewRateLimiter initializes an empty map with make() and stores the capacity and refill rate configuration for creating future buckets.
  • GetBucket locks the map, checks if a bucket already exists for that client, and creates one if not. This is the "get or create" pattern, common in Go services.
  • Allow is a convenience method that fetches the client's bucket and delegates to the bucket's own Allow(). Callers only need a client ID to check rate limits.

Why two mutexes?

The RateLimiter has its own mutex to protect the map of buckets. Each TokenBucket also has its own mutex to protect the token count.

This two-level locking means clients don't block each other. The map lock is held only long enough to find or create a bucket, then released. The bucket lock is held only for the fast token check. Two clients making requests at the same time never compete for the same lock.

You won't see a change yet. The next substep updates main.go to test these methods.

Test per-client isolation

Now you can prove that each client gets their own independent token bucket. You will replace the single-bucket test in main.go with a loop that tests two different client IDs.

  • Replace the entire contents of main.go with this per-client test:
package main

import "fmt"

func main() {
	limiter := NewRateLimiter(3, 1)
	clients := []string{"client-a", "client-b"}
	for _, clientID := range clients {
		fmt.Printf("--- %s ---\n", clientID)
		for i := 0; i < 5; i++ {
			allowed, remaining := limiter.Allow(clientID)
			fmt.Printf("Request %d: allowed=%v, remaining=%d\n", i+1, allowed, remaining)
		}
	}
}

What does this test do?

  • NewRateLimiter(3, 1) creates a limiter where every new client starts with 3 tokens and refills at 1 token per second.
  • The loop sends 5 requests for each client. Since the capacity is 3, the first 3 requests per client should be allowed and the last 2 rejected.
  • The key insight: client-b still gets 3 allowed requests even after client-a exhausted its own bucket. That proves isolation is working.
  • Save both limiter.go and main.go.
  • Run the program by entering this command in your terminal:
go run .

You should see output like this, confirming each client gets their own independent budget:

--- client-a ---
Request 1: allowed=true, remaining=2
Request 2: allowed=true, remaining=1
Request 3: allowed=true, remaining=0
Request 4: allowed=false, remaining=0
Request 5: allowed=false, remaining=0
--- client-b ---
Request 1: allowed=true, remaining=2
Request 2: allowed=true, remaining=1
Request 3: allowed=true, remaining=0
Request 4: allowed=false, remaining=0
Request 5: allowed=false, remaining=0

Seeing a compile error?

  • Check that both files start with package main. All .go files in the same directory must share the same package name.
  • Make sure your main.go no longer references NewTokenBucket directly. It should call NewRateLimiter instead.
  • If you see "undefined: NewRateLimiter", confirm you saved limiter.go after adding the new struct and methods.

Help me debug my Go compile error for the RateLimiter

✔️ Awesome, I've got everything!

Great. Double check you have saved both files before moving on.

ⓧ I'd like to double check the full code

Here is the complete limiter.go at this point:

package main

import (
	"sync"
	"time"
)

// TokenBucket implements the token bucket rate limiting algorithm.
type TokenBucket struct {
	capacity   float64
	tokens     float64
	refillRate float64
	lastRefill time.Time
	mu         sync.Mutex
}

// NewTokenBucket creates a new token bucket starting at full capacity.
func NewTokenBucket(capacity, refillRate float64) *TokenBucket {
	return &TokenBucket{
		capacity:   capacity,
		tokens:     capacity,
		refillRate: refillRate,
		lastRefill: time.Now(),
	}
}

// refill adds tokens based on elapsed time since the last refill.
func (tb *TokenBucket) refill() {
	now := time.Now()
	elapsed := now.Sub(tb.lastRefill).Seconds()
	tokensToAdd := elapsed * tb.refillRate
	tb.tokens = min(tb.capacity, tb.tokens+tokensToAdd)
	tb.lastRefill = now
}

// Allow checks if a request is allowed and consumes one token if so.
// Returns whether the request was allowed and the remaining token count.
func (tb *TokenBucket) Allow() (bool, int) {
	tb.mu.Lock()
	defer tb.mu.Unlock()

	tb.refill()

	if tb.tokens >= 1 {
		tb.tokens--
		return true, int(tb.tokens)
	}
	return false, 0
}

// RateLimiter manages per-client token buckets.
type RateLimiter struct {
	buckets    map[string]*TokenBucket
	mu         sync.Mutex
	capacity   float64
	refillRate float64
}

// NewRateLimiter creates a rate limiter that tracks clients individually.
func NewRateLimiter(capacity, refillRate float64) *RateLimiter {
	return &RateLimiter{
		buckets:    make(map[string]*TokenBucket),
		capacity:   capacity,
		refillRate: refillRate,
	}
}

// GetBucket returns the token bucket for a given client, creating one if needed.
func (rl *RateLimiter) GetBucket(clientID string) *TokenBucket {
	rl.mu.Lock()
	defer rl.mu.Unlock()

	bucket, exists := rl.buckets[clientID]
	if !exists {
		bucket = NewTokenBucket(rl.capacity, rl.refillRate)
		rl.buckets[clientID] = bucket
	}
	return bucket
}

// Allow checks if a request from the given client should be allowed.
func (rl *RateLimiter) Allow(clientID string) (bool, int) {
	bucket := rl.GetBucket(clientID)
	return bucket.Allow()
}

And here is the complete main.go:

package main

import "fmt"

func main() {
	limiter := NewRateLimiter(3, 1)
	clients := []string{"client-a", "client-b"}
	for _, clientID := range clients {
		fmt.Printf("--- %s ---\n", clientID)
		for i := 0; i < 5; i++ {
			allowed, remaining := limiter.Allow(clientID)
			fmt.Printf("Request %d: allowed=%v, remaining=%d\n", i+1, allowed, remaining)
		}
	}
}

Your rate limiter now tracks each client independently. Next up, you will wrap this logic in an HTTP server so real network requests get rate-limited automatically with proper response headers.

Build the HTTP Server with Rate Limiting Middleware

Your token bucket algorithm works and your per-client rate limiter isolates traffic between users.

Now it is time to connect your rate limiter to a real HTTP server so clients can actually hit it with requests. You will build middleware that sits between every incoming request and your handler, deciding whether to let traffic through or reject it with a 429 status code.

In this step, get ready to:
  • Create rate limiting middleware that wraps any HTTP handler.
  • Build an HTTP server with protected and unprotected endpoints.
  • Test the rate limiter end-to-end with curl to see requests get throttled.
Create the rate limiting middleware

Middleware is a function that wraps an HTTP handler, adding behavior before or after the handler runs. Your middleware will intercept every request, check the client's token bucket, and either pass the request through or reject it.

  • Create a new file called middleware.go in your ratelimiter folder.
  • Add the following code to middleware.go:
package main

import (
	"fmt"
	"net"
	"net/http"
)

// RateLimitMiddleware wraps an HTTP handler with per-client rate limiting.
func RateLimitMiddleware(limiter *RateLimiter, next http.HandlerFunc) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ip, _, err := net.SplitHostPort(r.RemoteAddr)
		if err != nil {
			ip = r.RemoteAddr
		}

		allowed, remaining := limiter.Allow(ip)

		w.Header().Set("X-RateLimit-Limit", fmt.Sprintf("%.0f", limiter.capacity))
		w.Header().Set("X-RateLimit-Remaining", fmt.Sprintf("%d", remaining))

		if !allowed {
			w.Header().Set("Retry-After", "1")
			http.Error(w, "Too Many Requests", http.StatusTooManyRequests)
			return
		}

		next(w, r)
	}
}

What does this code do?

RateLimitMiddleware takes a *RateLimiter and a handler function, then returns a new handler function that wraps the original with rate limiting logic.

  • net.SplitHostPort(r.RemoteAddr) extracts the client's IP address from the incoming request. Every HTTP request carries the sender's IP and port as a host:port string.
  • X-RateLimit-Limit and X-RateLimit-Remaining headers tell the client their quota and how many tokens are left. These headers are set on every response, whether allowed or denied.
  • If the bucket is empty, the middleware sets a Retry-After header and returns HTTP 429 (Too Many Requests) without ever calling the actual handler.
  • Save middleware.go.

You will not see a change yet. The middleware needs a server to attach to, which you will build next.

Why middleware instead of checking inside each handler?

Middleware lets you apply rate limiting to any endpoint with a single line of code. Without it, you would copy-paste the same check into every handler. If you add 10 endpoints later, the middleware approach means zero extra rate limiting code.

✔️ Awesome, I've got everything!

Great. Double check you have saved middleware.go.

ⓧ I'd like to double check the full code

package main

import (
	"fmt"
	"net"
	"net/http"
)

// RateLimitMiddleware wraps an HTTP handler with per-client rate limiting.
func RateLimitMiddleware(limiter *RateLimiter, next http.HandlerFunc) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ip, _, err := net.SplitHostPort(r.RemoteAddr)
		if err != nil {
			ip = r.RemoteAddr
		}

		allowed, remaining := limiter.Allow(ip)

		w.Header().Set("X-RateLimit-Limit", fmt.Sprintf("%.0f", limiter.capacity))
		w.Header().Set("X-RateLimit-Remaining", fmt.Sprintf("%d", remaining))

		if !allowed {
			w.Header().Set("Retry-After", "1")
			http.Error(w, "Too Many Requests", http.StatusTooManyRequests)
			return
		}

		next(w, r)
	}
}
Replace main.go with the HTTP server

The temporary test loop in main.go has served its purpose. Now you will replace it with an actual HTTP server that exposes endpoints and wires up the middleware.

  • Open main.go in your editor.
  • Select all the existing code and replace it with the following:
package main

import (
	"fmt"
	"log"
	"net/http"
)

func main() {
	limiter := NewRateLimiter(10, 2)

	http.HandleFunc("/api/message", RateLimitMiddleware(limiter, handleMessage))
	http.HandleFunc("/api/health", handleHealth)

	fmt.Println("Rate Limiter Service running on http://localhost:8080")
	fmt.Println("Configuration: 10 requests burst, 2 tokens/second refill")
	log.Fatal(http.ListenAndServe(":8080", nil))
}

func handleMessage(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "Hello! Your request was processed successfully.\n")
}

func handleHealth(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "OK\n")
}

What does this code do?

  • NewRateLimiter(10, 2) creates a limiter that allows 10 requests as a burst, then refills at 2 tokens per second.
  • /api/message is wrapped with RateLimitMiddleware, so every request to this endpoint goes through the rate limiter before reaching handleMessage.
  • /api/health has no middleware. Health checks should never be rate limited because monitoring systems need them to always respond.
  • http.ListenAndServe(":8080", nil) starts the server on port 8080. The log.Fatal wrapper ensures the program exits with a clear error message if the port is already in use.
  • Save main.go.
  • Start the server by running this command in your terminal:
go run .

You should see two lines printed in your terminal:

Rate Limiter Service running on http://localhost:8080
Configuration: 10 requests burst, 2 tokens/second refill

Seeing a "port already in use" error?

Another process is using port 8080. Stop it first, or check if you have a previous go run still running in another terminal tab. Close that tab or press Ctrl+C in it to stop the process.

help me fix a port already in use error when running my Go server

✔️ Awesome, I've got everything!

Great. Keep that terminal running. You will need a second terminal for the next substep.

ⓧ I'd like to double check the full code

package main

import (
	"fmt"
	"log"
	"net/http"
)

func main() {
	limiter := NewRateLimiter(10, 2)

	http.HandleFunc("/api/message", RateLimitMiddleware(limiter, handleMessage))
	http.HandleFunc("/api/health", handleHealth)

	fmt.Println("Rate Limiter Service running on http://localhost:8080")
	fmt.Println("Configuration: 10 requests burst, 2 tokens/second refill")
	log.Fatal(http.ListenAndServe(":8080", nil))
}

func handleMessage(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "Hello! Your request was processed successfully.\n")
}

func handleHealth(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "OK\n")
}
Test the rate limiter with curl

Your server is running. Now you will bombard it with rapid requests to prove the rate limiter actually blocks traffic after the burst capacity is exhausted.

  • Open a new terminal tab or window. Your server must keep running in the first terminal.
  • Send 12 rapid requests to the rate-limited endpoint by running this command:
for i in $(seq 1 12); do curl -i http://localhost:8080/api/message; done

What should I see?

The first 10 requests succeed with HTTP/1.1 200 OK and show the X-RateLimit-Remaining header counting down from 9 to 0.

Requests 11 and 12 return HTTP/1.1 429 Too Many Requests with a Retry-After: 1 header. The body says Too Many Requests.

  • Wait 2 seconds, then run the same command again.

You should see a few requests succeed again. The bucket refills at 2 tokens per second, so after 2 seconds you have approximately 4 tokens available.

  • Test the health endpoint by running:
curl -i http://localhost:8080/api/health

This endpoint always returns OK with no rate limit headers, because it has no middleware wrapping it.

Getting "connection refused" errors?

Make sure your server is still running in the other terminal. If you accidentally closed it or pressed Ctrl+C, go back to that terminal and run go run . again.

help me debug connection refused when curling my Go server

Your rate limiter is live and throttling traffic. Next up, you will write a concurrency test and use Go's race detector to prove your mutex is doing its job.

Detect and Fix a Race Condition

Your rate limiter is serving HTTP traffic and throttling clients correctly. But how do you know it works safely under heavy concurrent load?

Concurrent code can hide bugs that only surface when multiple goroutines access the same memory at the exact same moment. Go ships with a built-in race detector that instruments your code at compile time and flags these unsafe accesses. In this step, you will write a concurrency test, prove it passes cleanly, then intentionally break your mutex to see exactly what a data race looks like.

In this step, get ready to:
  • Write a concurrency test that hammers your token bucket from 50 goroutines simultaneously.
  • Run Go's race detector and confirm your mutex protects shared state.
  • Intentionally break the mutex to observe a real data race report, then restore the fix.
Write the concurrency test file

Go test files live alongside your source code and must end in _test.go. Test functions must be named TestXxx where Xxx starts with a capital letter.

  • Create a new file called limiter_test.go in your ratelimiter folder and add the following code:
package main

import (
	"sync"
	"testing"
)

func TestTokenBucketConcurrency(t *testing.T) {
	bucket := NewTokenBucket(100, 10)

	var wg sync.WaitGroup
	numGoroutines := 50
	requestsPerGoroutine := 10

	for i := 0; i < numGoroutines; i++ {
		wg.Add(1)
		go func() {
			defer wg.Done()
			for j := 0; j < requestsPerGoroutine; j++ {
				bucket.Allow()
			}
		}()
	}

	wg.Wait()

	// If we get here without a race condition or panic, the mutex is working.
	t.Log("All goroutines completed without race conditions")
}

What does this test do?

This test creates a single TokenBucket with capacity 100 and refill rate 10, then launches 50 goroutines that each call Allow() 10 times. That is 500 concurrent calls hitting the same bucket.

sync.WaitGroup acts as a counter. Add(1) increments before each goroutine launches, Done() decrements when a goroutine finishes, and Wait() blocks until the counter hits zero.

You won't run this yet. The next chunk adds a second test for the multi-client scenario.

  • Add the following function below TestTokenBucketConcurrency in the same file:
func TestRateLimiterConcurrency(t *testing.T) {
	limiter := NewRateLimiter(50, 5)

	var wg sync.WaitGroup
	clients := []string{"client-a", "client-b", "client-c", "client-d", "client-e"}

	for _, clientID := range clients {
		for i := 0; i < 10; i++ {
			wg.Add(1)
			go func(id string) {
				defer wg.Done()
				limiter.Allow(id)
			}(clientID)
		}
	}

	wg.Wait()
	t.Log("All concurrent client requests completed without race conditions")
}

What does this second test cover?

This test creates 5 different clients and launches 10 goroutines per client, all hitting the same RateLimiter simultaneously. It exercises both the per-client map (protected by RateLimiter.mu) and the individual bucket mutexes at the same time.

  • Save limiter_test.go.

✔️ Awesome, I've got everything!

Great. Double check you have saved the file before moving on.

ⓧ I'd like to double check the full code

package main

import (
	"sync"
	"testing"
)

func TestTokenBucketConcurrency(t *testing.T) {
	bucket := NewTokenBucket(100, 10)

	var wg sync.WaitGroup
	numGoroutines := 50
	requestsPerGoroutine := 10

	for i := 0; i < numGoroutines; i++ {
		wg.Add(1)
		go func() {
			defer wg.Done()
			for j := 0; j < requestsPerGoroutine; j++ {
				bucket.Allow()
			}
		}()
	}

	wg.Wait()

	// If we get here without a race condition or panic, the mutex is working.
	t.Log("All goroutines completed without race conditions")
}

func TestRateLimiterConcurrency(t *testing.T) {
	limiter := NewRateLimiter(50, 5)

	var wg sync.WaitGroup
	clients := []string{"client-a", "client-b", "client-c", "client-d", "client-e"}

	for _, clientID := range clients {
		for i := 0; i < 10; i++ {
			wg.Add(1)
			go func(id string) {
				defer wg.Done()
				limiter.Allow(id)
			}(clientID)
		}
	}

	wg.Wait()
	t.Log("All concurrent client requests completed without race conditions")
}
Run the race detector

Go's race detector instruments every memory access at compile time. When two goroutines touch the same variable without synchronization, it flags the conflict immediately.

  • Make sure your HTTP server from Step 5 is stopped (press Ctrl+C in the terminal where it was running).
  • Run your tests with the race detector enabled by running this command:
go test -race -v

What do these flags do?

  • -race enables the race detector, which watches for two goroutines accessing the same memory concurrently where at least one is writing.
  • -v enables verbose output so you can see each test function's name and log messages.
  • Confirm you see PASS in the output with no WARNING: DATA RACE messages. You should also see the log lines from both test functions.

Seeing a compile error?

  • Make sure your file is named limiter_test.go (not limiter_tests.go or test_limiter.go). Go requires the _test.go suffix.
  • Confirm the file starts with package main to match the other files in your project.
  • Check that your HTTP server is not still running. Only one process can use the package at a time.

Help me debug my Go test file

Break the mutex to see a real data race

Seeing a passing test is good, but understanding what it protects you from is better. You will temporarily remove the mutex to trigger a real data race, see exactly what Go reports, then restore the fix.

  • In limiter.go, scroll to the Allow() method on TokenBucket.
  • Find these two lines at the top of the function body:
	tb.mu.Lock()
	defer tb.mu.Unlock()
  • Comment out both lines by adding // at the start of each, so they look like this:
	// tb.mu.Lock()
	// defer tb.mu.Unlock()
  • Save limiter.go.
  • Run the race detector again by running this command:
go test -race -v

This time the test will fail. You should see WARNING: DATA RACE in the output along with stack traces pointing to the exact lines where two goroutines accessed the same variable unsafely.

How to read the race report

The report shows two sections: one for the goroutine doing a write and one for the goroutine doing a concurrent read (or write). Each section includes the file name and line number where the access happened.

The race detector does not issue false positives. If it reports a race, there is a real bug. The exit code for a detected race is 66.

  • Now restore the mutex. In limiter.go, find the two commented-out lines:
	// tb.mu.Lock()
	// defer tb.mu.Unlock()
  • Remove the // from each line so they look like this again:
	tb.mu.Lock()
	defer tb.mu.Unlock()
  • Save limiter.go.
  • Run the race detector one final time to confirm the fix is back in place:
go test -race -v
  • Confirm you see PASS with no WARNING: DATA RACE messages.

Why this matters for production systems

Race conditions are some of the hardest bugs to find because they only appear under specific timing conditions. A rate limiter handling thousands of concurrent requests will hit these timing conditions constantly.

Go's race detector is based on ThreadSanitizer, the same tool used internally at Google. Running go test -race in CI pipelines is standard practice for any Go service that uses goroutines.

Still seeing a race after restoring the mutex?

  • Double check that both lines are uncommented with no leading //. Even a space before tb.mu.Lock() is fine, but the // must be completely gone.
  • Make sure you saved limiter.go before re-running the test. The race detector compiles the current file on disk.

Help me fix the data race in my rate limiter

Secret mission

Add Idle Client Cleanup

Real rate limiters serving millions of clients need to evict idle buckets or memory grows unbounded. Your mission is to add a background goroutine that periodically removes stale entries from the client map, preventing your service from leaking memory at scale.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your resources for future reference, pause them, or delete them entirely.

Resources you used:

  • The ratelimiter project folder on your Desktop (Go source files, design doc, test file).
  • Go 1.26.1 installed on your system.
  • VS Code Go extension (golang.Go).
  • Go module cache (downloaded dependencies stored locally).

Keep everything running

No action needed. Choose this if you plan to keep building on this project or want it as a portfolio reference.

  • Your ratelimiter folder stays on your Desktop.
  • Go stays installed and ready for your next project.
  • The VS Code Go extension remains active for future Go development.

Pause - I'll come back to this later

Since everything is local, there is nothing to shut down or pause. Your files sit quietly on disk using minimal space.

  • Optionally clear the Go module cache to free disk space by running:
  • Your project files and Go installation remain intact for when you return.
go clean -modcache

This removes downloaded dependencies. They will re-download automatically if you run go mod tidy later.

Delete - I don't want to use this again

Remove all project resources and optionally uninstall Go from your system.

  • Delete the project folder and clean the Go caches by running these commands:
go clean -cache
go clean -modcache
  • Delete the ratelimiter folder from your Desktop:

macOS

rm -rf ~/Desktop/ratelimiter

Windows

rmdir /s /q %USERPROFILE%\Desktop\ratelimiter
  • To uninstall Go from macOS, delete the Go directory and the PATH configuration file:
sudo rm -rf /usr/local/go
sudo rm -f /etc/paths.d/go
  • To uninstall Go from Windows, open Control Panel, go to Add/Remove Programs, select Go Programming Language, and click Uninstall.
  • To remove the VS Code Go extension, open the Extensions panel in VS Code, find the Go extension, right-click it, and select Uninstall.

Nice Work!

Nice Work!

You just built a production-pattern rate limiter service in Go from scratch. This is a classic system design interview question turned into working code, proving you can evaluate technology choices, implement distributed systems primitives, and debug concurrency issues.

You've learned how to:

  • Write a design decision document comparing rate limiting algorithms and justify the token bucket approach with documented trade-offs.
  • Implement the token bucket algorithm with per-client isolation and thread-safe concurrency using sync.Mutex.
  • Use Go's race detector to find and fix a data race, and wrap rate limiting in HTTP middleware with standard response headers.
  • Added a background cleanup goroutine that automatically evicts idle client buckets to prevent memory leaks at scale.

Ready to quiz yourself?