Design a Distributed API Rate Limiter

Design a Redis-backed API rate limiter that stays accurate as services scale.

Introduction

30 Second Summary

A sudden rush of traffic can overwhelm a service that usually works perfectly. The service needs a fair way to decide which requests continue before its critical resources buckle.

In this project, you will use draw.io to design a per-user API rate limiter that starts on one server. You will expose its horizontal scaling bug before evolving it into a Redis-backed design with atomic decisions.

What You'll Build

Picture presenting a polished diagram where one user's requests cross multiple API instances yet still reach one consistent Allow or Deny decision.

By the end of this project, you'll have:

A visible single-instance request trace that lets you point from Client through API Instance and Local Counter to Allow or Deny.
A scaling failure demonstration that shows two local counters admitting more combined requests than one shared quota allows.
An exported interview diagram showing both API instances using one shared Redis state. Your decision notes defend the sliding-window counter, atomic updates, and your outage policy.
Secret Mission: Extend the limiter across Region A and Region B. You will defend centralized global enforcement or region-local counters by explaining the consistency-versus-latency trade-off.

Are there any prerequisites?

The guide sets up draw.io from scratch. No Redis deployment or prior system design experience is required.

Before We Start

Before drawing, lock in the interview story that guides every design choice. You are designing a per-user API rate limiter that immediately allows or denies requests to protect a downstream resource you name.

Set Up Your Interview Canvas

Your interview story now defines a per-user API rate limiter. A prepared canvas removes diagram-tool friction from the next design decisions.

In this step, you will open draw.io without creating an account. You will prove the canvas works with a simple three-shape flow before saving it locally.

In this step, get ready to:

Access a blank draw.io canvas without creating an account.
Build a connected temporary flow to verify the canvas mechanics.
Save the editable diagram as a local file.

Open a blank draw.io canvas

The browser editor gives you the diagramming space needed for the interview exercise. It works without an account.

Click the address bar in your web browser.
Paste https://app.diagrams.net/ into the address bar.
Press Enter to load the editor.
Select the local-device storage option if draw.io asks where to store files.
Start a blank diagram from the creation screen.

You should see an empty canvas ready for shapes. You can continue without signing in.

Add labels and connectors

A system-design diagram uses labeled shapes to represent components. Connectors show request flow between those components.

Drag a rectangle from the left shape library onto the left side of the canvas.
Enter Client in the left shape after double-clicking its text area.
Drag a second rectangle into the center of the canvas.
Enter API in the center shape after double-clicking its text area.
Drag a third rectangle onto the right side of the canvas.
Enter Counter in the right shape after double-clicking its text area.

What Does This Tool Check Prove?

The three components prove you can position shapes. Their labels prove text editing works.

Connecting the components proves you can show a request moving through a system.

Move your pointer over the right edge of Client to reveal a connector handle.
Drag the connector handle from Client to API.
Move your pointer over the right edge of API to reveal a connector handle.
Drag the connector handle from API to Counter.

You should see Client on the left. API sits in the center. Counter sits on the right with two visible connectors completing the flow.

Shapes or Connectors Not Behaving?

Restore a moved shape to its left-to-right position before reconnecting it.
Double-click inside any unlabeled shape before re-entering its label.

Help me fix my temporary draw.io shapes or connectors.

Save the editable diagram

An editable .drawio file preserves the canvas structure for later design changes.

Before you save, picture what you expect the canvas to show after the local file is created.

Click File in the top menu.
Select Save As.
Enter distributed-rate-limiter.drawio as the file name.
Choose your Desktop as the local save location.
Confirm the save in your browser's file dialog.

The file distributed-rate-limiter.drawio is now saved locally with the three-shape flow still visible. That's your canvas ready for the real rate-limiter design.

File Missing After the Save?

Return to the File menu if the save flow closes early.
Select Save As to restart the local save.
Re-enter distributed-rate-limiter.drawio as the file name.

Help me save my draw.io diagram locally.

Your interview canvas is ready. Next, you will turn this temporary flow into the smallest working rate-limiter design.

Draw the Local Rate Limiter

Your temporary flow proved that the draw.io canvas can handle shapes, labels, and connectors. You can now turn that test flow into the first version of your interview design.

The smallest useful rate limiter identifies one caller and returns an immediate decision. A local counter creates a visible single-instance baseline for the scaling test that follows.

In this step, get ready to:

Define the requirements for your per-user rate limiter.
Build a single-instance request path with visible outcomes.
Trace one request through the complete design.

Write the design brief

A design brief keeps your answer focused on request protection. It also prevents billing or analytics from distracting from the quota decision.

Drag a wide rectangle from the shape palette on the left to the space above the temporary flow.
Double-click the new rectangle.
Type Design Brief on the first line.
Add Identity: User ID on the next line.
Add Quota: Small test quota on the next line.
Add Decision: Immediate Allow or Deny on the next line.
Add a Protects: line that names the downstream resource from your interview story.
Add Out of scope: Billing and analytics on the final line.

You should see a titled design brief above the temporary request flow.

Why define the scope first?

The brief gives every shape a purpose. Each request belongs to one user and receives one immediate decision.

The scope line keeps your interview answer centered on protecting the downstream resource. Billing and analytics can remain separate systems.

Replace the temporary flow

Each shape now represents one responsibility in the request path. The client sends a request before the API checks its local count.

Double-click the temporary API label.
Replace the label with API Instance.

The middle shape now represents the server handling the request.

Double-click the temporary Counter label.
Replace the label with Local Counter.

Your original path should now read from Client through API Instance to Local Counter.

Drag a rectangle from the shape palette to the upper-right side of Local Counter.
Label the new rectangle Allow by double-clicking its text.
Drag another rectangle from the shape palette to the lower-right side of Local Counter.
Label the second rectangle Deny by double-clicking its text.

You should see two distinct outcomes to the right of the local counter.

Connect and trace the decisions

The local counter holds the state needed for one API instance to choose an outcome. Connector labels make the reason for each outcome visible.

Confirm that the existing connector runs from Client to API Instance.
Confirm that the next connector runs from API Instance to Local Counter.
Draw a connector from Local Counter to Allow.
Label the connector Under quota.
Draw a connector from Local Counter to Deny.
Label the connector Quota reached.
Drag a note shape beside Local Counter.
Enter Stores the current request count for one user within the active time window. in the note.

What does the local counter represent?

The Local Counter stores one user's current request count during the active time window. The API uses that count to select the allow or deny path.

This creates a complete decision loop for one API instance. Every visible outcome connects back to the same local state.

Your editable diagram now contains the complete design. Save this state before rehearsing the request trace.

Open the save flow by pressing Cmd+Shift+S.
Set the file name to distributed-rate-limiter.drawio.
Complete the local save.

You should now have an updated editable copy of distributed-rate-limiter.drawio on your Mac.

Before you trace the flow, which outcome do you expect while the user's count remains below the quota?

Trace one request from Client to API Instance.
Continue the request from API Instance to Local Counter.
Follow the Under quota path to Allow.
Follow the Quota reached path to Deny.

That's your first working rate-limiter design. Every request reaches a visible allow-or-deny outcome through the local counter.

Is the request path hard to follow?

Recreate any connector that does not stay attached when you move its destination shape.
Move overlapping connector labels until both outcomes remain readable.
Place the counter note beside the shape if it covers either decision path.

Ask for help with making the request path clear in draw.io.

Your single-instance baseline is ready. Next, you'll introduce horizontal scaling and test whether local state can still enforce one quota.

Expose the Scaling Failure

Your single-instance rate limiter now makes a complete allow-or-deny decision for each request.

Horizontal scaling adds more API instances behind a load balancer. The same user can reach different instances with separate counters.

You will expose that problem directly in draw.io. The request trace will prove that separate local counters cannot enforce one shared quota.

In this step, get ready to:

Expand the request path behind a load balancer.
Trace one user across two independent API instances.
Mark the duplicated counters as the source of the scaling failure.

Scale the request path

The scaled design needs two API instances that can receive requests independently. Each instance keeps its own local view of the user's request count.

Select the existing API Instance shape on the canvas.
Double-click its label to rename it API Instance A.
Press Cmd+D to duplicate the selected shape.
Drag the duplicate below API Instance A.
Double-click the duplicate to rename it API Instance B.
Drag a rectangle from the left shape library into the space between Client and the two API instances.
Label the new rectangle Load Balancer.
Reconnect the existing client path so it ends at Load Balancer.

What Changed in the Request Path?

The client no longer selects an API instance directly. The load balancer can send each incoming request to either instance.

That distribution creates the conditions for the quota bug. Each instance only sees the requests routed to itself.

Draw a connector from Load Balancer to API Instance A.
Draw a connector from Load Balancer to API Instance B.
Select the existing Local Counter shape.
Press Cmd+D to duplicate the counter.
Drag the duplicated counter beside API Instance B.
Connect API Instance B to its duplicated Local Counter.
Connect the duplicated counter to Allow.
Connect the duplicated counter to Deny.

You should now see two request branches. Each API instance has its own local counter plus a visible allow-or-deny path.

Do the Branches Look Tangled?

Drag the two API instances farther apart if their connectors overlap. Place each local counter beside the API instance that owns it.

Check that the client reaches both instances only through the load balancer.

Help me untangle the two request paths in my draw.io rate-limiter diagram.

Trace one user across both instances

Before you draw the trace, do you think two independent counters will still enforce the user's shared quota?

Use the small test quota from your design brief as the shared limit for this trace.
Draw one more request than that quota from Client to Load Balancer.
Label every traced request Same user.
Route the first traced request to API Instance A.
Route the second traced request to API Instance B.
Continue alternating the remaining requests between the two instances.
Add a running request count beside each Local Counter.
Mark each traced request Allow according to the counter for its assigned instance.

Both local counters remain at or below the stated quota. Their combined admitted requests exceed that quota by one.

Why Does the Quota Break?

Each API instance sees only part of the user's traffic. Its local count therefore looks valid when considered alone.

The intended quota applies to the user's combined traffic. Separate counters accidentally create one quota per instance.

Mark the failure source

The failed trace is easier to explain when the diagram names the broken assumption. A visible annotation turns the canvas into an interview-ready failure demonstration.

Add a note beneath the two local counters.
Enter Per-instance state creates multiple independent quotas in the note.
Set the note text to red using the format controls.
Draw an unfilled ellipse around the first Local Counter.
Press Cmd+D to duplicate the selected ellipse.
Position the duplicated ellipse around the second Local Counter.
Press Cmd+S to save your changes to distributed-rate-limiter.drawio.

Before your final trace, do you expect either API instance to know the other instance's local count?

Trace the same user's requests from Client through Load Balancer.
Follow the alternating requests assigned to API Instance A.
Follow the remaining requests assigned to API Instance B.
Compare the two independent counter totals with the shared quota.
Confirm that every request path ends at Allow or Deny.

You should see both local counters accepting their assigned requests. The combined admitted total exceeds the intended shared quota.

✔️ My Scaling Failure Is Visible

You have exposed the bug. Your saved diagram now proves why per-instance counters fail after horizontal scaling.

ⓧ I'd Like to Double Check the Diagram

Compare your canvas with this final-state checklist.

Confirm the request path starts with Client followed by Load Balancer.
Confirm the load balancer connects to API Instance A plus API Instance B.
Confirm each API instance has an independent Local Counter plus an allow-or-deny path.
Confirm the alternating trace uses one user across both instances.
Confirm the combined admitted requests exceed the shared quota.
Confirm the red note reads Per-instance state creates multiple independent quotas.
Confirm both duplicated counters are circled as the failure source.

You have proved that local state cannot protect one shared quota after scaling. Next, you will replace the duplicated state with one shared quota view and choose the algorithm behind it.

Centralize State and Select an Algorithm

Your last diagram proved that horizontal scaling splits one user's quota across separate counters. Both API instances can still approve requests from their own state.

Now the design needs shared state that every API instance can query. A single Redis component gives every instance one quota view.

The algorithm determines how accurately the limiter handles window boundaries.

Memory use matters because every user creates stored rate-limit state. Burst behavior matters because some APIs should absorb short spikes.

In this step, get ready to:

Document the trade-offs among three rate-limiting algorithms.
Select an algorithm for the general-purpose API.
Replace the duplicated local counters with one shared quota store.

Compare the rate-limiting algorithms

Each algorithm answers the same quota question with a different amount of stored state. The comparison gives you a concise way to defend your choice during an interview.

How the options differ

Fixed window uses one string key per window. Its reset can admit a burst at the boundary.
Token bucket uses one hash key. It provides exact accounting with controlled bursts.
Sliding-window counter uses two string keys. It smooths window boundaries with near-exact accuracy.

Create a comparison note in open canvas space by dragging a note-shaped rectangle from the shape palette on the left.
Double-click the comparison note to edit its text.

You should see a text cursor inside the new note.

Enter the first row as fixed window is simple but permits boundary bursts.
Enter the second row as token bucket allows controlled bursts.

Your note now captures the simplest option plus the option designed for controlled bursts.

Enter the third row as sliding-window counter uses two counters to smooth boundaries with near-exact accuracy.

You should see three distinct rows covering simplicity, burst handling, and boundary accuracy.

Select the sliding-window counter

The sliding-window counter is the recommended starting point for most APIs. Its weighted estimate reduces the sharp reset behavior created by fixed windows.

Create a smaller decision note directly below the comparison note.
Enter Sliding Window Counter as the selected algorithm.

Your algorithm choice should now stand apart from the comparison rows.

Add the weighted estimate to the decision note by entering previous_count * (1 - elapsed) + current_count.

You should now see the selected algorithm with its decision formula directly beneath it.

How the weighted estimate works

previous_count holds the request count from the previous window.
elapsed represents the fraction of the current window that has passed.
current_count holds the request count from the current window.
The previous count contributes less as the current window advances.

Replace local counters with shared Redis state

A shared store removes the duplicated quota views that caused the scaling failure. Both API instances now ask the same component for each decision.

Select the Local Counter beneath API Instance A.
Press Delete.

The first API instance should no longer have a private counter beneath it.

Select the Local Counter beneath API Instance B.
Press Delete.

The duplicated Local Counter components should now be gone from the canvas.

Create one data-store shape below the two API instances.
Label the new shape Redis.

You should see one Redis component positioned where both API instances can reach it.

Draw a connector from API Instance A to Redis.
Draw a connector from API Instance B to Redis.

Both API paths should now converge on the same Redis component.

Create a decision note beside the Redis component.

The new note gives you space to state what Redis contributes to the architecture.

Enter single per-user quota view on the first line.
Enter allow-or-deny decisions on the second line.

Your central component now states both the shared state it holds and the decision it returns.

Before you trace the finished flow, where do you expect both API instances to look for one user's quota?

Trace one request from API Instance A to Redis.
Trace another request from API Instance B to Redis.
Check the decision note for Sliding Window Counter.
Check the decision note for previous_count * (1 - elapsed) + current_count.
Use the save action from earlier to update distributed-rate-limiter.drawio.

You should see both request paths converge on one Redis component. You should also see the selected algorithm with its weighted estimate.

That scaling bug is now resolved. Both API instances enforce one per-user quota through shared Redis state.

Connectors missing Redis?

Drag any loose connector endpoint until it attaches to the edge of the Redis shape.
Move the Redis shape lower on the canvas if its connectors overlap the allow-or-deny paths.

Help me fix the shared Redis connectors in my distributed rate-limiter diagram.

✔️ My diagram matches

Your diagram has one shared quota path plus a documented algorithm decision.

ⓧ I'd like to double check

The Client connects to the Load Balancer. The Load Balancer connects to API Instance A plus API Instance B.

Both API instances connect to one shared Redis component. No Local Counter component remains.

The comparison note contains fixed window is simple but permits boundary bursts. It also contains token bucket allows controlled bursts.

The final comparison row reads sliding-window counter uses two counters to smooth boundaries with near-exact accuracy.

The selected algorithm is Sliding Window Counter. Its weighted estimate is previous_count * (1 - elapsed) + current_count.

Next, you will protect the shared decision from concurrent requests. You will also choose the limiter's Redis outage behavior.

Make Decisions Atomic and Plan for Failure

Your draw.io diagram now routes both API instances through one shared Redis quota view. Its Sliding Window Counter gives you one cross-instance estimate.

Shared storage still leaves a concurrency gap. Two requests can read the same stale count before either increment is recorded.

An outage can also remove that shared decision point. Your architecture needs a policy that chooses availability or downstream protection.

In this step, get ready to:

Demonstrate how separate state changes race under concurrent requests.
Collapse the decision into one atomic Redis operation.
Finalize an outage-aware diagram for export.

Expose the race condition

A race condition appears when overlapping requests depend on timing. A short timeline lets you test whether separate mutations preserve one quota.

Add a temporary callout beside Redis from the shape library on the left.
Enter Current estimate: one request below quota on the first line.

Your temporary callout now gives both requests one shared starting count.

Enter read, decide, and increment on the next three lines.
Draw two request connectors entering the read line at the same point.

Before you trace them, predict whether both requests can reach Allow from this starting count.

Trace Request A through read, decide, and increment.
Trace Request B through the same sequence before Request A changes the shared count.

You will see that both requests can choose Allow from the same stale count. Together they can exceed the intended quota.

Add a permanent problem callout beside the temporary trace.
Enter Separate read, decide, and increment actions can allow concurrent requests to use the same stale count.

The concurrency failure is now visible on the main interview path.

This Shortfall Is Expected

The shared store fixed the per-instance quota problem. The timeline proves that separate mutations can still interleave under concurrency.

Make the Redis decision atomic

Atomicity prevents another request from interrupting the decision halfway through. A Lua script executed through EVAL groups the read, decision, and update into one operation.

Delete the temporary callout that lists read, decide, and increment.
Add one process box between the API instances and Redis.

The temporary operation sequence is gone. One decision box now owns that part of the path.

Enter Atomic Lua script (EVAL) inside the new box.
Reroute the connector from API Instance A so it ends at the new box.

API Instance A now reaches the atomic decision box before Redis.

Reroute the connector from API Instance B so it ends at the new box.
Draw one connector from Atomic Lua script (EVAL) to Redis.

Both API instances now converge on one operation before reaching shared state.

Add the annotation Single read-decide-update operation beside the atomic box.

Before you trace the updated path, predict whether another request can enter midway through this decision.

Trace one request from either API instance through Atomic Lua script (EVAL) to Redis.

You will see one read-decide-update boundary. Concurrent requests cannot interleave state changes inside that boundary.

How Does EVAL Protect the Decision?

Redis executes the EVAL script as one atomic operation. The count check and update complete before another request can change the same state.

Does the Path Still Fork?

Remove any connector that still runs directly from an API instance to Redis.
Confirm both API connectors terminate at Atomic Lua script (EVAL).

Help me reconnect the atomic Redis decision path.

Choose an outage policy and export

An outage policy defines what your limiter does when Redis cannot return a decision. Your choice depends on which failure is safer for the downstream resource you named earlier.

What Do the Two Policies Mean?

A fail open policy allows requests when Redis is unavailable. This preserves availability while the quota cannot be enforced.

A fail closed policy denies requests when Redis is unavailable. This protects the downstream resource while legitimate traffic can be blocked.

Review the downstream resource named in your interview story.
Choose the failure mode that creates the safer consequence for that resource.
Add an outage decision note beside Redis.
Enter Redis outage policy: fail open or Redis outage policy: fail closed in the note.

Your diagram now states what happens when the shared store is unavailable.

Add one sentence that justifies the policy for your named downstream resource.
Read the completed outage note aloud as part of your interview explanation.

Your explanation should connect the selected policy to one concrete consequence for the downstream resource.

The .drawio file keeps the architecture editable. Your PNG export creates a shareable snapshot of the finished design.

Press Cmd+Shift+S to open the save flow.

The save flow opens for the editable .drawio file.

Keep the filename set to distributed-rate-limiter.drawio.
Complete the local save.

Your editable diagram is now saved with the atomic path and outage policy.

Select File from the top menu.
Select Export as.

The available export formats are now visible.

Select PNG.
Complete the export using the default options.

You should now have an exported file named distributed-rate-limiter.png.

Before you inspect the PNG, predict whether every request reaches an allow-or-deny outcome without crossing a disconnected shape.

Preview the exported distributed-rate-limiter.png file.
Trace the branch through API Instance A from the client to an allow-or-deny outcome.
Trace the branch through API Instance B from the client to an allow-or-deny outcome.
Check that every architecture shape touches the request or decision flow.

Both API branches should reach Atomic Lua script (EVAL) before shared Redis. Every architecture component should connect to the request or decision flow.

The race-condition callout should remain visible. The algorithm comparison should still show Sliding Window Counter as selected.

You should also be able to read previous_count * (1 - elapsed) + current_count. Your outage note should name one policy with a one-sentence justification.

Is Something Missing From the PNG?

Return to the open draw.io canvas.
Reconnect any isolated architecture shape.
Save distributed-rate-limiter.drawio again.
Export a fresh distributed-rate-limiter.png.

Help me find the missing or disconnected part of my exported rate-limiter diagram.

That completes the single-region design. Your diagram now defends concurrent decisions plus Redis outages.

Secret mission

Take the Limiter Multi-Region

A second region forces your limiter to balance global quota accuracy against local response time. Extend the diagram across two regions. Then defend the consequence of your chosen enforcement strategy.

Clean Up Your Resources

Everything in this project lives in local files with no ongoing costs. Choose a cleanup option below.

Resources you used:

The local distributed-rate-limiter.drawio editable diagram created in the draw.io Online Editor.
The local distributed-rate-limiter.png image showing the completed multi-region design.

Keep everything running

No action is required. Choose this option for future interview practice.

Keep distributed-rate-limiter.drawio as the editable source for future changes.
Keep distributed-rate-limiter.png as the shareable version of your multi-region design.

Pause - I'll come back to this later

No cloud service or local process needs to be paused. Your saved files preserve the design while the editor is closed.

Close the existing draw.io Online Editor browser tab.
Keep distributed-rate-limiter.drawio in its current saved location.
Keep distributed-rate-limiter.png in its current saved location.

Delete - I don't want to use this again

Deleting these files becomes permanent when the Bin is emptied. Until then, you can restore them.

Close the existing draw.io Online Editor browser tab.
In Finder, locate distributed-rate-limiter.drawio in the folder where you saved it.
Move distributed-rate-limiter.drawio to the Bin.
In Finder, locate distributed-rate-limiter.png in the folder where you saved it.
Move distributed-rate-limiter.png to the Bin.
Review the Bin for unrelated files you still need.
Empty the Bin.
Search Finder for each filename after emptying the Bin.

You should see no matches for distributed-rate-limiter.drawio or distributed-rate-limiter.png.

Nice Work!

You turned a deliberately fragile local-counter design into an interview-ready distributed API rate limiter. Your exported distributed-rate-limiter.png now captures the single-region fix plus your defended multi-region strategy.

You learned how to:

Built a visible allow-or-deny request path for a per-user quota. Demonstrated how horizontal scaling breaks enforcement when each API instance owns a local counter.
Compared rate-limiting algorithms for accuracy. Evaluated their memory use. Evaluated their burst behavior. Selected the Sliding Window Counter for a general-purpose API.
Centralized quota state in Redis so every API instance uses one shared view. Created an atomic decision path with EVAL. Recorded an outage policy for the protected downstream resource.
Secret Mission: Extended the design across Region A and Region B. Chose a multi-region enforcement strategy. Defended its consistency-versus-latency trade-off.

Ready to quiz yourself?