Course contentsShow
← System Design
Lesson 978 of 1,91925. Rate Limiting and ThrottlingFree lesson

Why Distributed Rate Limiting Is Hard

Understanding the challenges of coordinating rate limits across multiple nodes without a single source of truth.

Why Distributed Rate Limiting Is Hard

You'll learn: Why coordinating rate limits across multiple servers is fundamentally difficult without a single source of truth.

The Core Problem

When you have one server, rate limiting is straightforward—you keep a counter in memory and increment it with each request. But when you scale to multiple servers, each node only sees its own requests. If your limit is 100 requests per minute and you have 5 servers, each might allow 100 requests, giving the user 500 total—5x your intended limit!

Why There's No Easy Fix

The challenge boils down to three conflicting requirements:

1. Accuracy

To enforce a true distributed limit, all nodes need to know the global count in real-time. But checking a central store (like Redis) on every request adds latency and creates a bottleneck.

2. Performance

Rate limiting must be fast—ideally microseconds. Waiting for network calls to synchronize state across nodes defeats the purpose of horizontal scaling.

3. Consistency

Different nodes might have slightly different views of the counter due to replication lag, network delays, or clock skew. A user might hit Server A 50 times and Server B 60 times, both thinking they're under the 100-request limit.

The Analogy

Imagine five toll booths counting cars for a bridge with a 100-car limit. If each booth only knows its own count and can't instantly communicate with the others, they'll collectively let 400+ cars through before anyone realizes the limit was exceeded.

Key Takeaway: Distributed rate limiting is hard because achieving accuracy requires coordination, but coordination costs performance—forcing you to accept either over-limiting, under-limiting, or increased latency as tradeoffs.