Why Distributed Rate Limiting Is Hard
You'll learn: Why coordinating rate limits across multiple servers is fundamentally difficult without a single source of truth.
The Core Problem
When you have one server, rate limiting is straightforward—you keep a counter in memory and increment it with each request. But when you scale to multiple servers, each node only sees its own requests. If your limit is 100 requests per minute and you have 5 servers, each might allow 100 requests, giving the user 500 total—5x your intended limit!
Why There's No Easy Fix
The challenge boils down to three conflicting requirements:
1. Accuracy
To enforce a true distributed limit, all nodes need to know the global count in real-time. But checking a central store (like Redis) on every request adds latency and creates a bottleneck.
2. Performance
Rate limiting must be fast—ideally microseconds. Waiting for network calls to synchronize state across nodes defeats the purpose of horizontal scaling.
3. Consistency
Different nodes might have slightly different views of the counter due to replication lag, network delays, or clock skew. A user might hit Server A 50 times and Server B 60 times, both thinking they're under the 100-request limit.
The Analogy
Imagine five toll booths counting cars for a bridge with a 100-car limit. If each booth only knows its own count and can't instantly communicate with the others, they'll collectively let 400+ cars through before anyone realizes the limit was exceeded.
Key Takeaway: Distributed rate limiting is hard because achieving accuracy requires coordination, but coordination costs performance—forcing you to accept either over-limiting, under-limiting, or increased latency as tradeoffs.