Centralized vs Distributed Rate Limiting
What you'll learn: How rate limiting architecture changes when you move from a single server to multiple servers, and the tradeoffs involved.
The Core Problem
When rate limiting runs on one machine (centralized), tracking requests is straightforward—you maintain counters in local memory or a single Redis instance. Every request increments a counter, and the decision is 100% accurate.
But when your system scales to multiple servers (distributed), each server only sees its own traffic. A user might send 50 requests to Server A and 50 to Server B—both servers think the user is within a 75-request limit, but the user has actually exceeded it with 100 total requests.
Centralized Approach
All rate limiting decisions go through a single shared store (like Redis):
- Accuracy: Near-perfect. All servers check the same counter.
- Latency: Every request needs a network hop to the central store (~1-5ms added).
- Bottleneck: The central store becomes a single point of failure and a throughput limit.
Analogy: All cashiers at different stores call headquarters to verify your account balance before each purchase.
Distributed Approach
Each server maintains local counters and periodically syncs:
- Accuracy: Reduced. Brief windows exist where limits can be exceeded by ~(number_of_servers × local_limit).
- Latency: Very low. Decision happens in-memory locally.
- Scalability: Excellent. No central bottleneck.
Analogy: Each store tracks your purchases locally and syncs with headquarters every few minutes—you might temporarily overspend.
Hybrid Solutions
Many systems use local caching with periodic central validation, accepting ~5-10% inaccuracy for 95% latency reduction.
Key Takeaway: Centralized rate limiting offers precision but adds latency and a bottleneck; distributed approaches trade accuracy for speed and scalability—choose based on whether strict enforcement or low latency matters more for your use case.