Rate Limiting with Consistent Hashing
What You'll Learn
You'll learn how to use consistent hashing to distribute rate limit state across multiple Redis nodes, enabling horizontal scalability and fault tolerance in a distributed rate limiter.
The Problem: Single Redis Bottleneck
When your rate limiter uses a single Redis instance (as we've done so far), you hit a ceiling: memory limits, CPU constraints, and single-point-of-failure risks. To scale horizontally, you need to partition your rate limit counters across multiple Redis nodes.
How Consistent Hashing Solves It
Consistent hashing maps each rate limit key (like user:12345:api_calls) to a specific Redis node using a hash function. Instead of random partitioning, consistent hashing minimizes data movement when nodes are added or removed.
Analogy: Imagine a circular clock face where each Redis node occupies several positions. Keys are also mapped to positions on the clock. A key is stored on the nearest node clockwise from its position. Adding/removing nodes only shifts a small fraction of keys.
Implementation Flow
- Hash the key: Apply a consistent hash function to your rate limit key (e.g.,
hash("user:12345") % 360) - Find the node: Look up which Redis node owns that hash range
- Route the request: Direct
INCR, Lua scripts, or sliding window operations to that specific node
Key → hash("user:12345") → 127 → Node B
Benefits
- Horizontal scalability: Add more Redis nodes as traffic grows
- Fault tolerance: If one node fails, only its keys are affected (not the entire limiter)
- Even load distribution: Hash function spreads keys uniformly
Trade-offs
You sacrifice global accuracy slightly—each node tracks its partition independently. For strict limits, add a coordination layer or use virtual nodes for better distribution.
Key Takeaway: Consistent hashing partitions rate limit state across Redis nodes by mapping keys to nodes via hash ranges, enabling horizontal scaling while minimizing reshuffling when the cluster topology changes.