Course contentsShow
← System Design
Lesson 1806 of 1,91943. Real-World System: Rate Limiter ServiceFree lesson

Rate Limiting with Consistent Hashing

Partition rate limit state across multiple Redis nodes using consistent hashing for horizontal scalability and fault tolerance.

Rate Limiting with Consistent Hashing

What You'll Learn

You'll learn how to use consistent hashing to distribute rate limit state across multiple Redis nodes, enabling horizontal scalability and fault tolerance in a distributed rate limiter.

The Problem: Single Redis Bottleneck

When your rate limiter uses a single Redis instance (as we've done so far), you hit a ceiling: memory limits, CPU constraints, and single-point-of-failure risks. To scale horizontally, you need to partition your rate limit counters across multiple Redis nodes.

How Consistent Hashing Solves It

Consistent hashing maps each rate limit key (like user:12345:api_calls) to a specific Redis node using a hash function. Instead of random partitioning, consistent hashing minimizes data movement when nodes are added or removed.

Analogy: Imagine a circular clock face where each Redis node occupies several positions. Keys are also mapped to positions on the clock. A key is stored on the nearest node clockwise from its position. Adding/removing nodes only shifts a small fraction of keys.

Implementation Flow

  1. Hash the key: Apply a consistent hash function to your rate limit key (e.g., hash("user:12345") % 360)
  2. Find the node: Look up which Redis node owns that hash range
  3. Route the request: Direct INCR, Lua scripts, or sliding window operations to that specific node
Key → hash("user:12345") → 127 → Node B

Benefits

  • Horizontal scalability: Add more Redis nodes as traffic grows
  • Fault tolerance: If one node fails, only its keys are affected (not the entire limiter)
  • Even load distribution: Hash function spreads keys uniformly

Trade-offs

You sacrifice global accuracy slightly—each node tracks its partition independently. For strict limits, add a coordination layer or use virtual nodes for better distribution.

Key Takeaway: Consistent hashing partitions rate limit state across Redis nodes by mapping keys to nodes via hash ranges, enabling horizontal scaling while minimizing reshuffling when the cluster topology changes.