Capacity Estimation for Rate Limiter
You'll learn how to calculate the infrastructure requirements—QPS, memory, and bandwidth—needed to support a rate limiter at scale.
Why Capacity Estimation Matters
Before building a rate limiter service, you need to know if your design can handle the expected load. Just like estimating how many cashiers a supermarket needs based on customer traffic, you must calculate three core metrics: queries per second (QPS), memory for storing counters, and network bandwidth.
Calculating QPS
Start with your expected request volume. If your system handles 10 million API requests per day and you apply rate limiting to all of them, that's:
10,000,000 requests / 86,400 seconds ≈ 116 QPS
But traffic isn't uniform—plan for peak load (often 3-5x average). So you might need to support ~500 QPS for rate limit checks.
Memory for Counters
Each rate limit rule needs a counter (or sliding window state). If you track:
- 1 million unique keys (user IDs or API tokens)
- 50 bytes per counter (timestamp + count + metadata)
That's 50 MB per rate limit rule. With multiple rules (per-user, per-IP, per-endpoint), memory adds up quickly. Cache eviction policies (LRU) help manage this.
Network Bandwidth
Each rate limit check involves small request/response payloads (~100-200 bytes). At 500 QPS:
500 requests/sec × 200 bytes ≈ 100 KB/sec ≈ 0.8 Mbps
This is modest, but multiplied across multiple data centers or edge locations, it matters.
Key Takeaway:
Capacity estimation for rate limiters requires calculating peak QPS to size compute resources, estimating memory based on unique keys and counter storage, and accounting for network bandwidth—ensuring your service can scale without becoming a bottleneck itself.