Course contentsShow
← System Design
Lesson 1793 of 1,91943. Real-World System: Rate Limiter ServiceFree lesson

Centralized vs Distributed Rate Limiting

Compare single-node rate limiting with distributed approaches and understand the tradeoffs in accuracy and latency.

Centralized vs Distributed Rate Limiting

What you'll learn: How rate limiting architecture changes when you move from a single server to multiple servers, and the tradeoffs involved.


The Core Problem

When rate limiting runs on one machine (centralized), tracking requests is straightforward—you maintain counters in local memory or a single Redis instance. Every request increments a counter, and the decision is 100% accurate.

But when your system scales to multiple servers (distributed), each server only sees its own traffic. A user might send 50 requests to Server A and 50 to Server B—both servers think the user is within a 75-request limit, but the user has actually exceeded it with 100 total requests.

Centralized Approach

All rate limiting decisions go through a single shared store (like Redis):

  • Accuracy: Near-perfect. All servers check the same counter.
  • Latency: Every request needs a network hop to the central store (~1-5ms added).
  • Bottleneck: The central store becomes a single point of failure and a throughput limit.

Analogy: All cashiers at different stores call headquarters to verify your account balance before each purchase.

Distributed Approach

Each server maintains local counters and periodically syncs:

  • Accuracy: Reduced. Brief windows exist where limits can be exceeded by ~(number_of_servers × local_limit).
  • Latency: Very low. Decision happens in-memory locally.
  • Scalability: Excellent. No central bottleneck.

Analogy: Each store tracks your purchases locally and syncs with headquarters every few minutes—you might temporarily overspend.

Hybrid Solutions

Many systems use local caching with periodic central validation, accepting ~5-10% inaccuracy for 95% latency reduction.

Key Takeaway: Centralized rate limiting offers precision but adds latency and a bottleneck; distributed approaches trade accuracy for speed and scalability—choose based on whether strict enforcement or low latency matters more for your use case.