High-Level Architecture: Edge vs Centralized
You'll learn: Where to place rate limiting logic—at the edge of your infrastructure or in a centralized service—and the tradeoffs each approach presents.
Core Concept
When designing a rate limiter service, you must decide where to enforce limits: at the edge (load balancers, API gateways, or CDN nodes close to users) or in a centralized service within each region.
Edge Rate Limiting
Deploying rate limiters at the edge means every incoming request hits the limiter before reaching your application servers. Think of it like security checkpoints at every entrance to a building.
Advantages:
- Lowest latency impact—decisions happen closest to the user
- Rejected requests never touch backend infrastructure
- Reduces bandwidth and load on internal services
Challenges:
- Coordination across edge nodes is harder (distributed state)
- Less accurate counting when traffic splits across multiple edge locations
- Configuration updates must propagate to many endpoints
Centralized Rate Limiting
A centralized service sits within each region/data center, with all application servers calling it before processing requests. This is like having one central security office that all departments must check with.
Advantages:
- Single source of truth for counts within a region
- More accurate limit enforcement
- Easier to update rules and monitor globally
Challenges:
- Extra network hop adds latency (2-5ms typically)
- Becomes a critical dependency—downtime blocks all traffic
- Requires high availability and horizontal scaling
Hybrid Approach
Many systems use both: coarse limits at the edge (block obvious abuse) and fine-grained limits centrally (accurate business logic enforcement).
Key Takeaway: Edge rate limiting optimizes for speed and infrastructure protection; centralized rate limiting optimizes for accuracy and consistency. Choose based on whether you prioritize latency or precision.