Course contentsShow
← System Design
Lesson 885 of 1,91923. API Gateway PatternFree lesson

Rate Limiting and Throttling

Applying per-client or per-endpoint rate limits at the gateway to protect backend services.

Rate Limiting and Throttling

You'll learn how API gateways protect backend services by controlling request rates on a per-client or per-endpoint basis.

The Core Idea

Rate limiting and throttling are protective mechanisms implemented at the API gateway to prevent any single client or endpoint from overwhelming your backend services. Think of it like a bouncer at a popular club: they control how many people enter per minute to prevent overcrowding and ensure everyone inside has a decent experience.

Rate limiting sets hard caps on requests (e.g., "1000 requests per hour per API key"), while throttling may slow down or delay requests that exceed thresholds rather than rejecting them outright.

Why at the Gateway?

Since the gateway is the single entry point (as you've learned), it's the ideal place to enforce these policies uniformly across all backend services. The gateway can track request counts per client identifier (API key, IP address, user token) and per endpoint, rejecting or delaying excess requests before they reach your microservices.

Common Strategies

  • Per-client limits: Each API key gets 5000 requests/day
  • Per-endpoint limits: The expensive /search endpoint allows 100 requests/minute, while /health is unlimited
  • Burst allowances: Allow short spikes above the baseline rate
  • Priority tiers: Premium clients get higher limits than free-tier users

This protects against both malicious attacks (DoS attempts) and accidental overuse (buggy client loops), ensuring fair resource allocation and predictable backend load.

Key Takeaway:
Rate limiting and throttling at the API gateway act as your first line of defense, controlling request flow per-client and per-endpoint to prevent backend overload and ensure equitable service access.