Course contentsShow
← System Design
Lesson 972 of 1,91925. Rate Limiting and ThrottlingFree lesson

Adaptive Rate Limiting

How to dynamically adjust rate limits based on system load, error rates, or resource availability.

Adaptive Rate Limiting

What you'll learn: How rate limiters can intelligently adjust their thresholds based on real-time system conditions instead of using fixed limits.

The Core Idea

While traditional rate limiting uses static thresholds (e.g., "allow 100 requests per minute"), adaptive rate limiting dynamically adjusts those limits based on current system health. Think of it like cruise control that automatically slows down when it senses poor road conditions, rather than maintaining a fixed speed regardless of circumstances.

The system monitors key metrics like:

  • CPU and memory utilization — tighten limits when resources are strained
  • Error rates — reduce traffic if errors spike (cascading failures, database timeouts)
  • Latency percentiles — lower limits when response times degrade
  • Downstream service health — protect struggling dependencies

How It Works

You establish baseline limits during normal operation, then apply multipliers based on health signals:

adaptive_limit = base_limit × health_factor

When your database CPU hits 90%, the health_factor might drop to 0.5, halving allowed requests. As conditions improve, limits gradually increase again.

Why This Matters

Fixed limits require guessing capacity in advance — set them too high and you risk overload during incidents; too low and you waste capacity during healthy periods. Adaptive limits provide:

  • Automatic protection during unexpected load or degradation
  • Better resource utilization when systems are healthy
  • Self-healing behavior without manual intervention

You can combine this with the algorithms you already know: adjust the token bucket refill rate, modify sliding window thresholds, or change concurrency limiter caps based on real-time signals.

Key Takeaway:
Adaptive rate limiting dynamically adjusts request quotas based on system health metrics, providing automatic protection during stress while maximizing throughput during normal operation.