Course contentsShow
← AI Engineering
Lesson 990 of 1,88624. Building AI APIsFree lesson

Rate Limiting with Redis

Using Redis for distributed rate limiting with sliding window counters and atomic operations.

Rate Limiting with Redis

You'll learn how to implement distributed, production-grade rate limiting using Redis for your AI API endpoints.

Why Redis for Rate Limiting?

When you have multiple API servers handling LLM requests (common in production), each server needs to share rate limit state. Redis is perfect for this because it's an in-memory database that provides:

  • Atomic operations — increment counters without race conditions
  • Key expiration — automatic cleanup of old data
  • Sub-millisecond latency — won't slow down your API
  • Distributed access — all servers read the same state

Think of Redis as a shared scoreboard that all your API servers can see and update simultaneously, ensuring accurate rate limits even under heavy load.

Sliding Window Counter Pattern

The sliding window approach tracks requests in time buckets:

import redis
import time

redis_client = redis.Redis(host='localhost', decode_responses=True)

def check_rate_limit(api_key: str, max_requests: int, window_seconds: int) -> bool:
    now = int(time.time())
    key = f"ratelimit:{api_key}:{now // window_seconds}"
    
    # Atomic increment and get count
    pipe = redis_client.pipeline()
    pipe.incr(key)
    pipe.expire(key, window_seconds * 2)  # Cleanup
    count, _ = pipe.execute()
    
    return count[0] <= max_requests

The pipeline ensures both the increment and expiration happen atomically — no other request can slip between them. The key includes the time window, so old windows naturally expire.

For more precision, you can use sorted sets with timestamps, removing old entries before counting — this gives true sliding windows rather than fixed buckets.

Key Takeaway: Redis pipelines and atomic operations provide race-condition-free rate limiting that works correctly across distributed API servers, essential for fair and accurate quota enforcement in production AI systems.