Course contentsShow
← Python
Lesson 819 of 1,06013. Data Integration and Web APIsFree lesson

Implementing Client-Side Request Throttling

Learners will be able to prevent local system overload by enforcing request rate limits using token bucket algorithms for outgoing API traffic.

In this lesson, you will learn how to prevent local system overload and respect API constraints by implementing a Token Bucket algorithm to throttle your outgoing HTTP requests.

While exponential backoff (which you previously learned) helps you react after you’ve been rate-limited, client-side throttling allows you to be proactive by pacing your requests before they even leave your system. The Token Bucket algorithm is the industry standard for this. Imagine a bucket that starts full of tokens. Every time you want to make a request, you must "spend" a token. If the bucket is empty, you must wait until it refills at a steady rate.

The Token Bucket Implementation

You can implement this logic easily using Python’s time module to track when tokens are replenished:

import time

class TokenBucket:
    def __init__(self, capacity, fill_rate):
        self.capacity = capacity  # Max tokens
        self.fill_rate = fill_rate  # Tokens added per second
        self.tokens = capacity
        self.last_update = time.monotonic()

    def consume(self):
        # Refill tokens based on elapsed time
        now = time.monotonic()
        delta = now - self.last_update
        self.tokens = min(self.capacity, self.tokens + delta * self.fill_rate)
        self.last_update = now

        if self.tokens >= 1:
            self.tokens -= 1
            return True
        return False

# Usage
bucket = TokenBucket(capacity=5, fill_rate=1) # 5 burst, 1 req/sec
while True:
    if bucket.consume():
        # Perform your API request here
        print("Request sent!")
    else:
        time.sleep(0.1) # Wait for tokens to replenish

Why This Matters

By enforcing this logic locally, you avoid "429 Too Many Requests" errors entirely. This approach is superior to a simple time.sleep() because it allows for "burstiness." If your application has been idle for a while, the bucket fills up, letting you send a small cluster of requests immediately before settling into the sustained fill_rate. This balances your system's performance with the strict requirements of the remote API, ensuring that your background tasks or data integration scripts remain stable and reliable.

Key Takeaway: Use the Token Bucket algorithm to proactively pace your outgoing requests, balancing your need for throughput with the strict rate limits imposed by external APIs.