In this lesson, you will learn how to prevent local system overload and respect API constraints by implementing a Token Bucket algorithm to throttle your outgoing HTTP requests.
While exponential backoff (which you previously learned) helps you react after you’ve been rate-limited, client-side throttling allows you to be proactive by pacing your requests before they even leave your system. The Token Bucket algorithm is the industry standard for this. Imagine a bucket that starts full of tokens. Every time you want to make a request, you must "spend" a token. If the bucket is empty, you must wait until it refills at a steady rate.
The Token Bucket Implementation
You can implement this logic easily using Python’s time module to track when tokens are replenished:
import time
class TokenBucket:
def __init__(self, capacity, fill_rate):
self.capacity = capacity # Max tokens
self.fill_rate = fill_rate # Tokens added per second
self.tokens = capacity
self.last_update = time.monotonic()
def consume(self):
# Refill tokens based on elapsed time
now = time.monotonic()
delta = now - self.last_update
self.tokens = min(self.capacity, self.tokens + delta * self.fill_rate)
self.last_update = now
if self.tokens >= 1:
self.tokens -= 1
return True
return False
# Usage
bucket = TokenBucket(capacity=5, fill_rate=1) # 5 burst, 1 req/sec
while True:
if bucket.consume():
# Perform your API request here
print("Request sent!")
else:
time.sleep(0.1) # Wait for tokens to replenish
Why This Matters
By enforcing this logic locally, you avoid "429 Too Many Requests" errors entirely. This approach is superior to a simple time.sleep() because it allows for "burstiness." If your application has been idle for a while, the bucket fills up, letting you send a small cluster of requests immediately before settling into the sustained fill_rate. This balances your system's performance with the strict requirements of the remote API, ensuring that your background tasks or data integration scripts remain stable and reliable.
Key Takeaway: Use the Token Bucket algorithm to proactively pace your outgoing requests, balancing your need for throughput with the strict rate limits imposed by external APIs.