In this lesson, you will learn how to design a resilient queueing layer that keeps your background workers within an external provider’s strict rate limits.
When dealing with third-party APIs that enforce rate limits (like 50 requests per second), simply scaling up your background workers can lead to 429 Too Many Requests errors or temporary IP bans. Instead of letting workers hammer the API, you must decouple request processing from the actual API invocation using a Throttled Producer-Consumer pattern.
Implementation Strategy
The most robust approach in C# is to use a Channel<T> as a local, thread-safe buffer combined with a SemaphoreSlim or a RateLimiter (available in System.Threading.RateLimiting). By using a dedicated "API Dispatcher" background service, you ensure that even if you have hundreds of incoming messages in your queue, only the allowed number of requests are dispatched per second.
// Inside your BackgroundService
private readonly Channel<ApiRequest> _requestQueue;
private readonly TokenBucketRateLimiter _limiter = new(new() { ReplenishmentPeriod = TimeSpan.FromSeconds(1), TokensPerPeriod = 50 });
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
await foreach (var request in _requestQueue.Reader.ReadAllAsync(stoppingToken))
{
// Acquire a permit before allowing the execution
using var lease = await _limiter.AcquireAsync(1, stoppingToken);
if (lease.IsAcquired)
{
await _httpClient.SendAsync(request.Message, stoppingToken);
}
else
{
// Re-queue or delay if the rate limit is hit
await _requestQueue.Writer.WriteAsync(request, stoppingToken);
}
}
}
Why this works
By introducing this internal "gatekeeper," you transition from a "push-as-fast-as-possible" model to a "pull-when-allowed" model. If the external provider changes their limits, you only need to adjust the RateLimiter configuration rather than re-engineering your entire message-processing infrastructure. Furthermore, because this logic resides within the worker, it handles spikes in traffic gracefully by allowing the internal queue to grow temporarily rather than dropping messages or overwhelming the remote host.
Key Takeaway: Decouple API consumption from message processing by using a local rate-limited dispatcher to ensure your background workers never exceed external constraints regardless of traffic volume.