Semaphore-Based Bulkheads: Limiting Concurrent Requests
You'll learn how to implement lightweight bulkhead isolation using semaphores instead of dedicated thread pools.
The Core Idea
In the previous lesson, you saw how thread pool isolation creates separate execution environments for each dependency. But thread pools come with overhead: they consume memory, introduce context-switching costs, and require tuning pool sizes.
Semaphore-based bulkheads offer a lighter alternative. Instead of assigning threads to separate pools, you use a counting semaphore to cap the maximum number of concurrent requests to a specific dependency. When the limit is reached, additional requests fail immediately rather than queuing indefinitely.
How It Works
A semaphore is essentially a counter with two operations:
- acquire(): Decrements the counter (blocks if zero)
- release(): Increments the counter
You configure a semaphore with a maximum permit count (e.g., 10 concurrent requests to Service B). Each incoming request attempts to acquire a permit. If successful, it proceeds. If not, it fails fast. When the request completes, it releases the permit for others to use.
Analogy
Think of a parking garage with 50 spaces. The gate only lets cars in if spaces remain. Once full, new arrivals are turned away immediately—no waiting line. When a car leaves, a space frees up for the next arrival.
Advantages Over Thread Pools
- Lower memory footprint: No dedicated threads sitting idle
- Simpler tuning: One number (max permits) instead of pool sizes, queue depths, and rejection policies
- Same-thread execution: Request runs on the caller's thread, reducing context switches
Key Takeaway: Semaphore-based bulkheads provide lightweight concurrency control by limiting permits rather than segregating threads, offering isolation with minimal overhead.