Resource Exhaustion and OOM Protection

In this lesson, you will learn how to build "safety valves" to prevent your Rust services from crashing under memory pressure by proactively rejecting work before the system OOMs.

When building high-concurrency systems, the greatest danger isn't just high CPU usage—it’s memory exhaustion. If your async tasks allocate buffers based on incoming requests, a sudden spike in traffic can cause memory usage to grow until the operating system's OOM (Out-of-Memory) killer terminates your process. To stay resilient, you must decouple accepting work from allocating resources.

The Safety Valve Pattern

The most effective way to protect against OOM is to implement a backpressure-based admission controller. Instead of allowing an unbounded number of tasks to enter your system, you wrap your task spawning logic in a semaphore. This limits the number of concurrent "in-flight" allocations.

use tokio::sync::Semaphore;
use std::sync::Arc;

struct RequestProcessor {
    // Limits concurrent memory-intensive tasks to 100
    semaphore: Arc<Semaphore>,
}

impl RequestProcessor {
    async fn handle_request(&self) {
        // Acquire a permit. If 100 tasks are active, 
        // this suspends the current task until one finishes.
        let _permit = self.semaphore.acquire().await.unwrap();
        
        // Now perform your memory-intensive work...
        let data = vec![0u8; 1024 * 1024]; // 1MB allocation
        process(data).await;
    }
}

Why this works

By using a Semaphore, you move the "queueing" of requests from the RAM (where requests wait in memory buffers) to an async-aware structure that merely suspends the task. If your semaphore limit is reached, new incoming requests will simply sit in the connection buffer or be rejected at the load balancer layer, keeping the application's heap footprint stable.

You should couple this with Load Shedding. If the semaphore acquisition times out, don't wait indefinitely; return a 503 Service Unavailable error immediately. This signals to upstream clients to slow down, protecting your system's health. By strictly controlling the concurrency of operations that trigger heavy allocations, you ensure that even under extreme load, your service operates at a constant, predictable memory ceiling rather than expanding until it crashes.

Key Takeaway: Use semaphores as admission controllers to bound active memory allocations, turning potentially fatal OOM crashes into manageable backpressure signals.