Avoiding Thundering Herd Problems

In this lesson, you will learn how to prevent the "thundering herd" problem by selectively waking only the threads necessary to handle a pending task.

The Thundering Herd Phenomenon

When multiple threads are waiting on a Condvar (Condition Variable) and a single event occurs, you might be tempted to use notify_all(). If 50 threads are waiting for a resource that only one thread can consume at a time, calling notify_all() wakes every single thread simultaneously. All 50 threads immediately scramble to acquire the Mutex lock.

Only one thread succeeds; the other 49 threads wake up, context switch, contend for the lock, realize the resource is gone, and go back to sleep. This "thundering herd" of wasted CPU cycles creates a performance bottleneck that degrades as you add more threads.

The Solution: Targeted Signaling

To avoid this, use notify_one() instead of notify_all() when the resource is consumed by exactly one thread. This wakes only one waiter, leaving the others in their efficient suspended state.

Consider a producer-consumer queue where you only want to signal one worker:

use std::sync::{Arc, Mutex, Condvar};

let pair = Arc::new((Mutex::new(false), Condvar::new()));
// ... inside worker loop ...
let (lock, cvar) = &*pair;
let mut started = lock.lock().unwrap();

while !*started {
    started = cvar.wait(started).unwrap();
}

// Perform work...

// Only signal one other thread if more work is potentially available
cvar.notify_one(); 

When is notify_all() appropriate?

Use notify_all() only when the state change affects all waiting threads. For example, if you are signaling threads to shut down or updating a global configuration variable that every thread needs to see, notify_all() is the correct tool. If the task is discrete—like popping a single item off a work queue—stick to notify_one() to keep your system's throughput stable under high contention.

Key Takeaway: Use notify_one() to signal specific tasks to individual threads; reserve notify_all() only for broad state changes that invalidate the waiting condition for every thread simultaneously.