Managing Re-entrant Code

In this lesson, you will learn how to design functions that remain memory-safe and consistent even when interrupted by recursive calls or asynchronous signal handlers.

The Problem of Re-entrancy

A function is re-entrant if it can be interrupted in the middle of its execution and safely called again (either by a signal handler or recursion) before the first invocation completes. In low-level Rust, the primary danger occurs when a function modifies global state or shared heap data. If an interrupt occurs while a structure is in an inconsistent state, the "nested" call might attempt to access that same structure, leading to data races or memory corruption.

Designing Re-entrant Safe Code

To ensure re-entrancy, your code must avoid global side effects that rely on non-atomic sequences. If you must modify shared state, you cannot use standard Mutex types, as they are not async-signal-safe and can lead to deadlocks if a signal handler tries to lock a mutex already held by the interrupted thread.

Instead, use Atomic types with Ordering::Relaxed or Ordering::SeqCst to manage state flags. Consider this pattern for a re-entrant logging counter:

use std::sync::atomic::{AtomicBool, Ordering};

static IS_BUSY: AtomicBool = AtomicBool::new(false);

fn safe_log_operation() {
    // Attempt to enter the critical section
    if IS_BUSY.compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst).is_err() {
        // We were interrupted; safely abort or skip
        return;
    }

    // Perform non-blocking, re-entrant safe logic
    // ... logic here ...

    IS_BUSY.store(false, Ordering::SeqCst);
}

Critical Guidelines

1. Avoid Heap Allocation: Many allocators are not re-entrant. If a signal handler triggers malloc (via Box::new) while the main thread is already inside the allocator, the heap structures can become corrupted.
2. Use Atomic Flags: Use AtomicBool or AtomicUsize to act as "guards." If the flag is already set, you know the function has been re-entered and can choose to skip the logic or queue the work for later.
3. Minimize Critical Sections: The code between setting and clearing your atomic guard should be as small as possible to reduce the window of vulnerability.

Key Takeaway: Re-entrant code must rely exclusively on atomic primitives or stack-local variables, strictly avoiding non-atomic global state or heap allocations that can cause deadlocks and memory corruption during interruptions.