Numerical Stability Considerations
You'll learn which operations must stay in FP32 during mixed precision training and how automatic casting protects numerical accuracy.
The Precision Problem
Mixed precision training uses FP16/BF16 for speed and memory savings, but not all operations are safe in lower precision. Some computations—especially those involving very small gradients, accumulating sums, or exponentials—can overflow, underflow, or lose critical precision when computed in reduced precision formats.
Operations That Need FP32
Certain operations must remain in FP32 to maintain numerical stability:
- Loss functions (especially those with log or exp operations)
- Softmax and log-softmax (exponentials can overflow in FP16)
- Batch normalization statistics (mean/variance accumulation needs precision)
- Large reductions (summing thousands of values compounds rounding errors)
- Layer normalization and other normalization layers
Think of it like financial calculations: you can round to the nearest dollar for most transactions, but when calculating interest on millions of accounts, you need cent-level precision or errors compound catastrophically.
Automatic Casting Rules in AMP
PyTorch's Automatic Mixed Precision (AMP) uses op-specific whitelists and blacklists:
- FP16-safe ops (matmuls, convolutions): automatically cast to FP16
- FP32-required ops (softmax, norms): stay in or promote to FP32
- Flexible ops: match input precision
AMP's autocast context manager handles this automatically, inserting casts only where needed while keeping the master weights in FP32.
Key Takeaway: Not all operations tolerate reduced precision—losses, normalizations, and reductions typically require FP32, and AMP's automatic casting intelligently maintains numerical stability by keeping risky operations in higher precision while accelerating safe matrix operations.