Temperature Scaling
What you'll learn: How to improve neural network probability calibration by dividing logits by a single learned temperature parameter before applying softmax.
What Is Temperature Scaling?
Temperature scaling is the simplest and most popular calibration method for deep neural networks performing multi-class classification. Instead of recalibrating each class probability independently (like Platt Scaling or Isotonic Regression), it applies a single scalar parameter—the temperature T—to all logits before the softmax operation.
How It Works
Normally, a neural network produces logits (raw scores) z₁, z₂, ..., zₖ for K classes, then converts them to probabilities via softmax:
P(class i) = exp(zᵢ) / Σ exp(zⱼ)
With temperature scaling, we divide all logits by T > 0:
P(class i) = exp(zᵢ/T) / Σ exp(zⱼ/T)
- T > 1: "softens" the probabilities, making the model less confident
- T < 1: "sharpens" the probabilities, making the model more confident
- T = 1: no change (original predictions)
The optimal T is found by minimizing negative log-likelihood (cross-entropy) on a held-out validation set.
Why It's Powerful
Temperature scaling only learns one parameter while affecting all classes proportionally. It doesn't change the predicted class (argmax stays the same), but dramatically improves calibration—making predicted probabilities match actual frequencies. Modern neural networks are often overconfident; temperature scaling corrects this systematically.
Key Takeaway: Temperature scaling divides logits by a single learned temperature T before softmax, providing a simple, effective way to calibrate neural network confidence without changing predictions.