Course contentsShow
← Machine Learning and Deep Learning
Lesson 1193 of 3,53827. Pretrained Language Models: GPT Family and BeyondFree lesson

Temperature Sampling

Controlling randomness in generation by scaling logits before softmax to adjust probability distributions.

Temperature Sampling

What you'll learn: How to control the randomness and creativity of text generation by adjusting the "temperature" parameter that scales probabilities before sampling.

The Core Idea

When generating text autoregressively (one token at a time), we don't always want the most probable token — that can lead to repetitive, boring output. Temperature sampling gives us a knob to dial between predictable and creative generation.

Here's how it works: Before applying softmax to convert logits (raw model outputs) into probabilities, we divide the logits by a temperature value T. This scaling changes the shape of the probability distribution.

How Temperature Changes Behavior

Think of temperature like adjusting the confidence of a shy vs. bold speaker:

  • T = 1.0 (baseline): Use the model's original probability distribution — no change
  • T < 1.0 (e.g., 0.5): Makes high-probability tokens even more likely; the model becomes more "confident" and conservative. Output is more focused and deterministic.
  • T > 1.0 (e.g., 1.5): Flattens the distribution, giving lower-probability tokens more chance. The model becomes more "creative" and random, exploring diverse possibilities.
  • T → 0: Approaches greedy decoding (always pick the most likely token)
  • T → ∞: All tokens become equally likely (pure randomness)

The Math Briefly

Original probability: P(token) = softmax(logits)
With temperature: P(token) = softmax(logits / T)

Lower temperature sharpens peaks; higher temperature smooths them out.

Key Takeaway: Temperature sampling lets you control the randomness-creativity trade-off in text generation by scaling logits before computing probabilities — lower temperatures produce safer, more predictable text while higher temperatures encourage diversity and exploration.