Course contentsShow
← AI Engineering
Lesson 92 of 1,8863. API-First AI EngineeringFree lesson

Temperature, Top-p, and Generation Parameters

Control output randomness and creativity using temperature, top_p, frequency penalty, and presence penalty.

Temperature, Top-p, and Generation Parameters

What you'll learn: How to control the randomness and creativity of LLM outputs using generation parameters.

The Core Problem

When an LLM generates text, it doesn't just pick the single "best" next word—it samples from a probability distribution. Generation parameters let you control how that sampling happens, giving you a dial to turn between predictable, focused outputs and creative, diverse ones.

Key Parameters Explained

Temperature (0.0 to 2.0)

Think of temperature like a creativity dial. At temperature = 0, the model always picks the most likely next token (deterministic). At temperature = 1.0, it samples naturally from the full probability distribution. Above 1.0, it becomes more random and experimental.

  • Low (0.0-0.3): Factual tasks, code generation, structured output
  • Medium (0.7-1.0): Balanced creativity
  • High (1.5+): Creative writing, brainstorming

Top-p (Nucleus Sampling, 0.0 to 1.0)

Instead of considering all possible tokens, top-p only samples from the smallest set of tokens whose cumulative probability exceeds p. For example, top_p=0.9 means "only consider tokens that make up the top 90% probability mass."

This cuts off unlikely "tail" options while keeping variety.

Frequency Penalty & Presence Penalty

  • Frequency penalty: Reduces repetition based on how often a token has appeared
  • Presence penalty: Discourages tokens that have appeared at all, encouraging new topics

These typically range from 0.0 to 2.0, with 0 meaning no penalty.

Practical Guidance

For most tasks, start with temperature=0.7 and top_p=0.9. Lower temperature for deterministic tasks, raise it for creativity. Use penalties sparingly to reduce repetitive outputs.

Key Takeaway: Temperature controls randomness breadth, top-p controls quality thresholds, and penalties reduce repetition—together they let you tune LLM personality from robotic to wildly creative.