Temperature, Top-p, and Generation Parameters
What you'll learn: How to control the randomness and creativity of LLM outputs using generation parameters.
The Core Problem
When an LLM generates text, it doesn't just pick the single "best" next word—it samples from a probability distribution. Generation parameters let you control how that sampling happens, giving you a dial to turn between predictable, focused outputs and creative, diverse ones.
Key Parameters Explained
Temperature (0.0 to 2.0)
Think of temperature like a creativity dial. At temperature = 0, the model always picks the most likely next token (deterministic). At temperature = 1.0, it samples naturally from the full probability distribution. Above 1.0, it becomes more random and experimental.
- Low (0.0-0.3): Factual tasks, code generation, structured output
- Medium (0.7-1.0): Balanced creativity
- High (1.5+): Creative writing, brainstorming
Top-p (Nucleus Sampling, 0.0 to 1.0)
Instead of considering all possible tokens, top-p only samples from the smallest set of tokens whose cumulative probability exceeds p. For example, top_p=0.9 means "only consider tokens that make up the top 90% probability mass."
This cuts off unlikely "tail" options while keeping variety.
Frequency Penalty & Presence Penalty
- Frequency penalty: Reduces repetition based on how often a token has appeared
- Presence penalty: Discourages tokens that have appeared at all, encouraging new topics
These typically range from 0.0 to 2.0, with 0 meaning no penalty.
Practical Guidance
For most tasks, start with temperature=0.7 and top_p=0.9. Lower temperature for deterministic tasks, raise it for creativity. Use penalties sparingly to reduce repetitive outputs.
Key Takeaway: Temperature controls randomness breadth, top-p controls quality thresholds, and penalties reduce repetition—together they let you tune LLM personality from robotic to wildly creative.