Course contentsShow
← AI Engineering
Lesson 142 of 1,8864. Prompt Engineering FoundationsFree lesson

Frequency and Presence Penalties

Applying penalties to reduce repetition and encourage diverse vocabulary in generated text.

Frequency and Presence Penalties

What you'll learn: How to apply penalties that reduce repetition and encourage more diverse vocabulary in AI-generated text.

The Problem with Raw Generation

Without guidance, language models sometimes get stuck in loops, repeating the same phrases or words over and over. Imagine a friend who keeps saying "you know" or "basically" in every sentence—it gets distracting and limits expression.

What Are Penalties?

Frequency penalty and presence penalty are numerical parameters (typically ranging from -2.0 to 2.0) that modify how likely the model is to reuse tokens:

Frequency Penalty

Penalizes tokens based on how many times they've already appeared. If "delicious" has been used three times, it gets penalized more heavily than a word used once. Think of it like a tax that increases with repetition count.

Presence Penalty

Penalizes tokens simply for having appeared at all, regardless of count. Once "delicious" shows up even once, it's discouraged from appearing again—a flat penalty for any prior presence.

When to Use Each

  • Frequency penalty: Use when you want to reduce excessive repetition (0.5–1.0 works well for most cases)
  • Presence penalty: Use when you want maximum diversity and topic exploration (0.5–1.0 encourages new concepts)
  • Both together: Combine them for balanced control—frequency handles word-level variety, presence encourages topic shifts

Higher positive values push toward more creative, varied outputs. Negative values do the opposite (encouraging repetition, rarely useful). Zero means no penalty at all.

Key Takeaway: Frequency and presence penalties are your tools for controlling repetition—frequency penalizes proportionally to reuse count, while presence applies a flat discouragement to any previously-seen token, giving you fine-grained control over vocabulary diversity.