Course contentsShow
← AI Engineering
Lesson 144 of 1,8864. Prompt Engineering FoundationsFree lesson

Logit Bias and Token Control

Directly manipulating token probabilities to encourage or suppress specific words in outputs.

Logit Bias and Token Control

What you'll learn: How to directly manipulate which words an LLM is more or less likely to generate by adjusting token probabilities at inference time.

The Core Idea

When an LLM generates text, it produces a probability distribution over all possible next tokens (words or word pieces). Logit bias lets you add or subtract from these probabilities before the model selects a token, essentially putting your thumb on the scale for specific words.

Think of it like a DJ's mixing board: the model produces the baseline volume for each word, but logit bias gives you sliders to turn certain words up (make them more likely) or down (suppress them).

How It Works

Logit bias accepts a dictionary mapping token IDs to bias values:

  • Positive values (e.g., +10) make tokens much more likely
  • Negative values (e.g., -10) suppress tokens
  • -100 effectively bans a token completely

This happens during the generation process you learned about earlier, working alongside settings like temperature and top_p. Unlike those parameters that affect how probabilities are sampled, logit bias changes the probabilities themselves.

Common Use Cases

  • Preventing specific words: Ban profanity or brand names
  • Enforcing format: Boost punctuation tokens to ensure proper JSON structure
  • Steering vocabulary: Prefer "happy" over "joyful" for consistency
  • Debugging: Force specific tokens to test model behavior

Limitations

You need the exact token ID (from the model's tokenizer) for each word—some words split into multiple tokens, making this technique precise but manual.

Key Takeaway: Logit bias gives you fine-grained control over token selection by directly adjusting probability scores, letting you encourage desired words or ban unwanted ones at the probability level.