Course contentsShow
← Machine Learning and Deep Learning
Lesson 1312 of 3,53829. Core NLP TasksFree lesson

Decoding Strategies: Greedy and Beam Search

Learn how greedy decoding selects highest-probability tokens and how beam search maintains multiple candidate sequences for better quality.

Decoding Strategies: Greedy and Beam Search

What you'll learn: How language models choose which words to generate next, comparing the simple greedy approach with the more sophisticated beam search method.

The Decoding Challenge

When a language model generates text, it doesn't just produce one word—it calculates probabilities for thousands of possible next tokens. Decoding strategies determine how we select from these possibilities to build coherent sequences.

Greedy Decoding

Greedy decoding takes the simplest approach: at each step, pick the single token with the highest probability. It's like always choosing the most obvious next word in a sentence.

Pros: Fast and deterministic—you'll always get the same output for the same input.

Cons: Shortsighted. A locally optimal choice might lead to a poor overall sentence. Imagine choosing "the" because it has highest probability, only to realize "a" would have led to a much better complete sentence.

Beam Search

Beam search maintains multiple candidate sequences simultaneously (the "beam width" determines how many). At each step, it expands all candidates, scores the resulting sequences, and keeps only the top-k most probable complete paths.

Think of it like exploring multiple conversation paths at once rather than committing to one direction immediately. If beam width = 3, you're juggling three potential sentences, allowing recovery from earlier suboptimal choices.

Pros: Better quality text by considering longer-term consequences.

Cons: Slower (processing multiple sequences), and larger beam widths don't always guarantee better results—they can favor generic, safe outputs.

When to Use Each

Greedy decoding works well for simple tasks where speed matters. Beam search (typically beam width 3-10) produces higher-quality output for translation, summarization, and question answering where you want the best overall sequence, not just the best next token.

Key Takeaway: Greedy decoding selects the single highest-probability token at each step (fast but myopic), while beam search tracks multiple candidate sequences simultaneously to find globally better outputs at the cost of computational efficiency.