Beam Search for Caption Generation
What you'll learn: How to use beam search to generate higher-quality image captions by exploring multiple candidate sequences simultaneously instead of committing to one word at a time.
The Problem with Greedy Decoding
When generating captions, greedy decoding picks the single most probable next word at each step. But the locally best choice at each moment doesn't guarantee the globally best caption. It's like choosing your next step in a maze by only looking one foot ahead—you might miss the better path just around the corner.
How Beam Search Works
Beam search keeps track of the k most promising partial captions at each decoding step (where k is the "beam width"). At each time step:
- For each of the k current candidates, generate all possible next words
- Score each new sequence by its cumulative probability
- Keep only the top k sequences overall
- Repeat until all beams generate an end token or reach max length
If beam width = 1, beam search becomes greedy decoding. If beam width equals your vocabulary size, it becomes exhaustive search (impractical).
Key Techniques
Beam Width Selection: Typical values are 3-10. Larger beams find better captions but cost more computation. Empirically, quality plateaus around width 5-7 for captioning.
Length Normalization: Longer sequences accumulate lower probabilities (more multiplications of fractions < 1). Without correction, beam search favors shorter captions. Normalize by dividing the log probability by sequence length (or length^α where α ≈ 0.6-0.7) to balance quality and length.
Key Takeaway: Beam search with appropriate width (3-7) and length normalization generates higher-quality captions than greedy decoding by maintaining multiple hypotheses, trading modest computational cost for significantly better outputs.