Example Ordering Effects
You'll learn why the sequence of few-shot examples dramatically impacts model predictions, and how recency bias influences what the model emphasizes.
The Order Matters More Than You Think
When you provide few-shot examples to a language model, you might assume the order doesn't matter—after all, shouldn't the model learn equally from all examples? Unfortunately, the sequence significantly affects predictions, often in surprising ways.
Language models exhibit recency bias: they give disproportionate weight to examples appearing near the end of the prompt. Think of it like a conversation where someone asks you to identify a pattern after showing several examples—you'll naturally remember and emphasize the most recent ones you just saw.
Why This Happens
Models process prompts sequentially, and the attention mechanism naturally emphasizes more recent context. The last few examples are "fresher" in the model's working memory when it begins generating its response. This means:
- Last example → strongest influence on output style, format, and reasoning pattern
- Middle examples → moderate influence, sometimes overlooked
- First example → foundational but can be overshadowed
Practical Implications
If your last example shows a verbose explanation, expect verbose outputs. If it demonstrates concise answers, you'll get brevity. Strategic ordering means:
- Place your most representative or highest-quality example last
- Put edge cases in middle positions to avoid overemphasis
- Consider randomizing order during evaluation to test robustness
Key Takeaway: The order of few-shot examples is a powerful but often overlooked lever—recency bias means your last example disproportionately shapes the model's response, so choose its position strategically.