Crafting Effective Reasoning Demonstrations
What you'll learn: How to design chain-of-thought examples that teach models to reason clearly without being too specific or too vague.
The Goldilocks Problem of Reasoning Examples
Now that you know how to use few-shot CoT (lesson 167), the question becomes: what makes a good reasoning example? Your demonstrations are teaching patterns to the model—too sparse and it won't follow the logic, too detailed and it becomes rigid, unable to generalize to new problems.
Think of it like showing someone how to solve math word problems. If you only show addition problems, they'll struggle with subtraction. If your steps are "I just knew the answer," they learn nothing. If you write a 20-step proof for simple arithmetic, they're overwhelmed.
Three Key Principles
Balance detail and clarity: Show enough steps to make reasoning transparent, but don't overcomplicate. For a calculation, show intermediate values. For classification, show what features you considered.
Demonstrate variety: Include examples covering different problem subtypes. If building a sentiment analyzer, show positive/negative/neutral cases and edge cases like sarcasm.
Maintain consistent structure: Keep your reasoning format similar across examples (e.g., "Analysis: ... Conclusion: ..."). This helps the model recognize the pattern of good reasoning, not just memorize specific solutions.
Practical Guidelines
- Use 3-5 diverse examples (building on few-shot CoT from lesson 167)
- Each should show 3-7 clear reasoning steps
- Vary complexity slightly across examples
- Label your reasoning sections explicitly
- Test if removing one example still works—redundancy suggests you can simplify
Key Takeaway: Effective reasoning demonstrations are diverse enough to cover problem variations, detailed enough to show clear logic, and simple enough that the model learns the reasoning process rather than memorizing solutions.