Balancing Critique Specificity
What You'll Learn
You'll learn how to calibrate the granularity of self-critique feedback so models improve effectively without getting stuck in over-correction loops or ignoring important issues.
The Goldilocks Problem of Critique
When a language model critiques its own output (or you design prompts for critique), you face a fundamental trade-off: too vague and the model can't act on the feedback; too specific and it may hyper-focus on minor details while missing broader issues—or make unnecessary changes that break what was already working.
Think of it like editing a draft essay. A critique saying "improve this" is useless. But marking every comma placement while ignoring structural problems wastes effort and creates churn without real improvement.
Finding the Sweet Spot
High-level critique works well for:
- Early refinement iterations
- Identifying strategic problems (logic, completeness, relevance)
- Avoiding unnecessary token costs
Detailed critique works well for:
- Later refinement stages after major issues are fixed
- Format compliance and precise schema requirements
- Edge cases and subtle errors
The optimal strategy often uses progressive specificity: start broad, then zoom in as major issues resolve. Your critique prompt might ask: "First identify any fundamental errors, then examine details only if the foundation is sound."
Practical Heuristics
- Critique at the level where change is actionable
- Avoid listing minor issues when major flaws exist
- Match specificity to your stopping criteria (lesson 1941)
- If refinement stalls or oscillates, your critique may be too granular
Key Takeaway: Effective self-critique balances breadth and depth—starting with strategic issues before drilling into details—preventing wasted refinement cycles while ensuring meaningful improvement at each iteration.