Course contentsShow
← Machine Learning and Deep Learning
Lesson 1939 of 3,53841. Structured Output and Tool CallingFree lesson

Self-Consistency Through Critique

Detecting and resolving contradictions in model outputs using self-evaluation.

Self-Consistency Through Critique

What you'll learn: How to use self-evaluation to detect and resolve contradictions in model outputs, making them more coherent and reliable.

The Core Idea

Self-consistency through critique combines two powerful techniques you've already learned: self-critique (where the model evaluates its own work) and self-consistency (generating multiple reasoning paths). Instead of just voting on multiple outputs, you use critique to identify contradictions within or across responses, then resolve them through refinement.

Think of it like a peer review process for a single author: you write multiple drafts, compare them for inconsistencies, flag contradictions, and revise until your story holds together.

How It Works

  1. Generate multiple candidate outputs using temperature sampling (like standard self-consistency)
  2. Use critique prompts to compare outputs and identify contradictions
  3. Resolve inconsistencies by generating refined outputs that reconcile differences
  4. Validate coherence through another critique pass

Why This Matters

Standard self-consistency voting can miss subtle issues: two outputs might reach the same conclusion through contradictory logic, or make incompatible claims that aren't immediately obvious. Critique-based consistency catches these problems by explicitly checking for logical conflicts.

For example, if one reasoning path says "The meeting is on Tuesday because..." and another says "Tuesday is a holiday so the meeting must be Wednesday," voting might miss this contradiction. Critique would flag it explicitly.

The Pattern

You prompt the model to act as a consistency checker: "Compare these two responses. Do they contradict each other? If so, where exactly?" Then use that feedback to generate a reconciled, consistent final answer.

Key Takeaway: Self-consistency through critique goes beyond voting by actively detecting and resolving logical contradictions, producing outputs that are internally coherent across multiple reasoning attempts.