Course contentsShow
← AI Engineering
Lesson 1591 of 1,88638. Bias, Fairness, and AlignmentFree lesson

Self-Critique and Revision

Training models to critique their own outputs against constitutional principles and revise responses accordingly.

Self-Critique and Revision

What you'll learn: How to train language models to evaluate and improve their own outputs by comparing them against constitutional principles.

The Core Idea

Self-critique and revision is a powerful extension of Constitutional AI where the model acts as its own reviewer. Instead of simply generating a response and stopping, the model learns to:

  1. Generate an initial response
  2. Critique that response against constitutional principles (honesty, harmlessness, helpfulness, fairness)
  3. Identify specific problems or violations
  4. Revise the response to address those issues

Think of it like writing an essay with built-in self-editing: you draft your first version, review it against your standards, spot the weaknesses, and rewrite before submitting.

How It Works in Practice

After Constitutional AI training (lesson 1590), models can be prompted or fine-tuned to perform explicit critique steps. For example:

  • Step 1 (Generate): The model produces an initial answer
  • Step 2 (Critique): The model examines its own output: "Does this response contain stereotypes? Is it unnecessarily evasive? Does it make unsupported claims?"
  • Step 3 (Revise): Based on identified issues, the model generates an improved version

This creates a feedback loop within a single generation process. The model essentially debates with itself, catching bias, errors, or alignment failures before they reach the user.

Benefits

  • Reduced harmful outputs without external filtering
  • Improved reasoning transparency (you can log the critique)
  • Scalable alignment that doesn't require constant human review
  • Adaptive correction based on constitutional principles rather than rigid rules

Key Takeaway: Self-critique and revision transforms models into self-monitoring systems that can identify and fix alignment issues in their own outputs before delivering final responses.