Course contentsShow
← Machine Learning and Deep Learning
Lesson 1823 of 3,53838. Instruction Tuning and AlignmentFree lesson

Writing and Selecting Constitutional Principles

How to design principles that are specific, testable, and cover diverse safety considerations.

Writing and Selecting Constitutional Principles

What you'll learn: How to craft effective constitutional principles that guide AI models toward safe, helpful behavior through clear, testable, and comprehensive guidelines.

The Core Idea

In Constitutional AI (from lessons 1820-1822), you've seen how principles guide model behavior through critique-revision and RL from AI feedback. But how do you actually write good principles? Not all principles are created equal—vague ones like "be good" are useless, while overly specific ones miss edge cases.

Think of constitutional principles like a referee's rulebook: they need to be specific enough that the AI labeler (from lesson 1819) can consistently apply them, yet broad enough to cover diverse scenarios.

Characteristics of Effective Principles

Specific and Actionable: Instead of "be harmless," write "Do not provide instructions for creating weapons or explosives." The AI can clearly judge whether a response violates this.

Testable: You should be able to apply the principle to any model output and get a clear yes/no answer. "Choose the response that is more helpful" is testable; "be better" is not.

Coverage of Safety Dimensions: Your principle set should span multiple concerns:

  • Harmlessness (avoid dangerous content)
  • Honesty (don't fabricate information)
  • Helpfulness (actually answer the user)
  • Bias and fairness (treat groups equitably)

Balanced Trade-offs: Sometimes principles conflict—being maximally helpful might reduce safety. Good principles explicitly handle these: "If the request could cause harm, politely decline even if less helpful."

Selection Strategy

Start with 10-20 principles covering your main safety areas. Test them on diverse prompts. Remove redundant ones, split overly broad ones, and refine ambiguous wording based on how consistently the AI labeler applies them.

Key Takeaway: Effective constitutional principles are specific enough to be testable, diverse enough to cover multiple safety dimensions, and refined through empirical testing of how consistently AI labelers can apply them.