Course contentsShow
← AI Engineering
Lesson 854 of 1,88621. Human Evaluation and FeedbackFree lesson

Annotator Training and Calibration

Onboarding human raters, establishing inter-annotator agreement, and maintaining consistency in preference judgments.

Annotator Training and Calibration

You'll learn how to prepare human raters to deliver consistent, high-quality preference judgments for your RLHF comparison data.

What This Means

When collecting comparison data for RLHF, you need multiple human annotators making judgment calls about which model output is better. But humans aren't machines—they interpret instructions differently, have personal biases, and may drift over time. Annotator training and calibration is the systematic process of teaching raters exactly what to look for and keeping their judgments aligned with your standards and each other.

Why Calibration Matters

Imagine three people judging "helpfulness" without training. One focuses on length, another on politeness, another on accuracy. Their disagreement doesn't reflect the true quality of responses—it reflects confusion about the task itself.

Calibration solves this by:

  • Onboarding with clear examples: Walk annotators through your rubric using labeled examples that show what "good" looks like
  • Testing understanding: Give quiz-style tasks with known correct answers before real annotation begins
  • Measuring agreement: Calculate inter-annotator agreement scores (from lesson 842) to identify where confusion persists
  • Refresher sessions: Periodically review edge cases and recalibrate to prevent drift

The Training Flow

  1. Written guidelines: Document your rubric with concrete examples
  2. Tutorial phase: Annotators practice on pre-labeled "gold standard" examples
  3. Assessment: They complete test cases; only those meeting agreement thresholds proceed
  4. Ongoing spot-checks: Inject gold examples into real tasks to catch quality degradation
  5. Feedback loops: When disagreements occur, discuss and refine guidelines

Key Takeaway: Well-trained, calibrated annotators turn subjective preference judgments into reliable training signals—calibration transforms human inconsistency into the consistent comparative data RLHF requires.