Documenting Safety Failures
What you'll learn: You'll learn how to systematically record red-team findings so they become actionable assets rather than forgotten test results.
Why Documentation Matters
Finding a safety failure during red-teaming is only half the battle. Without proper documentation, vulnerabilities get lost, patterns remain undetected, and teams waste time rediscovering the same issues. Structured records transform isolated test results into institutional knowledge that drives meaningful safety improvements.
Think of it like a medical diagnosis: discovering symptoms isn't enough—you need detailed patient notes, test results, and severity classifications so the right treatment can be prescribed and outcomes tracked over time.
Core Components of Safety Failure Records
Every documented failure should include:
1. Reproducible Examples
The exact prompt or interaction sequence that triggered the failure. Include context like conversation history, tool use, or multi-turn patterns. Someone should be able to copy-paste and see the same problem.
2. Observed Behavior
What actually happened—the unsafe output, leaked information, or refused safety mechanism. Include raw model responses before any filtering.
3. Expected Behavior
What should have happened according to your content policy or safety guidelines.
4. Severity Classification
A standardized scale (e.g., Critical/High/Medium/Low) based on potential harm, likelihood of occurrence, and ease of exploitation. This prioritizes remediation efforts.
5. Metadata
Test date, model version, configuration settings, tester identity, and any relevant system state. This enables trend analysis and regression testing.
From Records to Action
Well-documented failures feed directly into your improvement cycle. They become test cases for validation, training examples for safety classifiers, and evidence for measuring progress between model versions.
Key Takeaway: Systematic documentation transforms individual safety failures into a structured knowledge base that enables pattern detection, prioritized fixes, and measurable safety improvements over time.