Defense: Output Filtering and Moderation
What You'll Learn
You'll learn how to add a safety layer after your language model generates text, catching harmful content before it reaches users.
The Last Line of Defense
Even when you've sanitized inputs (Lesson 3421), attackers may still trick your model into generating harmful outputs. Output filtering acts as a quality-control checkpoint: before any model response reaches the user, it passes through classifiers and rule-based systems that screen for problematic content.
Think of it like a newspaper editor who reviews articles before publication—no matter how good the writer, the editor ensures nothing harmful gets printed.
How It Works
Classifier-Based Filtering: Train separate safety classifiers to detect categories like hate speech, violence, self-harm, or PII leakage. When your LLM generates a response, run it through these specialized models. If the safety score exceeds a threshold, block or modify the output.
Rule-Based Systems: Use regular expressions, keyword lists, or pattern matching to catch obvious violations. For example, reject outputs containing specific slurs, phone numbers, or suspicious URLs.
Hybrid Approaches: Combine both: rules catch blatant cases instantly (fast), while classifiers handle nuanced context (accurate). You might also use multiple classifiers in ensemble to reduce false positives.
Implementation Considerations
- Latency: Filtering adds processing time—optimize for speed
- False positives: Overly aggressive filtering frustrates legitimate users
- Evasion: Attackers may craft outputs that slip past filters (e.g., obfuscated text, indirect phrasing)
- Model drift: Adversaries adapt; retrain classifiers regularly with new attack patterns
Output filtering complements input sanitization: together they create defense-in-depth, where multiple layers protect against jailbreaks and harmful generation.
Key Takeaway: Output filtering and moderation uses classifiers and rule-based systems as a post-processing safety net, screening model outputs for harmful content before delivery—your last chance to prevent bad responses from reaching users.