Keyword and Regex-Based Filtering
What You'll Learn
You'll learn how to implement fast, rule-based filters using keyword lists and regex patterns as your first line of defense against unwanted content.
The Fast Lane of Content Moderation
Before sending text to your LLM or releasing generated output to users, keyword and regex-based filters act like security checkpoints at airport entrances—quick, deterministic checks that catch obvious problems instantly.
Think of this approach as a bouncer at a club with a clear "do not admit" list. If someone's name is on that list, they don't get in—no need for complex reasoning or AI judgment calls.
Why Start Here?
While ML-based content classifiers (from our previous lesson) are powerful, they:
- Add latency (100-500ms per call)
- Cost money per API request
- Can miss exact matches
Rule-based filters are:
- Instant: Microseconds, not milliseconds
- Free: No API costs
- Deterministic: Same input always yields same result
- Transparent: You know exactly why content was blocked
Common Patterns
Keyword blocklists: Maintain lists of prohibited terms, slurs, or banned topics. Match against user input or model output.
Regex patterns: Detect structured threats like:
- Email addresses or phone numbers (PII leakage)
- Credit card patterns
- SQL injection attempts
- Specific harmful instruction formats
Allowlists: In high-stakes domains, only permit known-safe patterns.
The Reality Check
These filters are necessary but not sufficient. Adversaries quickly learn to bypass simple keyword matching with obfuscation ("cr3dit c4rd"), but catching the low-hanging fruit reduces load on your more expensive ML layers.
Key Takeaway: Keyword and regex filtering provides blazing-fast, cost-free first-line defense that blocks obvious policy violations before they reach expensive ML classifiers or your LLM—but must be combined with smarter layers for robust protection.