Unicode Normalization Vulnerabilities
What you'll learn: How attackers exploit different Unicode representations of "the same" text to bypass security checks that don't normalize input first.
The Core Problem
Unicode allows multiple byte sequences to represent visually identical characters. The letter "é" can be:
- NFC (Composed): One codepoint
U+00E9 - NFD (Decomposed): Two codepoints
U+0065(e) +U+0301(combining acute accent)
If your validation checks for dangerous strings before normalizing Unicode, attackers can encode malicious input in an equivalent form that slips past your filters.
The Attack Pattern
Imagine a blocklist rejecting filenames containing ../ for path traversal protection:
- Validator checks:
"safe.txt"✅ |"../etc/passwd"❌ - But attacker submits:
"..%E2%81%84etc/passwd"where%E2%81%84isU+2044(FRACTION SLASH) - Validator sees no literal
../— passes validation ✅ - Later code normalizes to NFKC (compatibility form), converting
⁄→/ - Result processed:
"../etc/passwd"💥
The compatibility normalizations (NFKC, NFKD) are especially dangerous because they convert "similar-looking" characters to canonical forms: ① → 1, ⓐ → a, < (fullwidth) → <.
Why This Happens
Validation and processing code operate at different layers. If validation works on raw input but later filesystem APIs, databases, or output encoders normalize Unicode differently, the semantic meaning changes after validation.
Analogy: Like a security guard checking IDs in dim light, then the bouncer inside sees the same ID under bright light and reads different information.
Defense
Always normalize first, validate second using a consistent form (typically NFC or NFKC) at your trust boundary, before any security checks.
Key Takeaway: Unicode equivalence means byte-different strings can be semantically identical; always normalize input to a canonical form before validation to prevent bypass attacks exploiting different representations.