Course contentsShow
← Security
Lesson 1167 of 3,05224. Application Security - Input ValidationFree lesson

Unicode Normalization and Equivalence

Learn how different Unicode representations of the same character can bypass validation and create security vulnerabilities.

Unicode Normalization and Equivalence

What you'll learn: How attackers exploit different Unicode representations of identical characters to slip past input validation filters.


The Core Problem

Unicode allows multiple byte sequences to represent the same visual character. For example, the letter "é" can be encoded two ways:

  1. Precomposed form (NFC): A single codepoint U+00E9
  2. Decomposed form (NFD): Base letter "e" (U+0065) + combining acute accent (U+0301)

These look identical to humans, but to a naive string comparison, they're completely different. An attacker can use this to bypass validation that checks for dangerous strings.

The Security Impact

Imagine your validation blocks filenames containing "../" to prevent directory traversal. An attacker submits a path using the decomposed form of special characters. Your filter doesn't recognize it—but the filesystem normalizes it later and accepts ../etc/passwd.

Similarly, consider username blacklists. If you block "admin" but the attacker registers "admın" (using Turkish dotless i, U+0131), they might bypass your check while the authentication system later treats them as equivalent.

Why This Matters

Different systems normalize Unicode at different stages:

  • Your validation layer might check the raw input
  • The database might normalize before storage
  • The filesystem or browser might normalize before use

If normalization happens after validation, attackers slip through.

Example Attack Flow

  1. Validation checks for blacklisted string "script"
  2. Attacker sends "ſcript" (using long s, U+017F)
  3. Validation passes—strings don't match
  4. Later processing normalizes ſ → s
  5. Result: "script" executes despite being "blocked"

Key Takeaway: Always normalize Unicode input to a canonical form (like NFC) before validation, and ensure all downstream systems use the same normalization to prevent attackers from exploiting character equivalence.