Normalization and Preprocessing
What You'll Learn
You'll learn why embeddings are normalized (using the L2 norm) and how this ensures consistent, reliable similarity comparisons.
Why Normalize Embeddings?
When you create embeddings, different models or text inputs can produce vectors with wildly different magnitudes. A short sentence might produce a vector with length 2.5, while a longer one produces length 10.3. If you compare these directly using dot products, the longer vector will dominate—not because it's more similar, but simply because it's bigger.
Normalization solves this by scaling all vectors to the same length (typically 1.0), so comparisons measure direction rather than magnitude. Think of it like comparing compass directions instead of how far someone walked—you care about which way they're pointing, not how long their steps were.
L2 Normalization (Unit Vectors)
L2 normalization divides each vector component by the vector's length (its L2 norm). The formula is:
L2 norm = √(x₁² + x₂² + ... + xₙ²)
After normalization, every vector has length exactly 1.0—it becomes a unit vector pointing in the same direction as the original.
Why This Matters for Similarity
Once normalized, dot product similarity equals cosine similarity. This makes comparisons consistent: two vectors pointing the same direction score ~1.0, opposite directions score ~-1.0, and perpendicular vectors score ~0.0, regardless of their original magnitudes.
Most embedding APIs return pre-normalized vectors, but always verify this—especially when combining embeddings from different sources or models.
Key Takeaway: Normalizing embeddings with the L2 norm ensures that similarity measurements reflect semantic direction rather than vector magnitude, making comparisons fair and consistent.