Course contentsShow
← AI Engineering
Lesson 882 of 1,88622. Testing and CI/CD for AI SystemsFree lesson

Testing Embedding Generation

Unit testing embedding model calls, dimension validation, and normalization logic.

Testing Embedding Generation

You'll learn how to write unit tests that validate embedding model calls, check vector dimensions, and verify normalization logic.

What Are We Testing?

When your AI system generates embeddings—numerical vector representations of text—you need to ensure three critical properties: the API call succeeds, the output has the correct shape, and the vectors are properly normalized (if required). Unlike testing LLM text generation where outputs vary, embeddings are deterministic for the same input, making them ideal for unit testing.

Core Testing Strategy

Think of embedding generation like a factory stamp: the same text input should always produce the same vector output. Your tests should verify:

  1. Dimension validation: Does the embedding have the expected length (e.g., 768 for text-embedding-ada-002, 1536 for newer models)?
  2. Normalization logic: If you normalize vectors for cosine similarity, does ||v|| = 1?
  3. API call structure: Are you passing the correct model name and handling responses properly?
import numpy as np

def test_embedding_dimensions():
    text = "Hello world"
    embedding = generate_embedding(text)
    assert len(embedding) == 1536, "Unexpected embedding dimension"

def test_embedding_normalization():
    text = "Test input"
    embedding = generate_embedding(text, normalize=True)
    magnitude = np.linalg.norm(embedding)
    assert abs(magnitude - 1.0) < 1e-6, "Vector not normalized"

def test_deterministic_output():
    text = "Same input"
    emb1 = generate_embedding(text)
    emb2 = generate_embedding(text)
    assert np.allclose(emb1, emb2), "Non-deterministic embeddings"

These tests catch silent failures like API version mismatches (changing dimensions) or normalization bugs that break similarity searches downstream.

Key Takeaway: Test embeddings by validating dimensions, verifying normalization math, and confirming deterministic behavior—embeddings are predictable enough to unit test without mocks.