Product Quantization (PQ)
What you'll learn: How to compress high-dimensional vectors into compact codes that enable memory-efficient approximate nearest neighbor search.
The Memory Problem
When you store millions or billions of vectors (each with hundreds or thousands of dimensions), memory becomes your bottleneck. A single 768-dimensional vector of 32-bit floats uses ~3KB. Multiply that by a million documents, and you're looking at gigabytes of RAM just for storage—before any indexing structures.
Product Quantization solves this by compressing vectors into short codes while preserving enough information to compute approximate similarities.
How Product Quantization Works
Think of PQ like compressing a photograph using a limited color palette instead of millions of colors. Here's the core idea:
-
Split the vector into subvectors: Divide your 768-dimensional vector into, say, 8 chunks of 96 dimensions each.
-
Create a codebook for each chunk: For each chunk position, cluster all possible values into (typically) 256 representative centroids. Store these centroids in a "codebook."
-
Encode with IDs: Replace each chunk with just the ID (0-255) of its nearest centroid. Now each chunk needs only 1 byte instead of 96 floats!
Your 768-dimensional vector (3KB) becomes just 8 bytes—a 375x compression.
Searching with PQ
During search, you compare the query against the codebook centroids and use precomputed distances to estimate similarity scores. It's approximate, not exact, but extremely fast and memory-efficient.
Key Takeaway: Product Quantization compresses vectors by splitting them into chunks and encoding each chunk with a small codebook ID, dramatically reducing memory while enabling fast approximate search at scale.