Evaluating Vector Search Quality

Use metrics like recall and precision to tune search performance.

In this lesson, you will learn how to quantitatively measure the performance of your vector search systems using recall and precision to ensure your retrieval strategy is actually serving relevant results.

When performing Approximate Nearest Neighbor (ANN) searches, we often trade perfect accuracy for speed. To ensure this trade-off remains acceptable, we must measure the quality of our search engine against a "ground truth"—a set of exact nearest neighbors calculated via exhaustive search (Brute Force).

Precision and Recall in Vector Space

Recall@K: This is the most critical metric for vector databases. It measures the proportion of relevant items (the true K-nearest neighbors) that your index actually returned within the top K results. If your query returns 10 results and 8 of them are in the "ground truth" set of 10, your Recall@10 is 0.8.
Precision@K: This measures how many of the retrieved K items are actually relevant. If you request 10 results and only 5 are correct, your Precision@10 is 0.5.

Tuning the Index

High recall often comes at the cost of higher latency or larger index sizes. If your Recall@K is too low, you aren't finding the right data. To tune this, you generally adjust your index parameters, such as the ef (effective) parameter in HNSW indices or the number of clusters (nlist) in IVF indices.

Consider this conceptual check of your search quality:

-- Conceptual logic to evaluate search quality
WITH GroundTruth AS (
    SELECT id FROM vector_table 
    ORDER BY cosine_distance(query_embedding, vector_col) 
    LIMIT 10 -- The "perfect" set
),
ApproximateSearch AS (
    SELECT id FROM vector_table 
    WHERE vector_col <-> query_embedding -- Using index
    LIMIT 10
)
SELECT 
    (SELECT COUNT(*) FROM ApproximateSearch WHERE id IN (SELECT id FROM GroundTruth))::float / 10 
    AS recall_at_10;

If your recall_at_10 is below your threshold (e.g., < 0.95), you must adjust your indexing strategy. You might increase the number of probes in an IVF index or increase the construction-time hyper-parameters of your graph index, effectively trading off query speed for higher retrieval accuracy.

Key Takeaway: Use Recall@K as your primary metric to balance the trade-off between search speed and the accuracy of your vector retrieval results.