Course contentsShow
← Machine Learning and Deep Learning
Lesson 1952 of 3,53842. Retrieval-Augmented Generation: FoundationsFree lesson

Top-K Retrieval and Similarity Metrics

How cosine similarity and other distance metrics rank documents, and selecting the top K most relevant chunks.

Top-K Retrieval and Similarity Metrics

What you'll learn: How retrieval systems use similarity metrics to rank and select the most relevant document chunks for your query.

The Ranking Challenge

Once you have embedded both your query and a large collection of document chunks into the same vector space (using bi-encoders), you need to rank them by relevance. You don't want all thousands of chunks—just the best ones. This is where similarity metrics and top-K selection come in.

Similarity Metrics: Measuring Closeness

The most common metric in dense retrieval is cosine similarity, which measures the angle between two vectors. It ranges from -1 (opposite) to 1 (identical direction), with higher scores meaning more similar content. Cosine similarity focuses on direction, not magnitude—perfect for comparing semantic meaning.

Other metrics include:

  • Dot product: Similar to cosine but sensitive to vector magnitude
  • Euclidean distance: Measures straight-line distance between points (lower = more similar)

Think of it like a music recommendation system: cosine similarity asks "do these songs point in the same stylistic direction?" rather than "how loud are they?"

Top-K Selection

After computing similarity scores between your query embedding and every document chunk embedding, you sort the chunks by score and take the top K results—typically K=3 to K=10 for RAG systems.

Why not take everything? Token limits, cost, and noise. Irrelevant context confuses the LLM during generation. Top-K gives you the sweet spot: enough context to be comprehensive, but focused enough to be useful.

The choice of K depends on your task: complex questions might need K=10, while simple lookups work with K=3.

Key Takeaway: Similarity metrics like cosine similarity rank document relevance to your query, and top-K retrieval selects only the most relevant chunks to pass to your LLM, balancing context richness with focus.