Course contentsShow
← AI Engineering
Lesson 395 of 1,88610. RAG - Improving RetrievalFree lesson

Implementing Basic Reranking

Adding a reranking step after initial retrieval using models like ms-marco-MiniLM.

Implementing Basic Reranking

What You'll Learn

You'll learn how to add a reranking step after your initial vector retrieval to significantly improve the relevance of your final RAG context.

The Core Idea

After your vector database returns candidate documents (say, the top 20), you feed them through a more powerful reranker model like ms-marco-MiniLM that scores each document based on how well it actually answers your specific query. You then pick the top N reranked results to send to your LLM.

Think of it like a two-stage filter: your vector search is a fast, rough net that catches many fish quickly, while the reranker is a careful inspector who examines each fish and picks only the best ones.

Why This Step Matters

Vector search uses embeddings to find semantically similar documents, but it can't deeply understand query-document relevance. Cross-encoder rerankers (the models you learned about in the previous lesson) actually look at the query and each document together, giving far more accurate relevance scores.

Basic Implementation Flow

  1. Retrieve broadly: Get top-k candidates from your vector DB (e.g., k=20)
  2. Score precisely: Pass query + each candidate through your reranker
  3. Reorder: Sort by reranker scores (highest first)
  4. Select: Take the top-n reranked results (e.g., n=5) for your LLM context

The reranker model takes the query and one document at a time, producing a relevance score. You don't need to embed anything—the cross-encoder handles the text directly.

Common Reranker Models

  • ms-marco-MiniLM-L-6-v2 (fast, lightweight)
  • ms-marco-MiniLM-L-12-v2 (better accuracy, slower)
  • Cohere Rerank API (managed service)

Key Takeaway: Implementing reranking means retrieving more candidates than you need, scoring them with a cross-encoder model, and selecting only the top-scored documents to maximize RAG accuracy.