The Reranking Problem
What you'll learn: Why retrieving many candidates first and then reranking them leads to better final results than trying to get perfect retrieval in one shot.
The Core Challenge
When you perform vector or hybrid search, you typically retrieve the top-K documents (say, top 100) based on similarity scores. However, initial retrieval optimizes for recall — casting a wide net to ensure relevant documents are somewhere in those 100 results. The problem? The most relevant items aren't always ranked first, and you might only show the user (or LLM) the top 5-10.
This is where reranking comes in.
Why Initial Recall Matters
Think of retrieval like a two-stage hiring process. The first stage (initial retrieval) screens hundreds of résumés quickly using simple criteria — you want to ensure qualified candidates aren't accidentally filtered out. You prioritize not missing good candidates over perfect ordering.
But you can't interview all 100. So stage two (reranking) carefully reviews the top candidates with deeper analysis, reordering them so the best rise to the top.
How Reranking Improves Precision
Reranking applies a more sophisticated model (often a cross-encoder) to the retrieved candidates, computing a more accurate relevance score by analyzing the query and document together. This is computationally expensive, which is why you only do it on the top-K results, not your entire corpus.
The result: higher precision in your final top-N, meaning the documents you actually use are more relevant to the query.
Key Takeaway: The reranking problem recognizes that fast, high-recall retrieval followed by expensive, high-precision reranking delivers better final results than trying to achieve perfect ranking in a single retrieval step.