Course contentsShow
← Machine Learning and Deep Learning
Lesson 2005 of 3,53843. Retrieval-Augmented Generation: Advanced TechniquesFree lesson

Cross-Encoder Rerankers

Using BERT-based models to score query-document pairs jointly for more accurate relevance.

Cross-Encoder Rerankers

You'll learn how cross-encoder models process query-document pairs jointly to dramatically improve retrieval accuracy in RAG systems.

The Joint Scoring Advantage

After your initial retrieval (vector search, BM25, or hybrid), you have dozens of potentially relevant documents. But which ones truly answer the query best? Cross-encoders solve this by scoring query-document pairs together, rather than separately.

Unlike bi-encoders (which embed queries and documents independently), a cross-encoder takes both the query and a candidate document as a single input to a BERT-based model. This allows deep attention mechanisms to identify subtle relationships between query terms and document content that independent embeddings miss.

How Cross-Encoders Work

Think of it like a judge who reads both the question and each answer side-by-side, rather than reading them separately and trying to match later. The model sees: [CLS] query text [SEP] document text [SEP] and outputs a relevance score.

This joint processing captures:

  • Term interactions: How query words relate to document phrases
  • Semantic nuance: Context-dependent meanings
  • Exact matches: When they matter most

The trade-off: Cross-encoders are too slow for first-stage retrieval (you can't score millions of documents), but perfect for reranking the top 50-100 candidates retrieved by faster methods.

The Reranking Pipeline

  1. Fast retrieval returns top-K candidates (e.g., K=100)
  2. Cross-encoder scores each query-document pair
  3. Resort by cross-encoder scores
  4. Return top-N to the LLM (e.g., N=5)

This two-stage approach combines speed (bi-encoders/BM25) with accuracy (cross-encoders).

Key Takeaway: Cross-encoder rerankers use BERT-based models to jointly score query-document pairs, achieving superior relevance ranking at the cost of higher latency—ideal as a second-stage reranker after fast initial retrieval.