Course contentsShow
← Machine Learning and Deep Learning
Lesson 1978 of 3,53842. Retrieval-Augmented Generation: FoundationsFree lesson

Cross-Encoders for Reranking

Learn how cross-encoders jointly process query-document pairs for higher accuracy at the cost of computational efficiency.

Cross-Encoders for Reranking

What you'll learn: How cross-encoders jointly process query-document pairs to dramatically improve retrieval accuracy as a second-stage reranking step.

The Core Idea

In multi-stage retrieval, bi-encoders (like Sentence Transformers) quickly retrieve candidate documents by comparing pre-computed embeddings. But they encode query and document separately, missing rich interaction signals. Cross-encoders solve this by processing the query and document together in a single forward pass, enabling the model to capture nuanced relevance patterns.

Think of it like dating: a bi-encoder creates separate profiles and matches based on compatibility scores computed independently. A cross-encoder is like an actual date—the two people interact directly, revealing chemistry that profiles alone can't capture.

How Cross-Encoders Work

Instead of producing separate embeddings, a cross-encoder takes the concatenated [query, document] pair as input and outputs a single relevance score. The transformer's attention mechanism can model token-level interactions between query and document, capturing:

  • Exact phrase matches in context
  • Semantic relationships between query terms and document passages
  • Negations, qualifiers, and subtle meaning shifts

This joint encoding is computationally expensive—you must run inference for every query-document pair. That's why cross-encoders work best as a reranker: use a fast bi-encoder to retrieve the top 100-1000 candidates, then rerank the top 20-50 with a cross-encoder for maximum accuracy.

The Trade-off

Cross-encoders deliver superior accuracy but cannot pre-compute document representations. For every query, you need fresh inference on all candidate pairs. Bi-encoders are fast but less precise; cross-encoders are slow but highly accurate.

Key Takeaway: Cross-encoders jointly encode query-document pairs for fine-grained relevance scoring, making them ideal second-stage rerankers after fast bi-encoder retrieval—accuracy at the cost of computational efficiency.