Course contentsShow
← AI Engineering
Lesson 394 of 1,88610. RAG - Improving RetrievalFree lesson

Cross-Encoder Models for Reranking

How cross-encoders work and why they're more accurate than bi-encoders for relevance scoring.

Cross-Encoder Models for Reranking

What you'll learn: How cross-encoder models work and why they deliver superior accuracy compared to bi-encoders when scoring relevance between queries and documents.


How Bi-Encoders vs Cross-Encoders Differ

In earlier lessons, you've used bi-encoders (embedding models) that encode queries and documents separately into vector space. They're fast because documents are pre-encoded, and retrieval is just nearest-neighbor search.

Cross-encoders take a fundamentally different approach: they process the query and each candidate document together as a single input pair. Think of it like this:

  • Bi-encoder: "Does this apple look like this orange?" (comparing two separate photos)
  • Cross-encoder: "How similar are this apple and orange when I look at them side-by-side?" (examining both together)

How Cross-Encoders Work

A cross-encoder concatenates your query and a candidate document, feeds them into a transformer model (like BERT), and outputs a single relevance score. The attention mechanism sees both texts simultaneously, allowing it to understand fine-grained relationships between query terms and document content.

Why this matters for reranking: After your initial retrieval (using fast bi-encoders) returns, say, 100 candidates, you can pass the top 20 through a cross-encoder. The cross-encoder rescores them with much higher precision, promoting truly relevant results to the top.

The Trade-off

Cross-encoders are more accurate because they see query-document interactions directly. But they're slower — you can't pre-compute document representations. That's why they're perfect for reranking a small set of candidates, not searching millions of documents.

Key Takeaway: Cross-encoders rerank by jointly processing query-document pairs for precise relevance scoring, delivering higher accuracy than bi-encoders at the cost of speed — making them ideal as a second-stage reranker in RAG pipelines.