Course contentsShow
← AI Engineering
Lesson 428 of 1,88611. RAG - Improving GenerationFree lesson

Cross-Encoder Relevance Scoring

Use cross-encoder models to score query-document pairs for more accurate relevance than bi-encoders.

Cross-Encoder Relevance Scoring

You'll learn how to use cross-encoder models to score query-document pairs more accurately than traditional bi-encoders.

The Difference Between Bi-Encoders and Cross-Encoders

By now, you've used embedding models (bi-encoders) that encode queries and documents separately, then compare their vectors with cosine similarity. While fast, this approach misses important interaction signals between your query and each document.

Cross-encoders work differently: they take both the query and a candidate document as a single input, processing them together through the model. This allows the model to understand how specific terms in the query relate to specific passages in the document.

How Cross-Encoders Improve Relevance

Think of it like the difference between comparing two people's dating profiles separately versus watching them have an actual conversation. The bi-encoder approach compares profiles (embeddings) independently. The cross-encoder approach observes how they interact together.

Cross-encoders output a single relevance score (typically 0-1) indicating how well the document answers the query. This scoring is much more nuanced because the model sees both pieces of text simultaneously.

When to Use Cross-Encoders

Cross-encoders are too slow for initial retrieval over millions of documents, but perfect for reranking your top candidates. The typical pattern:

  1. Use fast bi-encoder retrieval to get top-k candidates (e.g., 100 documents)
  2. Use cross-encoder to rescore just those candidates
  3. Return the top-n highest-scored documents (e.g., 5)

Popular cross-encoder models include cross-encoder/ms-marco-MiniLM-L-6-v2 for general relevance scoring.

Key Takeaway: Cross-encoders score query-document pairs by processing them together, providing more accurate relevance scores than bi-encoder similarity but at higher computational cost—ideal for reranking retrieved candidates.