Cross-Encoder Relevance Scoring
You'll learn how to use cross-encoder models to score query-document pairs more accurately than traditional bi-encoders.
The Difference Between Bi-Encoders and Cross-Encoders
By now, you've used embedding models (bi-encoders) that encode queries and documents separately, then compare their vectors with cosine similarity. While fast, this approach misses important interaction signals between your query and each document.
Cross-encoders work differently: they take both the query and a candidate document as a single input, processing them together through the model. This allows the model to understand how specific terms in the query relate to specific passages in the document.
How Cross-Encoders Improve Relevance
Think of it like the difference between comparing two people's dating profiles separately versus watching them have an actual conversation. The bi-encoder approach compares profiles (embeddings) independently. The cross-encoder approach observes how they interact together.
Cross-encoders output a single relevance score (typically 0-1) indicating how well the document answers the query. This scoring is much more nuanced because the model sees both pieces of text simultaneously.
When to Use Cross-Encoders
Cross-encoders are too slow for initial retrieval over millions of documents, but perfect for reranking your top candidates. The typical pattern:
- Use fast bi-encoder retrieval to get top-k candidates (e.g., 100 documents)
- Use cross-encoder to rescore just those candidates
- Return the top-n highest-scored documents (e.g., 5)
Popular cross-encoder models include cross-encoder/ms-marco-MiniLM-L-6-v2 for general relevance scoring.
Key Takeaway: Cross-encoders score query-document pairs by processing them together, providing more accurate relevance scores than bi-encoder similarity but at higher computational cost—ideal for reranking retrieved candidates.