Course contentsShow
← Machine Learning and Deep Learning
Lesson 1327 of 3,53829. Core NLP TasksFree lesson

Bi-Encoders vs Cross-Encoders

Learn the trade-offs: bi-encoders encode queries and documents separately; cross-encoders jointly but are slower.

Bi-Encoders vs Cross-Encoders

What you'll learn: You'll understand when to use bi-encoders versus cross-encoders for semantic search and text similarity tasks, and the crucial speed-versus-accuracy trade-off between them.

The Core Difference

Imagine you're comparing two documents. A bi-encoder processes each document independently through separate (or shared) neural networks, producing fixed embeddings. You then compare these embeddings using simple operations like cosine similarity. A cross-encoder, on the other hand, concatenates both documents and feeds them together through a single network that directly outputs a similarity score.

Bi-Encoders: Fast But Independent

Bi-encoders are the foundation of Sentence Transformers. They encode your query once and your document collection once. Because documents are encoded independently, you can:

  • Pre-compute and store all document embeddings
  • Compare millions of documents quickly using vector similarity
  • Scale efficiently for large-scale retrieval

Limitation: The query and document never "see" each other during encoding, missing subtle interactions.

Cross-Encoders: Accurate But Slow

Cross-encoders process query and document jointly, allowing full attention between every word pair. This captures nuanced relationships bi-encoders miss.

Limitation: You cannot pre-compute embeddings. Each query-document pair requires a fresh forward pass through the entire network—too slow for searching millions of documents.

The Practical Workflow

Most production systems use both: bi-encoders retrieve the top-k candidates quickly (e.g., top 100 from millions), then cross-encoders re-rank those candidates for maximum accuracy.

Key Takeaway: Bi-encoders excel at fast, large-scale retrieval by encoding independently; cross-encoders provide superior accuracy through joint encoding but are too slow for initial search—use bi-encoders first, then re-rank with cross-encoders.