Multi-Stage Retrieval: Bi-Encoders
What you'll learn: How bi-encoder architecture separately encodes queries and documents to enable lightning-fast similarity search across millions of documents.
The Bi-Encoder Architecture
A bi-encoder uses two independent neural networks (often the same architecture with shared or separate weights) to encode queries and documents into dense vector embeddings—separately. Unlike models that process query-document pairs jointly, bi-encoders create standalone representations for each piece of text.
Think of it like pre-translating every book in a library into a universal language. When someone asks a question (also translated), you simply compare translations rather than reading every book from scratch each time.
Why Bi-Encoders Enable Multi-Stage Retrieval
In retrieval systems handling millions of documents, you cannot afford to process every query-document pair through a complex model. Bi-encoders solve this by:
- Pre-computing document embeddings once during indexing
- Encoding the query at search time (milliseconds)
- Computing similarity via fast vector operations (cosine similarity, dot product)
This separation makes bi-encoders the first stage in multi-stage retrieval pipelines—they quickly narrow millions of candidates down to a manageable top-K set (say, 100 documents) that more expensive models can then re-rank.
The Architecture in Practice
Both the query encoder and document encoder typically share the same base model (like a Sentence Transformer). They map text to fixed-dimensional vectors (e.g., 768-dim) in a shared embedding space where semantic similarity corresponds to vector proximity.
The magic happens during contrastive training with hard negatives—teaching the model to place relevant query-document pairs close together while pushing irrelevants apart.
Key Takeaway: Bi-encoders enable scalable retrieval by encoding queries and documents independently, allowing pre-computation of document embeddings and fast similarity-based search as the first stage in efficient multi-stage pipelines.