Late Interaction Models (ColBERT)
What you'll learn: How ColBERT achieves the efficiency of bi-encoders while preserving the accuracy of cross-encoders through delayed token-level matching.
The Hybrid Approach
So far, you've learned that bi-encoders encode queries and documents separately (fast but less accurate), while cross-encoders process them together (accurate but slow). ColBERT (Contextualized Late Interaction over BERT) cleverly splits the difference.
The key innovation: encode queries and documents independently like bi-encoders, but instead of producing a single vector per text, preserve all token embeddings. Then at retrieval time, perform fine-grained interactions between query tokens and document tokens.
How It Works
Think of it like a restaurant menu search:
- Bi-encoder: Represents the entire menu as one summary vector. Fast, but loses details.
- Cross-encoder: Reads your question alongside each menu item together. Accurate, but must re-read everything for each query.
- ColBERT: Pre-processes each menu item into detailed ingredient-level descriptions. When you ask a question, it quickly compares your query words against those detailed descriptions without re-reading the full menu.
ColBERT computes a MaxSim operation: for each query token, find its maximum similarity with any document token, then sum these scores. This captures which query terms match which document parts.
The Benefits
You get document representations you can pre-compute and index (like bi-encoders), but with token-level precision that approaches cross-encoder quality. The retrieval is slower than pure bi-encoders but much faster than cross-encoders, making it practical for large-scale search.
Key Takeaway: ColBERT bridges bi-encoder efficiency and cross-encoder accuracy by encoding texts separately but deferring fine-grained token interactions until retrieval time, achieving a sweet spot for semantic search at scale.