Two-Tower Models for Candidate Generation
What you'll learn: How to build separate neural network towers for users and items to enable lightning-fast retrieval from millions of candidates in large-scale recommender systems.
The Core Idea
Imagine you're Netflix trying to recommend movies from a catalog of 100,000 titles. Running every user-item pair through a complex neural network would be impossibly slow. Two-tower models solve this by building two separate neural networks (towers): one encodes users into embedding vectors, another encodes items into embedding vectors—both mapping to the same vector space.
The magic? Once trained, you can:
- Pre-compute and store all item embeddings offline
- At inference time, encode only the current user
- Find top candidates using fast nearest-neighbor search (like cosine similarity or approximate methods)
This transforms an expensive prediction problem into a fast retrieval problem.
How It Works
User Tower: Takes user features (ID, demographics, history) → outputs user embedding vector
Item Tower: Takes item features (ID, metadata, content) → outputs item embedding vector
Training: The model learns to place users and items they interact with close together in embedding space, while pushing non-interacted pairs apart. You typically use dot product similarity: score = user_embedding · item_embedding
This builds on the embedding layers and neural collaborative filtering you've seen, but explicitly separates the computation paths for scalability.
Why This Matters
At scale (millions of items), you cannot afford to run full neural networks on every candidate. Two-tower models let you:
- Retrieve top 100-1000 candidates in milliseconds
- Feed these candidates to a more complex ranking model
- Update item embeddings without re-scoring every user
Key Takeaway: Two-tower models decouple user and item encoding into separate networks, enabling efficient large-scale candidate retrieval by pre-computing item embeddings and using fast similarity search at serving time.