Preparing Training Data
You'll learn how to create structured training pairs that teach an embedding model to better understand similarity in your specific domain.
The Core Idea
Fine-tuning embeddings requires showing the model examples of what you consider "similar" or "different." Your training data consists of pairs of text items labeled with their relationship:
- Positive pairs: items that should be close in vector space (similar queries and answers, duplicate questions, related documents)
- Negative pairs: items that should be far apart (unrelated questions, different topics)
Think of it like teaching a child which animals are "similar" — you show them many examples: "A tiger and a lion are similar, but a tiger and a butterfly are not."
Where Training Data Comes From
Labeled data: If you have existing classifications, tags, or categories, items with the same label form positive pairs. Different labels? Negative pairs.
User interactions: Click-through data is gold. If a user searches "reset password" and clicks on document #17, that's a positive pair. Documents they didn't click? Potential negatives.
Synthetic generation: Use your existing model or another LLM to generate questions for answers, paraphrases of queries, or similar content variations. This scales quickly when you lack interaction data.
Structuring the Pairs
Each training example typically looks like:
- Query/anchor text
- Positive match text
- (Optional) Negative match text
The contrastive learning process you'll use later needs this triplet structure to learn what "closer" and "farther" mean in your domain.
Key Takeaway: Training data for embeddings consists of labeled pairs showing what should be similar or different, sourced from labels, user behavior, or synthetic generation — preparing this carefully determines fine-tuning success.