Course contentsShow
← Machine Learning and Deep Learning
Lesson 1975 of 3,53842. Retrieval-Augmented Generation: FoundationsFree lesson

Training Data for Retrieval Models

Explore how positive and negative pairs are created from question-answer datasets, clickthrough data, and synthetic generation.

Training Data for Retrieval Models

You'll learn: How retrieval models like Sentence Transformers learn to distinguish relevant from irrelevant passages through carefully constructed positive and negative training pairs.

The Core Idea

Retrieval models trained with contrastive learning (as you learned earlier) need to see examples of what matches and what doesn't. The training data consists of positive pairs (query + relevant document) and negative pairs (query + irrelevant document). The quality and diversity of these pairs directly determines how well your retrieval system performs.

Sources of Training Pairs

Question-Answer Datasets

Datasets like MS MARCO or Natural Questions provide queries with known relevant passages. The question becomes the query, the passage containing the answer is a positive, and random other passages serve as negatives (called "random negatives").

Clickthrough Data

Real user behavior is gold. When someone searches "best Italian restaurants NYC" and clicks on a result, that's an implicit positive pair. Pages they skipped? Weak negatives. This captures actual information needs.

Synthetic Generation

You can create training data by:

  • Generating questions from documents using an LLM
  • Paraphrasing existing queries for data augmentation
  • Creating "hard negatives" by retrieving top-K results with BM25, then marking non-answers as challenging negatives

Why Negatives Matter

Random negatives are too easy—the model learns surface patterns. Hard negatives (passages that look relevant but aren't) force the model to learn semantic understanding. Think of it like studying: multiple-choice questions with obviously wrong answers teach you less than ones with plausible distractors.

Key Takeaway: Retrieval models require positive pairs (query + relevant doc) and especially hard negative pairs (query + plausible-but-wrong doc) from sources like QA datasets, user clicks, or synthetic generation to learn meaningful semantic similarity.