Course contentsShow
← AI Engineering
Lesson 240 of 1,8866. Embeddings and Semantic SearchFree lesson

Contrastive Learning for Embeddings

Learn the fundamentals of contrastive learning: positive/negative pairs and how models learn to cluster similar items.

Contrastive Learning for Embeddings

What you'll learn: How embedding models learn to place similar items close together and dissimilar items far apart in vector space.

The Core Idea

Contrastive learning is the training method that teaches embedding models what "similar" means. Instead of just memorizing vectors, models learn by comparing examples: pulling similar items (positive pairs) closer in vector space while pushing dissimilar items (negative pairs) farther apart.

Think of it like teaching someone to organize a library. You show them: "These two mystery novels go together" (positive pair), and "Keep this cookbook away from the mystery section" (negative pair). Over thousands of examples, they learn the organizing principle.

Positive and Negative Pairs

Positive pairs are items that should have similar embeddings:

  • A question and its correct answer
  • Two paraphrases of the same sentence
  • A document and its summary
  • Product title and description

Negative pairs are items that should have different embeddings:

  • A question paired with an irrelevant answer
  • Sentences about completely different topics
  • Random documents from your collection

How Models Learn

During training, the model sees a batch of these pairs. It computes embeddings and adjusts its weights to:

  1. Minimize distance between positive pairs (bring them closer)
  2. Maximize distance between negative pairs (push them apart)

The quality of your negatives matters enormously. Random negatives are easy to distinguish. Hard negatives—items that look similar but aren't—teach the model to make fine-grained distinctions.

Key Takeaway: Contrastive learning trains embeddings by explicitly teaching models to cluster similar items together and separate dissimilar ones through positive and negative example pairs.