Course contentsShow
← Machine Learning and Deep Learning
Lesson 1973 of 3,53842. Retrieval-Augmented Generation: FoundationsFree lesson

Contrastive Training for Embedding Models

Understand how contrastive loss trains embeddings to place similar texts close and dissimilar texts far apart in vector space.

Contrastive Training for Embedding Models

What You'll Learn

You'll understand how contrastive loss trains embedding models to push similar texts together and pull dissimilar texts apart in vector space.

The Core Idea

Contrastive training is the secret sauce behind powerful embedding models like Sentence Transformers. Instead of just learning to predict words or classify documents, these models learn relationships between texts.

Here's how it works: during training, the model sees pairs or triplets of examples:

  • Positive pairs: Similar texts (e.g., a question and its answer, or two paraphrases)
  • Negative pairs: Dissimilar texts (e.g., unrelated sentences)

The contrastive loss function rewards the model when it places positive pairs close together in vector space and penalizes it when similar texts end up far apart. Simultaneously, it pushes negative pairs away from each other.

The Magnetic Analogy

Think of embeddings as magnets on a 2D surface. Contrastive training adjusts the "magnetic charge" of similar texts so they attract each other, while dissimilar texts repel. Over thousands of training examples, the model learns a vector space where semantic similarity directly translates to geometric proximity.

Why This Matters for Retrieval

When you later perform vector search in a RAG system, you query the database with an embedded question. Because contrastive training organized the space meaningfully, the nearest neighbors are genuinely relevant documents—not just lexically similar ones.

Key Takeaway: Contrastive training creates semantically meaningful vector spaces by explicitly teaching the model to cluster similar texts and separate dissimilar ones, making dense retrieval possible.