Course contentsShow
← Machine Learning and Deep Learning
Lesson 1328 of 3,53829. Core NLP TasksFree lesson

Contrastive Learning for Embeddings

How contrastive loss functions train models to pull similar texts together and push dissimilar texts apart in embedding space.

Contrastive Learning for Embeddings

What You'll Learn

You'll discover how contrastive loss functions train embedding models to automatically group similar texts close together while pushing dissimilar ones apart in vector space.

The Core Idea

Think of contrastive learning like organizing a library by reading comprehension: books on similar topics should sit near each other, while unrelated books stay far apart. That's exactly what contrastive loss does for text embeddings.

When training embedding models like Sentence Transformers, contrastive learning works with pairs or triplets of examples:

  • Positive pairs: Similar texts (e.g., a question and its correct answer, or two paraphrases)
  • Negative pairs: Dissimilar texts (e.g., unrelated sentences)

The loss function mathematically rewards the model when:

  1. Similar texts have embeddings that are close in vector space (small distance)
  2. Dissimilar texts have embeddings that are far apart (large distance)

This creates a powerful training signal: the model learns to encode semantic meaning into the geometry of the embedding space itself.

Why This Matters for Bi-Encoders

Remember that bi-encoders encode texts independently and compare embeddings via similarity measures. Contrastive learning is the training method that teaches them what "similar" means. Without it, embeddings would be arbitrary—contrastive loss gives them semantic structure.

Common contrastive loss functions include:

  • Triplet loss: Anchor, positive, negative
  • InfoNCE loss: Used in many modern systems
  • Multiple Negatives Ranking Loss: Efficient batch-based training

Key Takeaway: Contrastive learning trains embeddings by pulling similar examples together and pushing dissimilar ones apart, creating semantically meaningful vector spaces perfect for retrieval and comparison tasks.