Contrastive Loss Functions
What You'll Learn
You'll learn the mathematical formulations behind InfoNCE, NT-Xent, and triplet loss—three powerful loss functions that teach models to pull similar examples together and push dissimilar ones apart in embedding space.
Core Idea
Contrastive learning needs a scoring mechanism to decide what's "good" (positive pairs close together) and "bad" (negatives far apart). The loss function is the mathematical recipe that guides this learning.
Think of it like teaching a bouncer to recognize VIPs: you show them who belongs together (positives) and who doesn't (negatives), then penalize mistakes until they learn the pattern.
Three Key Loss Functions
Triplet Loss
The original formulation uses anchor, positive, and negative examples:
L = max(0, d(anchor, positive) - d(anchor, negative) + margin)
The model learns to keep positives closer than negatives by at least some margin. If the constraint is already satisfied, loss = 0.
InfoNCE (Noise Contrastive Estimation)
Used in contrastive learning frameworks, it treats one positive and many negatives as a classification problem:
L = -log(exp(sim(anchor, positive)) / Σ exp(sim(anchor, negative_i)))
This is essentially a softmax over similarities—maximize the positive's probability among all candidates.
NT-Xent (Normalized Temperature-Scaled Cross Entropy)
A temperature-scaled variant used in SimCLR and CLIP:
L = -log(exp(sim(i,j)/τ) / Σ exp(sim(i,k)/τ))
Temperature τ controls how sharp the distribution is—lower values make the model more confident.
Why These Matter
These losses create discriminative embeddings where semantically similar items cluster together, enabling zero-shot transfer and semantic search in multimodal models.
Key Takeaway: InfoNCE, NT-Xent, and triplet loss are mathematical frameworks that teach models to create meaningful embedding spaces by contrasting positive pairs against negatives, forming the backbone of modern vision-language pretraining.