Course contentsShow
← Machine Learning and Deep Learning
Lesson 1330 of 3,53829. Core NLP TasksFree lesson

Hard Negatives Mining

Improve embedding quality by sampling difficult negative examples that are semantically close but not relevant.

Hard Negatives Mining

What you'll learn: How to strengthen your embedding models by strategically selecting challenging negative examples that push the model to make finer distinctions.

The Core Problem

When training embedding models for semantic search using contrastive learning, you need positive pairs (semantically similar) and negative pairs (dissimilar). A naive approach randomly samples negatives: if your query is "best pizza in Chicago," random negatives might be completely unrelated like "quantum physics equations."

But random negatives are too easy. The model quickly learns to separate them without developing nuanced understanding.

What Are Hard Negatives?

Hard negatives are semantically close but not actually relevant examples. They're tricky cases that force your model to learn subtle differences.

For the query "best pizza in Chicago," hard negatives might be:

  • "top-rated Italian restaurants in Chicago" (close topic, wrong focus)
  • "best pizza in New York" (right food, wrong city)
  • "Chicago deep dish recipe" (related but different intent)

These are hard because they share vocabulary and concepts with the positive answer, but shouldn't match.

Why This Matters

Hard negatives mining dramatically improves embedding quality because:

  • The model learns to distinguish fine-grained semantic differences
  • It prevents the model from relying on superficial keyword overlap
  • Training becomes more efficient—each example teaches more

The Mining Process

During training, you periodically:

  1. Use your current model to encode a pool of candidates
  2. Find examples with high similarity scores to your query but aren't true matches
  3. Use these as negatives in your next training batch

Key Takeaway: Hard negatives mining improves semantic embeddings by forcing models to learn from difficult, semantically-close-but-incorrect examples rather than easy random ones, leading to more discriminative representations.