Course contentsShow
← Machine Learning and Deep Learning
Lesson 2599 of 3,53856. Few-Shot and Meta-LearningFree lesson

Hard Negative Mining

Discover strategies for selecting informative negative examples that accelerate metric learning convergence.

Hard Negative Mining

What you'll learn: How to strategically select the most challenging negative examples to dramatically improve metric learning convergence and model performance.

The Core Problem

When training Siamese or Triplet networks, not all negative examples are equally useful. Imagine learning to distinguish dogs from cats: once you've mastered separating a poodle from a goldfish, that pair teaches you nothing new. But distinguishing a husky from a wolf? That's informative.

Hard negatives are examples that are close to your anchor in embedding space but should be pushed away. These are the "almost mistakes" your model might make—the challenging cases that force better representations.

Why It Matters

In a typical batch, most negative pairs are easy negatives—already far apart in embedding space. The contrastive or triplet loss barely changes for these. Your model wastes computation on examples it already handles well.

Hard negative mining focuses training on informative negatives: examples that are close to the anchor but belong to different classes. This accelerates convergence because every gradient update addresses actual weaknesses.

Mining Strategies

Online mining (within a batch):

  • For each anchor, find the negative example with the smallest distance in embedding space
  • Select "semi-hard" negatives: closer than positives but still outside the margin

Offline mining (across epochs):

  • Periodically scan the full dataset for hard negatives
  • Cache challenging pairs for focused training

The key insight: by concentrating on boundary cases where your model struggles, you build more discriminative embeddings faster than random sampling ever could.

Key Takeaway: Hard negative mining selects the most challenging negative examples—those close to the anchor in embedding space—to maximize learning efficiency and build stronger metric spaces for few-shot tasks.