Course contentsShow
← Machine Learning and Deep Learning
Lesson 2594 of 3,53856. Few-Shot and Meta-LearningFree lesson

Few-Shot Learning Benchmarks

Exploring standard datasets like miniImageNet and tieredImageNet for evaluating few-shot methods.

Few-Shot Learning Benchmarks

What you'll learn: You'll discover the standard datasets used to fairly evaluate and compare few-shot learning methods.

Why We Need Benchmarks

Just as track and field athletes compete on standardized courses, few-shot learning methods need consistent testing grounds. Without shared benchmarks, it's impossible to know whether your Prototype Network genuinely outperforms a Matching Network or if one researcher just got lucky with easier data.

The Standard Benchmarks

miniImageNet

miniImageNet is the most popular few-shot learning benchmark. It's a subset of ImageNet containing:

  • 100 classes (sampled from the full ImageNet dataset)
  • 600 images per class (84×84 pixels, downsampled for efficiency)
  • Typically split: 64 training classes, 16 validation, 20 test classes

Think of it as a "starter pack" from ImageNet—large enough to be challenging, small enough to train quickly during meta-training episodes.

tieredImageNet

tieredImageNet offers a more sophisticated split:

  • 608 classes from ImageNet
  • Organized hierarchically by ImageNet's semantic tree
  • Training/validation/test classes come from different high-level categories

The key insight: tieredImageNet ensures training classes (e.g., "vehicles") don't overlap semantically with test classes (e.g., "animals"). This tests whether your model truly learns to learn, rather than memorizing visual features that happen to transfer.

Why These Benchmarks Matter

These datasets let researchers:

  1. Report standardized 5-way 1-shot and 5-way 5-shot accuracy
  2. Compare methods fairly (same support/query splits)
  3. Track progress in the field over time

Key Takeaway: miniImageNet and tieredImageNet provide standardized testing environments where few-shot methods compete on equal footing—miniImageNet offers accessible quick evaluation, while tieredImageNet tests true generalization across semantic boundaries.