Course contentsShow
← Machine Learning and Deep Learning
Lesson 2603 of 3,53856. Few-Shot and Meta-LearningFree lesson

Distance Metrics and Embedding Dimensions

Explore choices between Euclidean and cosine distance, and how embedding dimensionality affects few-shot performance.

Distance Metrics and Embedding Dimensions

What you'll learn: How choosing the right distance metric and embedding size directly impacts few-shot learning performance.

The Core Problem

In few-shot learning, your model compares embeddings of new examples against support set examples. But how you measure similarity matters enormously. You've seen architectures like Siamese, Prototypical, and Relation Networks—now we explore the fundamental choices that make them work well (or fail).

Distance Metrics: Euclidean vs Cosine

Euclidean distance measures straight-line separation in embedding space: sqrt(sum((x - y)²)). It's sensitive to magnitude—larger vectors appear farther apart even if they point in similar directions.

Cosine distance (or similarity) measures the angle between vectors: 1 - (x·y)/(||x|| ||y||). It ignores magnitude and focuses purely on directional alignment.

When to use which?

  • Cosine works better when embeddings have varying norms but consistent direction patterns (common in language and normalized image embeddings)
  • Euclidean works well when magnitude carries semantic meaning or embeddings are already normalized
  • In practice, cosine distance often outperforms Euclidean in few-shot classification because it's more robust to scale variations

Embedding Dimensionality

The size of your embedding vector (64, 128, 512 dimensions) creates a trade-off:

Lower dimensions:

  • Faster computation
  • Less prone to overfitting with limited data
  • May lack capacity to distinguish fine-grained differences

Higher dimensions:

  • More expressive representations
  • Risk of overfitting in few-shot scenarios where support sets are tiny
  • The "curse of dimensionality"—distances become less meaningful in very high dimensions

Most few-shot methods use embeddings between 64-512 dimensions, with smaller datasets favoring the lower end.

Key Takeaway: Cosine distance often outperforms Euclidean in few-shot learning by focusing on directional similarity, and choosing embedding dimensions requires balancing expressiveness against overfitting risk—typically 128-256 dimensions work well for most few-shot tasks.