Course contentsShow
← Computer Science
Lesson 2864 of 2,87263. Classical ML AlgorithmsFree lesson

Distance Metrics for KNN

Learn Euclidean, Manhattan, and Minkowski distances and how metric choice affects neighbor selection.

Distance Metrics for KNN

What you'll learn: How different distance formulas determine which neighbors KNN considers "closest" to a data point.

The Core Idea

KNN relies entirely on measuring closeness between data points. But "close" isn't one-size-fits-all. Different distance metrics quantify similarity in different ways, and your choice fundamentally changes which neighbors get selected—and thus what predictions your model makes.

Three Essential Metrics

Euclidean Distance

The "straight-line" distance you know from geometry: the shortest path between two points. For points p and q in n-dimensional space:

d = √[(p₁-q₁)² + (p₂-q₂)² + ... + (pₙ-qₙ)²]

Analogy: "As the crow flies" distance on a map.

Manhattan Distance

Also called "taxicab" or "city block" distance—sum of absolute differences along each dimension:

d = |p₁-q₁| + |p₂-q₂| + ... + |pₙ-qₙ|

Analogy: Walking distance in a grid-like city where you can't cut diagonally through buildings.

Minkowski Distance

A generalization with parameter p:

d = (|p₁-q₁|ᵖ + |p₂-q₂|ᵖ + ... + |pₙ-qₙ|ᵖ)^(1/p)

When p=1, it's Manhattan. When p=2, it's Euclidean. Higher p values increasingly emphasize the largest coordinate difference.

Why Metric Choice Matters

  • Euclidean works well when all features are equally scaled and contribute smoothly
  • Manhattan is more robust to outliers and works better in high-dimensional spaces
  • Minkowski lets you tune sensitivity to coordinate differences

Different metrics define different "neighborhoods"—the same K neighbors may differ entirely based on how you measure distance.

Key Takeaway: The distance metric you choose for KNN directly controls neighbor selection; Euclidean, Manhattan, and Minkowski offer different ways to quantify similarity based on your data's characteristics.