Distance Metrics for KNN
What you'll learn: How different distance formulas determine which neighbors KNN considers "closest" to a data point.
The Core Idea
KNN relies entirely on measuring closeness between data points. But "close" isn't one-size-fits-all. Different distance metrics quantify similarity in different ways, and your choice fundamentally changes which neighbors get selected—and thus what predictions your model makes.
Three Essential Metrics
Euclidean Distance
The "straight-line" distance you know from geometry: the shortest path between two points. For points p and q in n-dimensional space:
d = √[(p₁-q₁)² + (p₂-q₂)² + ... + (pₙ-qₙ)²]
Analogy: "As the crow flies" distance on a map.
Manhattan Distance
Also called "taxicab" or "city block" distance—sum of absolute differences along each dimension:
d = |p₁-q₁| + |p₂-q₂| + ... + |pₙ-qₙ|
Analogy: Walking distance in a grid-like city where you can't cut diagonally through buildings.
Minkowski Distance
A generalization with parameter p:
d = (|p₁-q₁|ᵖ + |p₂-q₂|ᵖ + ... + |pₙ-qₙ|ᵖ)^(1/p)
When p=1, it's Manhattan. When p=2, it's Euclidean. Higher p values increasingly emphasize the largest coordinate difference.
Why Metric Choice Matters
- Euclidean works well when all features are equally scaled and contribute smoothly
- Manhattan is more robust to outliers and works better in high-dimensional spaces
- Minkowski lets you tune sensitivity to coordinate differences
Different metrics define different "neighborhoods"—the same K neighbors may differ entirely based on how you measure distance.
Key Takeaway: The distance metric you choose for KNN directly controls neighbor selection; Euclidean, Manhattan, and Minkowski offer different ways to quantify similarity based on your data's characteristics.