Course contentsShow
← Machine Learning and Deep Learning
Lesson 323 of 3,5387. Instance-Based and Probabilistic ClassifiersFree lesson

Distance Metrics for KNN

Euclidean, Manhattan, Minkowski, and cosine distances, and how metric choice affects neighborhood structure and predictions.

Distance Metrics for KNN

What you'll learn: How different ways of measuring "closeness" between data points fundamentally shape KNN's predictions.

Why Distance Matters

In K-Nearest Neighbors, everything hinges on one question: which points are "nearest"? But "nearness" isn't as obvious as it seems. The way you measure distance between points completely changes which neighbors get selected—and therefore what your model predicts.

Common Distance Metrics

Euclidean Distance

The straight-line "as the crow flies" distance. For two points in 2D: sqrt((x₁-x₂)² + (y₁-y₂)²). This is your intuitive everyday distance and works well when all features have similar scales and importance.

Analogy: Walking diagonally across a park.

Manhattan Distance

The "city block" distance—only horizontal and vertical moves allowed: |x₁-x₂| + |y₁-y₂|. Useful when movement is constrained or features shouldn't be mixed geometrically.

Analogy: Navigating a grid of streets in Manhattan.

Minkowski Distance

A generalization that includes both Euclidean (p=2) and Manhattan (p=1) as special cases. Higher values of p emphasize larger differences more strongly.

Cosine Distance

Measures the angle between points (as vectors from the origin), not their magnitude. Perfect for text data or when direction matters more than absolute values: 1 - cos(θ).

Analogy: Two arrows pointing in similar directions are "close," regardless of length.

Impact on Predictions

Different metrics create different neighborhood structures. Euclidean might find 5 neighbors in a circular region; Manhattan finds them in a diamond shape; cosine ignores magnitude entirely. A point could be nearest under one metric but distant under another—completely changing your prediction.

Key Takeaway: The distance metric is a critical hyperparameter in KNN; choosing between Euclidean, Manhattan, Minkowski, or cosine fundamentally alters which neighbors influence predictions and should match your data's geometry.