Course contentsShow
← Computer Science
Lesson 2868 of 2,87263. Classical ML AlgorithmsFree lesson

Feature Scaling in KNN

Learn why normalization or standardization is critical for distance-based methods like KNN.

Feature Scaling in KNN

You'll learn why KNN fails without proper feature scaling and how normalization fixes distance calculations.

Why KNN Demands Scaled Features

KNN relies entirely on distance metrics (Euclidean, Manhattan, etc.) to find nearest neighbors. Here's the problem: if your features have wildly different scales, the distance calculation becomes dominated by the large-scale feature, effectively ignoring smaller-scale ones.

The Scale Domination Problem

Imagine predicting house prices using two features:

  • Square footage: ranges from 500 to 5000 (large numbers)
  • Number of bedrooms: ranges from 1 to 5 (small numbers)

When computing Euclidean distance, the squared difference in square footage might be (3000 - 2000)² = 1,000,000, while bedrooms contribute only (3 - 2)² = 1. The distance is essentially only measuring square footage—bedrooms are invisible to the algorithm.

The Solution: Normalization or Standardization

Feature scaling brings all features to comparable ranges:

  • Min-max normalization: Scales features to [0, 1] range
  • Standardization (z-score): Centers features around mean=0, std=1

After scaling, both features contribute meaningfully to distance calculations. A house with 2000 sq ft and 3 bedrooms is now properly compared to one with 2100 sq ft and 4 bedrooms—both dimensions matter equally.

When to Apply It

Always scale features for KNN before splitting train/test data, or fit the scaler on training data only and transform both sets. Never fit on test data—that causes data leakage.

Key Takeaway: KNN's distance-based predictions require feature scaling (normalization or standardization) to prevent large-scale features from drowning out smaller ones; without it, your model ignores important information.