Course contentsShow
← Machine Learning and Deep Learning
Lesson 326 of 3,5387. Instance-Based and Probabilistic ClassifiersFree lesson

Weighted KNN and Distance Weighting

Giving closer neighbors more influence through inverse distance weighting to improve prediction quality at boundaries.

Weighted KNN and Distance Weighting

What You'll Learn

You'll discover how to make KNN smarter by giving closer neighbors more say in predictions, especially near decision boundaries.

The Core Idea

In standard KNN, all K neighbors get an equal vote—whether they're right next door or relatively far away. Weighted KNN improves this by giving closer neighbors more influence using inverse distance weighting.

Think of asking friends for restaurant recommendations. You'd probably trust your neighbor who lives on the same street more than someone from across town, even if both went to the restaurant. That's exactly what weighted KNN does with data points.

How Inverse Distance Weighting Works

Instead of each neighbor contributing equally (weight = 1), each neighbor's contribution is proportional to how close it is:

Weight = 1 / distance

or sometimes:

Weight = 1 / distance²

This means:

  • A neighbor at distance 0.5 has weight 2 (or 4 with squared distance)
  • A neighbor at distance 2 has weight 0.5 (or 0.25 with squared distance)
  • Closer neighbors dominate the prediction

Why This Matters

Weighted KNN shines at decision boundaries—those tricky regions where classes meet. Without weighting, a test point near the boundary might be swayed by distant neighbors from the wrong class. With distance weighting, nearby points (which are more likely to share the same class) have stronger influence, leading to cleaner, more accurate boundaries.

This is especially valuable when K is larger, preventing faraway outliers from diluting the signal from your true nearest neighbors.

Key Takeaway: Weighted KNN uses inverse distance to give closer neighbors more voting power, improving prediction accuracy especially at class boundaries where standard KNN struggles.