Course contentsShow
← Computer Science
Lesson 2867 of 2,87263. Classical ML AlgorithmsFree lesson

KNN for Regression: Averaging Predictions

Understand how KNN computes continuous outputs by averaging or weighting neighbor values.

KNN for Regression: Averaging Predictions

What you'll learn: How K-Nearest Neighbors produces continuous numerical predictions by combining values from nearby training examples.

From Voting to Averaging

You've seen KNN for classification use voting among neighbors. For regression tasks (predicting continuous values like house prices or temperatures), KNN takes a different approach: it averages the target values of the K nearest neighbors.

The Core Mechanism

When predicting a continuous output for a new data point:

  1. Find the K nearest neighbors using your chosen distance metric
  2. Extract their actual target values (labels)
  3. Compute the average (mean) of those K values
  4. Return this average as your prediction

Analogy: Imagine estimating a house's price. You find the 5 most similar houses (same size, location, age). Instead of voting on a category, you average their actual sale prices: (300K+300K + 310K + 295K+295K + 305K + 290K)÷5=290K) ÷ 5 = 300K.

Weighted Averaging (Optional Extension)

Some implementations use distance-weighted averaging, where closer neighbors contribute more to the prediction. A neighbor at distance d might receive weight 1/d, giving nearby points stronger influence than distant ones.

prediction = Σ(weight_i × value_i) / Σ(weight_i)

This refines predictions by acknowledging that the closest neighbor probably provides better information than the K-th farthest one.

Why It Works

Averaging smooths out individual anomalies. If one neighbor is an outlier, the other K-1 neighbors balance it out, producing a stable estimate based on local patterns in your feature space.

Key Takeaway: KNN regression predicts continuous values by averaging (or distance-weighting) the target values of the K nearest training examples, turning local neighborhood information into a numerical estimate.