Course contentsShow
← Computer Science
Lesson 2866 of 2,87263. Classical ML AlgorithmsFree lesson

KNN Classification: Voting Mechanisms

Learn majority voting and weighted voting strategies for combining neighbor predictions.

KNN Classification: Voting Mechanisms

You'll learn how K-Nearest Neighbors combines predictions from multiple neighbors using majority and weighted voting strategies.

Why Voting Matters

Once KNN identifies the K nearest neighbors to a query point, it needs a strategy to combine their class labels into a single prediction. The voting mechanism determines how these neighbors "vote" for the final classification.

Majority Voting (Uniform Voting)

The simplest approach: each of the K neighbors gets one equal vote for their class label. The class with the most votes wins.

Example: For K=5, if three neighbors are "cat" and two are "dog", the prediction is "cat" (3 votes > 2 votes).

Think of it like a simple democracy—every neighbor's opinion counts equally, regardless of how close or far they are.

Tie-breaking: When classes tie, common strategies include:

  • Choose the class of the single nearest neighbor
  • Pick randomly among tied classes
  • Use a predefined class priority

Weighted Voting

More sophisticated: neighbors closer to the query point get more influence. Each neighbor's vote is weighted by the inverse of its distance.

Weight formula: weight = 1 / distance (or 1 / distance² for steeper decay)

This makes intuitive sense—a neighbor 0.1 units away should matter more than one 5 units away, even if both are in the top K.

Example: For K=3 with distances [0.5, 1.0, 2.0] and labels ["cat", "dog", "cat"]:

  • Cat votes: 1/0.5 + 1/2.0 = 2.5
  • Dog votes: 1/1.0 = 1.0
  • Prediction: "cat"

Key Takeaway: Majority voting treats all K neighbors equally, while weighted voting gives closer neighbors more influence—often improving accuracy when the optimal K includes some distant, less-relevant points.