Course contentsShow
← Machine Learning and Deep Learning
Lesson 322 of 3,5387. Instance-Based and Probabilistic ClassifiersFree lesson

The K-Nearest Neighbors Algorithm

Step-by-step mechanics: measuring distances to all training points, finding K nearest neighbors, and majority voting for classification.

The K-Nearest Neighbors Algorithm

What you'll learn: How K-Nearest Neighbors makes predictions by finding the closest training examples and letting them vote on the answer.

Core Idea

K-Nearest Neighbors (KNN) is wonderfully simple: to classify a new data point, find the K training examples that are closest to it, then take a majority vote of their labels. If 7 out of your 10 nearest neighbors are "cats," you predict "cat."

Think of it like asking your K nearest neighbors in a theater what movie they liked. If most of them loved it, you'll probably love it too.

Step-by-Step Mechanics

1. Measure Distance to All Training Points

When a new example arrives, calculate its distance to every point in your training set. Common distance metrics include Euclidean distance (straight-line) or Manhattan distance (grid-based).

2. Find the K Nearest Neighbors

Sort all training points by distance and select the K closest ones. The choice of K is a hyperparameter you decide (e.g., K=5 means you consider the 5 nearest neighbors).

3. Majority Voting for Classification

Look at the labels of those K neighbors. Count how many belong to each class. The class with the most votes wins and becomes your prediction.

Example: For a new patient's medical record, find K=7 nearest patients. If 5 were diagnosed "healthy" and 2 were "sick," predict "healthy."

Why It Works

KNN assumes that similar inputs have similar outputs. Points close together in feature space likely share the same label—a natural assumption for many real-world problems.

Key Takeaway: K-Nearest Neighbors classifies by measuring distances to all training points, selecting the K nearest, and using majority voting among their labels—no complex math required, just proximity and democracy.