K-Means Objective Function
What you'll learn: How K-Means measures clustering quality by minimizing the within-cluster sum of squares (WCSS).
The Core Idea
Remember how K-Means assigns points to clusters and updates centroids? But how does it know if it's doing a good job? The objective function is K-Means' scoring system—it measures how tightly grouped points are within their assigned clusters.
The metric is called Within-Cluster Sum of Squares (WCSS), also known as inertia. It calculates the sum of squared distances from each point to its assigned cluster centroid:
WCSS = Σ Σ ||x - μ_k||²
Where:
xis each data pointμ_kis the centroid of cluster k||·||²means squared Euclidean distance
Why Squared Distance?
Just like in linear regression, squaring has benefits:
- Penalizes points far from their centroid more heavily
- Makes the math differentiable and smooth
- Always positive (no cancellation)
The Minimization Process
K-Means minimizes WCSS through its two alternating steps:
- Assignment step: Assign points to nearest centroid (reduces WCSS)
- Update step: Move centroids to cluster means (reduces WCSS further)
Each iteration guarantees WCSS never increases—though it may converge to a local minimum, not necessarily the global best.
Analogy
Think of WCSS as measuring "total unhappiness" where each person (point) is unhappy based on how far they live from their community center (centroid). K-Means rearranges communities to minimize total unhappiness.
Key Takeaway: K-Means minimizes WCSS (inertia)—the sum of squared distances between points and their cluster centroids—to create tight, cohesive clusters.