The Mean Squared Error Loss Function
What you'll learn: How Mean Squared Error (MSE) quantifies prediction quality by averaging squared residuals, and why squaring matters for optimization.
Why We Need a Loss Function
You've learned that linear regression creates predictions, and residuals measure the difference between predictions and actual values. But how do we judge if our model is good? We need a single number that summarizes all prediction errors across our dataset—that's where the loss function comes in.
Deriving Mean Squared Error
MSE takes each residual (prediction error), squares it, then averages them all:
MSE = (1/n) × Σ(yᵢ - ŷᵢ)²
Where:
n= number of data pointsyᵢ= actual valueŷᵢ= predicted value(yᵢ - ŷᵢ)= residual for point i
Think of it like grading a test: you look at each wrong answer (residual), but instead of just counting mistakes, you square them and find the average.
Why Square the Errors?
Squaring serves three critical purposes:
-
Eliminates sign issues: A prediction that's 5 units too high and one that's 5 units too low shouldn't cancel out—both are equally bad. Squaring makes all errors positive.
-
Penalizes large errors more: A residual of 10 contributes 100 to MSE, while five residuals of 2 each contribute only 20 total. This pushes optimization toward models that avoid catastrophic mistakes.
-
Makes optimization smooth: Squared functions are convex (remember from optimization lessons!), which means gradient descent can reliably find the global minimum—no local optima traps.
Key Takeaway: MSE converts all your prediction errors into a single, optimization-friendly number by squaring residuals (making them positive and penalizing large errors) and averaging them, giving us a smooth target for gradient descent to minimize.