Course contentsShow
← Machine Learning and Deep Learning
Lesson 614 of 3,53815. Neural Network FoundationsFree lesson

Mean Squared Error for Regression

Learning MSE loss for continuous predictions, its mathematical form, gradient properties, and sensitivity to outliers.

Mean Squared Error for Regression

What you'll learn: How Mean Squared Error (MSE) quantifies the quality of regression predictions and why it's the most common loss function for continuous outputs.

What Is MSE?

Mean Squared Error measures the average squared difference between your model's predictions and the actual target values. Think of it as asking: "How far off are my predictions, on average, when I square each error to penalize larger mistakes more heavily?"

For a dataset with n examples, where ŷᵢ is your prediction and yᵢ is the true value:

MSE = (1/n) × Σ(yᵢ - ŷᵢ)²

Why Square the Errors?

Squaring serves three purposes:

  1. Makes all errors positive — otherwise positive and negative errors would cancel out
  2. Penalizes large errors heavily — an error of 10 contributes 100 to the loss, while an error of 1 only contributes 1
  3. Creates smooth gradients — the derivative is clean and proportional to the error, making gradient-based optimization straightforward

Gradient Properties

MSE has a convenient mathematical property: its gradient with respect to predictions is simply 2(ŷᵢ - yᵢ). This linear relationship means the gradient grows proportionally with error magnitude, naturally pushing larger errors toward correction more aggressively during training.

The Outlier Problem

MSE's squaring mechanism is both its strength and weakness. While it accelerates learning for typical errors, outliers get squared too, potentially dominating your loss. A single mislabeled example with error 100 contributes 10,000 to your loss—more than 100 examples with error 10 each (which contribute only 10,000 combined).

Key Takeaway: MSE is the standard regression loss function that squares prediction errors, providing smooth gradients for optimization but remaining highly sensitive to outliers due to its quadratic penalty.