Course contentsShow
← Machine Learning and Deep Learning
Lesson 615 of 3,53815. Neural Network FoundationsFree lesson

Mean Absolute Error and Huber Loss

Exploring MAE for robust regression and Huber loss as a compromise between MSE and MAE for outlier resilience.

Mean Absolute Error and Huber Loss

What you'll learn: How MAE and Huber loss provide more robust alternatives to MSE when your data contains outliers.

The Problem with MSE's Squaring

You've learned that Mean Squared Error (MSE) penalizes errors quadratically. While this works well for clean data, it becomes extremely sensitive to outliers. A single data point far from the prediction can dominate the loss because squaring amplifies large errors dramatically.

Mean Absolute Error (MAE)

Mean Absolute Error takes the absolute value of errors instead of squaring them:

MAE = (1/n) × Σ|y_actual - y_predicted|

Analogy: If MSE is like penalizing someone exponentially more for being farther off target, MAE treats all distances proportionally. Being twice as far off costs exactly twice as much.

Key advantages:

  • More robust to outliers (large errors don't explode)
  • Easier to interpret (same units as your target variable)

Trade-off: MAE's gradient is constant, which can make optimization less smooth near the minimum.

Huber Loss: The Best of Both Worlds

Huber loss is a hybrid that acts like MSE for small errors (smooth gradients) and like MAE for large errors (outlier protection):

  • For errors smaller than threshold δ: behaves like MSE
  • For errors larger than δ: behaves like MAE

This gives you smooth, fast convergence while protecting against outliers that would otherwise dominate training.

When to use what:

  • MSE: Clean data, you want to heavily penalize large errors
  • MAE: Outliers present, you want equal treatment of all errors
  • Huber: Best general-purpose choice when you're unsure about outliers

Key Takeaway: MAE provides outlier-robust regression through linear error penalization, while Huber loss combines MSE's smooth optimization with MAE's outlier resilience by switching behavior based on error magnitude.