Course contentsShow
← Machine Learning and Deep Learning
Lesson 474 of 3,53811. Model Evaluation MetricsFree lesson

Huber Loss and Robust Metrics

Learn Huber loss as a compromise between MAE and MSE, combining robustness to outliers with smooth gradients for optimization.

Huber Loss and Robust Metrics

What you'll learn: How Huber loss combines the best of both worlds—MSE's smooth gradients and MAE's robustness to outliers—for more stable model training.

The Problem with Pure MSE or MAE

You've learned that MSE heavily penalizes large errors (squaring them), making it sensitive to outliers, while MAE treats all errors equally, making optimization harder because its gradient is constant. What if you could get MSE's smooth, informative gradients and MAE's outlier resistance?

What Is Huber Loss?

Huber loss is a hybrid metric that acts like MSE for small errors and like MAE for large errors. It uses a threshold parameter called delta (δ) to decide when to switch:

  • For errors ≤ δ: Use squared error (like MSE) — smooth gradients help optimization
  • For errors > δ: Use absolute error (like MAE) — prevents outliers from dominating

Think of it like a shock absorber in a car: gentle bumps (small errors) are handled smoothly, but big potholes (outliers) don't wreck the whole system.

The Mathematical Intuition

The formula transitions smoothly between quadratic and linear behavior. When your prediction is slightly off, Huber loss guides the model with strong, proportional corrections. When your prediction is wildly off (an outlier), it caps the penalty growth, preventing one bad sample from throwing off your entire model.

Choosing Delta

The δ parameter controls the transition point. Smaller δ makes Huber loss more like MAE (more robust), while larger δ makes it more like MSE (more sensitive to all errors). This tunability is powerful for real-world data.

Key Takeaway: Huber loss is a robust compromise metric that combines MSE's optimization-friendly gradients with MAE's outlier resistance, controlled by a tunable threshold parameter delta.