Course contentsShow
← Machine Learning and Deep Learning
Lesson 224 of 3,5383. Linear RegressionFree lesson

L2 Regularization and Ridge Regression

Adding the squared magnitude of coefficients to the loss function to shrink parameters toward zero.

L2 Regularization and Ridge Regression

What You'll Learn

You'll discover how L2 regularization prevents overfitting by penalizing large coefficient values through Ridge Regression.

The Core Idea

Remember from our regularization introduction that we want to discourage overly complex models? L2 regularization adds a penalty term to our loss function based on the squared magnitude of all model coefficients.

Instead of just minimizing:

MSE = average of (y_true - y_pred)²

Ridge Regression minimizes:

Loss = MSE + λ × (sum of all coefficient squares)

The λ (lambda) is a hyperparameter controlling penalty strength. Higher λ means stronger regularization.

Why Square the Coefficients?

Think of L2 regularization like a magnetic force pulling all coefficients toward zero. By squaring them:

  • Large coefficients get penalized much more than small ones (squaring amplifies differences)
  • The penalty is always positive (no matter if coefficient is negative or positive)
  • All coefficients shrink proportionally — large ones shrink faster, but none reach exactly zero

The Shrinking Effect

Imagine you're packing a suitcase with a weight limit. L2 regularization is like saying: "You can bring items, but heavier items cost you exponentially more allowance." You'll naturally prefer lighter items and avoid packing anything unnecessarily heavy.

Similarly, Ridge Regression keeps all features but makes their coefficients smaller and more reasonable, reducing model complexity without eliminating features entirely.

When to Use Ridge

Use Ridge Regression when you suspect overfitting (your training R² is much better than validation R²) and you want to keep all features in your model with reduced influence.

Key Takeaway: L2 regularization (Ridge Regression) prevents overfitting by adding the squared sum of coefficients to the loss function, shrinking all parameters toward zero without eliminating them.