Course contentsShow
← Machine Learning and Deep Learning
Lesson 227 of 3,5383. Linear RegressionFree lesson

L1 Regularization and Lasso Regression

Using the absolute value of coefficients as a penalty to encourage sparsity and feature selection.

L1 Regularization and Lasso Regression

What You'll Learn

You'll discover how L1 regularization uses absolute values to penalize coefficients, automatically selecting the most important features by forcing others to zero.

The Core Idea

While Ridge regression (L2) shrinks coefficients smoothly, L1 regularization takes a different approach: it adds the absolute value of coefficients as a penalty to the loss function. This seemingly small change has a dramatic effect—L1 can drive some coefficients all the way to exactly zero, effectively removing features from your model.

The modified loss function looks like this:

Loss = MSE + λ × (|β₁| + |β₂| + ... + |βₙ|)

Where λ (lambda) controls regularization strength, just like in Ridge regression.

Why Absolute Values Matter

Think of it this way: Ridge regression is like gently taxing all your features based on their squared size—everyone pays proportionally. L1 regularization is like imposing a flat fee per feature—at some point, keeping small features isn't worth the cost, so they get eliminated entirely.

This property makes Lasso (Least Absolute Shrinkage and Selection Operator) incredibly valuable when you have many features but suspect only a few truly matter. It performs automatic feature selection during training.

When to Use Lasso

  • You have high-dimensional data (many features)
  • You believe only a subset of features are truly relevant
  • You want an interpretable model with fewer non-zero coefficients
  • You need built-in feature selection

Unlike Ridge, Lasso doesn't have a simple closed-form solution due to the absolute value function, so it requires iterative optimization methods.

Key Takeaway: L1 regularization (Lasso) penalizes the absolute value of coefficients, driving some to exactly zero, which provides automatic feature selection alongside overfitting prevention.