Course contentsShow
← Computer Science
Lesson 2811 of 2,87262. Machine Learning FundamentalsFree lesson

L1 Regularization (Lasso)

Understanding L1 regularization that adds absolute value of weights to the loss function, promoting sparsity.

L1 Regularization (Lasso)

What you'll learn: How L1 regularization combats overfitting by encouraging models to use fewer features through sparsity.

The Core Idea

L1 regularization, also known as Lasso (Least Absolute Shrinkage and Selection Operator), is a technique that modifies your loss function by adding a penalty based on the absolute values of your model's weights. Instead of just minimizing prediction error, your model now minimizes:

Total Loss = Original Loss + λ × (|w₁| + |w₂| + ... + |wₙ|)

Where λ (lambda) controls how strongly you penalize large weights, and each |wᵢ| is the absolute value of a weight.

Why Absolute Values Matter

Think of L1 as a strict budget officer who wants to eliminate unnecessary expenses entirely. Because you're penalizing the absolute magnitude of weights, L1 regularization tends to push many weights all the way to exactly zero. This creates sparse models—models that effectively ignore many features.

For example, if you're predicting house prices with 100 features, L1 might discover that only 15 features actually matter and set the other 85 weights to zero. This isn't just simpler—it's automatic feature selection built into your optimization process!

The Sparsity Advantage

Sparse models are:

  • Interpretable: Fewer active features means easier understanding
  • Efficient: Less computation at prediction time
  • Less prone to overfitting: Complexity is naturally constrained

The λ hyperparameter lets you control this tradeoff: higher values create sparser models (more aggressive regularization), while lower values allow more features to remain active.

Key Takeaway: L1 regularization adds the sum of absolute weight values to your loss function, creating sparse models by driving many weights to exactly zero—effectively performing automatic feature selection while preventing overfitting.