Course contentsShow
← Machine Learning and Deep Learning
Lesson 229 of 3,5383. Linear RegressionFree lesson

Elastic Net: Combining L1 and L2

Blending ridge and lasso penalties to balance sparsity with correlated feature handling.

Elastic Net: Combining L1 and L2

What You'll Learn

You'll discover how Elastic Net merges the strengths of Ridge (L2) and Lasso (L1) regularization to handle both correlated features and automatic feature selection.

The Problem with Choosing Between Ridge and Lasso

Remember that Ridge regression shrinks coefficients smoothly but keeps all features, while Lasso can zero out coefficients (creating sparsity) but struggles when features are highly correlated. What if you need both capabilities?

Elastic Net: The Best of Both Worlds

Elastic Net adds both L1 and L2 penalty terms to the cost function, controlled by two hyperparameters:

  • α (alpha): Overall regularization strength
  • l1_ratio: Balance between L1 and L2 (ranges from 0 to 1)

The penalty becomes:

α × [l1_ratio × |coefficients| + (1 - l1_ratio) × coefficients²]

When l1_ratio = 1.0, you get pure Lasso. When l1_ratio = 0.0, you get pure Ridge. Values in between blend both penalties.

Why This Combination Works

Think of Elastic Net as a diplomatic negotiator:

  • The L1 component performs feature selection, zeroing out irrelevant features
  • The L2 component handles groups of correlated features gracefully, keeping them together instead of arbitrarily picking one

This makes Elastic Net particularly valuable when you have many features, some irrelevant and some correlated, which is common in real-world datasets.

When to Use Elastic Net

Choose Elastic Net when:

  • You have correlated features and want sparsity
  • You're unsure whether Ridge or Lasso is better
  • You have more features than training samples

Key Takeaway: Elastic Net combines L1 and L2 regularization penalties, giving you both feature selection (sparsity) and stable handling of correlated features—ideal when you need the strengths of both Ridge and Lasso.