Course contentsShow
← Machine Learning and Deep Learning
Lesson 228 of 3,5383. Linear RegressionFree lesson

Lasso vs Ridge: Geometric Intuition

Visualizing constraint regions and understanding why lasso produces sparse solutions while ridge does not.

Lasso vs Ridge: Geometric Intuition

You'll learn why lasso regression often sets coefficients exactly to zero while ridge regression rarely does, using geometric visualization.

The Geometric Picture

Imagine you're trying to minimize your regression error while staying within a "budget" constraint on coefficient sizes. This creates two competing goals:

  1. The error contours: Elliptical shapes representing equal error levels (like elevation lines on a map)
  2. The constraint region: A boundary limiting how large your coefficients can be

The magic happens where these two meet—that's your regularized solution.

Why the Shapes Matter

Ridge (L2) constraint region: Forms a circle (or sphere in higher dimensions). This smooth, round boundary means when the error contours touch it, they typically make contact at a point where both coefficients are non-zero—just shrunk.

Lasso (L1) constraint region: Forms a diamond (or diamond-like polytope in higher dimensions) with sharp corners at the axes. When error contours expand outward seeking the constraint boundary, they often hit these pointy corners first—where one or more coefficients equal exactly zero!

The Sparse Solution Insight

Those sharp corners in lasso's diamond are positioned exactly where coordinates become zero (on the axes). Since many error contours will touch the diamond at these corners, lasso naturally produces sparse solutions—models where some features are completely excluded (coefficient = 0).

Ridge's smooth circle has no such corners, so coefficients shrink toward zero but rarely land exactly on it.

Key Takeaway: Lasso's diamond-shaped constraint region has sharp corners at zero-coefficient points, making sparse solutions geometrically likely, while ridge's circular region lacks these corners and keeps all features with small non-zero weights.