Optimization Problems with Constraints
You'll learn why finding maximum or minimum values becomes more complex when variables must satisfy an additional equation.
The Core Challenge
Until now, you've learned to optimize functions by finding critical points using gradients and the second derivative test. But what happens when your variables can't roam freely? What if they must satisfy a relationship like x² + y² = 1 (staying on a circle) or x + y + z = 10 (lying on a plane)?
This is constrained optimization: finding extreme values of a function f(x, y) subject to a constraint equation g(x, y) = c.
Why Gradients Alone Don't Work
When you're unconstrained, setting ∇f = 0 works because you can move freely in any direction. But with a constraint, you can only move along the curve or surface defined by g(x, y) = c.
Analogy: Imagine you want to find the highest point in a city (maximize elevation). Without constraints, you scan everywhere for flat spots (∇f = 0). But if you must stay on a specific road, the highest point on that road might occur where the road is still sloping—just not in the direction the road allows you to travel.
The constraint restricts your feasible region. The maximum or minimum might occur:
- Where the gradient ∇f would naturally be zero, if that point happens to satisfy the constraint
- Or more commonly, somewhere along the constraint curve where you simply can't improve by moving in the only allowed directions
What This Means
You need new techniques that respect the constraint. Simply finding where ∇f = 0 ignores the restriction entirely and may give you irrelevant points that violate g(x, y) = c.
Key Takeaway: In constrained optimization, gradient-based methods for unconstrained problems fail because optimal points often occur where the gradient is not zero, but where movement along the constraint cannot improve the objective function.