The Hessian Matrix
What you'll learn: How second-order partial derivatives organize into a matrix that reveals the curvature of multivariable functions.
What Is the Hessian Matrix?
You've learned that the gradient vector collects all first-order partial derivatives of a function. The Hessian matrix takes this one step further—it collects all second-order partial derivatives.
For a function f(x₁, x₂, ..., xₙ), the Hessian is an n×n symmetric matrix where each entry is a second partial derivative:
H[i,j] = ∂²f / (∂xᵢ ∂xⱼ)
For example, with f(x, y), the Hessian is:
H = [ ∂²f/∂x² ∂²f/(∂x∂y) ]
[ ∂²f/(∂y∂x) ∂²f/∂y² ]
Why Second Derivatives Matter
Think of the gradient as telling you the slope of a hill. The Hessian tells you the curvature—whether you're in a valley (curving upward), on a peak (curving downward), or on a saddle point (curving both ways).
- Diagonal entries (like ∂²f/∂x²) measure how the slope changes in each individual direction
- Off-diagonal entries (like ∂²f/(∂x∂y)) capture how changing one variable affects the rate of change with respect to another
The Hessian's symmetry comes from a mathematical theorem: mixed partial derivatives are equal (∂²f/(∂x∂y) = ∂²f/(∂y∂x)) under mild smoothness conditions.
Connection to Critical Points
At critical points you found earlier (where gradient = 0), the Hessian determines whether you've found a minimum, maximum, or saddle point—concepts you'll use extensively when optimizing machine learning models.
Key Takeaway: The Hessian matrix organizes all second-order partial derivatives, revealing the curvature of a multivariable function and helping classify critical points.