Course contentsShow
← Machine Learning and Deep Learning
Lesson 46 of 3,5381. Mathematical Foundations for Machine LearningFree lesson

The Hessian Matrix

Understanding second-order partial derivatives organized as the Hessian matrix.

The Hessian Matrix

What you'll learn: How second-order partial derivatives organize into a matrix that reveals the curvature of multivariable functions.

What Is the Hessian Matrix?

You've learned that the gradient vector collects all first-order partial derivatives of a function. The Hessian matrix takes this one step further—it collects all second-order partial derivatives.

For a function f(x₁, x₂, ..., xₙ), the Hessian is an n×n symmetric matrix where each entry is a second partial derivative:

H[i,j] = ∂²f / (∂xᵢ ∂xⱼ)

For example, with f(x, y), the Hessian is:

H = [ ∂²f/∂x²    ∂²f/(∂x∂y) ]
    [ ∂²f/(∂y∂x)  ∂²f/∂y²    ]

Why Second Derivatives Matter

Think of the gradient as telling you the slope of a hill. The Hessian tells you the curvature—whether you're in a valley (curving upward), on a peak (curving downward), or on a saddle point (curving both ways).

  • Diagonal entries (like ∂²f/∂x²) measure how the slope changes in each individual direction
  • Off-diagonal entries (like ∂²f/(∂x∂y)) capture how changing one variable affects the rate of change with respect to another

The Hessian's symmetry comes from a mathematical theorem: mixed partial derivatives are equal (∂²f/(∂x∂y) = ∂²f/(∂y∂x)) under mild smoothness conditions.

Connection to Critical Points

At critical points you found earlier (where gradient = 0), the Hessian determines whether you've found a minimum, maximum, or saddle point—concepts you'll use extensively when optimizing machine learning models.

Key Takeaway: The Hessian matrix organizes all second-order partial derivatives, revealing the curvature of a multivariable function and helping classify critical points.