Course contentsShow
← Mathematics
Lesson 2298 of 3,12147. Linear Algebra — Applications and Advanced TopicsFree lesson

Applications to Optimization and Hessian Matrices

Use positive definiteness of the Hessian to identify local minima in multivariable optimization.

Applications to Optimization and Hessian Matrices

What you'll learn: How the positive definiteness of the Hessian matrix helps you conclusively identify local minima in multivariable optimization problems.

The Core Idea

When optimizing a function of several variables—like finding the lowest cost in production or the best fit for a model—you first find critical points where all first-order partial derivatives equal zero. But how do you know if that critical point is a minimum, maximum, or saddle point?

This is where the Hessian matrix comes in. The Hessian is a square matrix containing all the second-order partial derivatives of your function. Think of it as the multivariable generalization of the "second derivative test" from single-variable calculus.

The Test for Local Minima

At a critical point, examine the Hessian matrix:

  • If the Hessian is positive definite, you've found a local minimum
  • If the Hessian is negative definite, you've found a local maximum
  • If the Hessian is indefinite, you've found a saddle point
  • If the Hessian is semidefinite, the test is inconclusive

Why does this work? A positive definite Hessian means that in every direction away from your critical point, the quadratic approximation of your function curves upward—just like a bowl. You're at the bottom!

Practical Application

You can verify positive definiteness using:

  • Eigenvalue criterion: All eigenvalues must be positive
  • Sylvester's criterion: All leading principal minors must be positive
  • Cholesky decomposition: The matrix admits a Cholesky factorization if and only if it's positive definite

This connects spectral properties of matrices directly to geometric properties of optimization landscapes.

Key Takeaway: The Hessian matrix's positive definiteness at a critical point guarantees a local minimum, bridging quadratic forms, eigenvalue analysis, and practical optimization.