Course contentsShow
← Machine Learning and Deep Learning
Lesson 47 of 3,5381. Mathematical Foundations for Machine LearningFree lesson

Second Derivative Test in Multiple Dimensions

Using the Hessian to classify critical points as maxima, minima, or saddle points.

Second Derivative Test in Multiple Dimensions

What you'll learn: How to use the Hessian matrix to determine whether a critical point is a minimum, maximum, or saddle point in multivariable functions.

The Core Idea

Remember how the second derivative of a single-variable function tells you if a critical point is a minimum (curve opens upward) or maximum (opens downward)? The Hessian matrix does the same job in multiple dimensions, but instead of checking a single value, we examine properties of the entire matrix.

At a critical point (where the gradient is zero), the Hessian matrix acts like a multidimensional "curvature detector":

Classification Rules

  1. Positive definite Hessian → The function curves upward in all directions → Local minimum
  2. Negative definite Hessian → The function curves downward in all directions → Local maximum
  3. Indefinite Hessian → The function curves up in some directions, down in others → Saddle point
  4. Semidefinite → The test is inconclusive

An Analogy

Imagine standing at a critical point on a landscape. The Hessian tells you about the terrain in every direction around you:

  • If all directions slope upward → you're in a valley (minimum)
  • If all directions slope downward → you're on a hilltop (maximum)
  • If some directions go up while others go down → you're on a mountain pass or saddle

Why This Matters in ML

When training neural networks, we search for minima of loss functions. The Hessian helps us understand whether we've found a good minimum or gotten stuck at a saddle point (common in high-dimensional spaces).

Key Takeaway: The Second Derivative Test uses the definiteness of the Hessian matrix to classify critical points—positive definite means minimum, negative definite means maximum, and indefinite means saddle point.