Course contentsShow
← Machine Learning and Deep Learning
Lesson 43 of 3,5381. Mathematical Foundations for Machine LearningFree lesson

Directional Derivatives

Computing rate of change in arbitrary directions using the gradient and direction vectors.

Directional Derivatives

What you'll learn: How to measure how fast a function changes when you move in any direction you choose, not just along coordinate axes.

The Core Idea

You already know that partial derivatives tell you how a function changes when you move along the x-axis or y-axis separately. But what if you want to walk northeast? Or in some custom direction defined by your problem?

A directional derivative answers: "If I start at point a and walk in the direction of unit vector u, how fast does my function change?"

Think of it like this: imagine standing on a hillside (your function). Partial derivatives tell you the slope if you walk due east or due north. But you might want to know the slope if you walk toward a specific landmark at an angle. That's what directional derivatives compute.

The Formula

The directional derivative of function f at point a in direction u is:

D_u f(a) = ∇f(a) · u

Where:

  • ∇f(a) is the gradient vector at point a (contains all partial derivatives)
  • u is a unit vector (length = 1) pointing in your chosen direction
  • · is the dot product

The gradient points in the direction of steepest ascent. When you dot it with u, you're essentially projecting the gradient onto your chosen direction—finding out how much of that "steepness" aligns with where you want to go.

Why It Matters

In machine learning, optimization algorithms often need to explore how loss changes in specific search directions, not just along parameter axes. Understanding directional derivatives helps you grasp why gradient descent works and how different directions affect convergence.

Key Takeaway: The directional derivative computes rate of change in any direction by taking the dot product of the gradient with a unit direction vector.