Directional Derivatives
What you'll learn: How to measure how fast a function changes when you move in any direction you choose, not just along coordinate axes.
The Core Idea
You already know that partial derivatives tell you how a function changes when you move along the x-axis or y-axis separately. But what if you want to walk northeast? Or in some custom direction defined by your problem?
A directional derivative answers: "If I start at point a and walk in the direction of unit vector u, how fast does my function change?"
Think of it like this: imagine standing on a hillside (your function). Partial derivatives tell you the slope if you walk due east or due north. But you might want to know the slope if you walk toward a specific landmark at an angle. That's what directional derivatives compute.
The Formula
The directional derivative of function f at point a in direction u is:
D_u f(a) = ∇f(a) · u
Where:
∇f(a)is the gradient vector at point a (contains all partial derivatives)- u is a unit vector (length = 1) pointing in your chosen direction
·is the dot product
The gradient points in the direction of steepest ascent. When you dot it with u, you're essentially projecting the gradient onto your chosen direction—finding out how much of that "steepness" aligns with where you want to go.
Why It Matters
In machine learning, optimization algorithms often need to explore how loss changes in specific search directions, not just along parameter axes. Understanding directional derivatives helps you grasp why gradient descent works and how different directions affect convergence.
Key Takeaway: The directional derivative computes rate of change in any direction by taking the dot product of the gradient with a unit direction vector.