The Gradient Vector
What You'll Learn
You'll learn how to combine all partial derivatives of a multivariable function into a single vector that points toward the function's steepest increase.
The Core Idea
Imagine you're standing on a hillside and want to know which direction leads uphill most steeply. The gradient vector answers exactly that question for mathematical functions.
When you have a function with multiple inputs (like f(x, y) or f(x₁, x₂, ..., xₙ)), you can compute a partial derivative for each input variable. Each partial derivative tells you how the function changes when you wiggle just that one variable.
The gradient vector simply assembles all these partial derivatives into a single vector:
For a function f(x, y):
∇f = [∂f/∂x, ∂f/∂y]
For a function with n variables f(x₁, x₂, ..., xₙ):
∇f = [∂f/∂x₁, ∂f/∂x₂, ..., ∂f/∂xₙ]
The symbol ∇ (nabla or "del") denotes the gradient.
Why It Matters
The gradient vector has a powerful geometric property: it points in the direction of steepest ascent. Its magnitude tells you how steep that direction is. If you walk in the direction of the gradient, the function value increases fastest.
Conversely, walking in the opposite direction (negative gradient) leads to the steepest descent—which is exactly what machine learning algorithms do during optimization!
Analogy
Think of the gradient as a compass on a mountain. Instead of pointing north, it always points toward the peak. Each component of the vector tells you how much to move in each coordinate direction to climb most efficiently.
Key Takeaway: The gradient vector combines all partial derivatives into a single vector that points in the direction of steepest ascent, forming the foundation for optimization in machine learning.