Partial Derivatives: Introduction
You'll learn how to find the rate of change in functions with multiple inputs by examining one variable at a time.
Why We Need Partial Derivatives
So far, you've worked with derivatives of single-variable functions like f(x)—functions with one input and one output. But in machine learning, functions almost always depend on multiple variables. For example, a loss function might depend on dozens, thousands, or even millions of parameters (weights).
Partial derivatives extend the derivative concept to multivariable functions by answering: "How does the output change when I tweak just ONE input variable, while keeping all others fixed?"
The Core Idea
Think of adjusting the temperature and humidity in a greenhouse to maximize plant growth. Growth depends on both variables: Growth = f(temperature, humidity).
A partial derivative with respect to temperature asks: "If I change only the temperature by a tiny amount (holding humidity constant), how much does growth change?" The partial derivative with respect to humidity asks the same question, but varies humidity while freezing temperature.
Mathematically, if f(x, y) is a function of two variables:
- The partial derivative with respect to x, written ∂f/∂x, treats y as a constant and differentiates normally
- The partial derivative with respect to y, written ∂f/∂y, treats x as a constant
Example Without Code
For f(x, y) = x² + 3xy + y²:
- ∂f/∂x = 2x + 3y (treat y as a constant, differentiate terms with x)
- ∂f/∂y = 3x + 2y (treat x as a constant, differentiate terms with y)
Key Takeaway: Partial derivatives measure how a multivariable function changes along one direction at a time, forming the foundation for gradient descent and neural network training.