The Multivariable Chain Rule
What You'll Learn
You'll learn how to differentiate composite functions with multiple variables—the mathematical backbone of how neural networks learn.
The Core Idea
Remember the chain rule from single-variable calculus? If you have a function inside another function, you multiply their derivatives. The multivariable chain rule extends this to functions with multiple inputs.
Imagine you're hiking, and your elevation depends on your position (x, y). But your position changes over time as you walk. How fast is your elevation changing? You need to account for both how elevation changes with position and how position changes with time.
The Mathematical Framework
If z depends on variables that themselves depend on other variables, we chain their partial derivatives:
Example: Say z = f(u, v) where u and v are functions of x and y.
To find how z changes with respect to x:
∂z/∂x = (∂z/∂u)(∂u/∂x) + (∂z/∂v)(∂v/∂x)
Notice you sum over all paths through which x affects z.
Why This Matters for ML
In neural networks, the output depends on weights in the final layer, which depend on activations from earlier layers, which depend on their weights, and so on. When training, we need to know how changing any weight affects the final loss.
The multivariable chain rule lets us "backpropagate" gradients through these nested dependencies, computing derivatives layer by layer. Without it, we couldn't efficiently train deep networks.
Key Takeaway: The multivariable chain rule tracks how changes propagate through multiple variables simultaneously, enabling us to compute gradients in complex, multi-layered functions—essential for training neural networks through backpropagation.