The Chain Rule in Computational Graphs
What you'll learn: How to systematically compute gradients by following paths backward through a computational graph and multiplying partial derivatives.
The Core Idea
You've already seen the chain rule in calculus and how computational graphs represent operations visually. Now we connect them: to find how the final output (loss) changes with respect to any input, you trace backward through the graph, multiplying partial derivatives along each edge.
Think of it like tracking influence through a chain of events. If pushing a domino (input) affects the next domino, which affects the next, and so on until the last one falls (output), the chain rule tells you the total effect of that first push by multiplying how each domino influences the next.
How It Works in Practice
In a computational graph:
- Each node represents a value (variable or operation result)
- Each edge represents a dependency (which values feed into which operations)
- Backward pass: Starting from the loss, compute the gradient with respect to each node by multiplying the "gradient from above" (∂Loss/∂node_output) by the "local gradient" (∂node_output/∂node_input)
For example, if z = f(y) and y = g(x), then:
∂z/∂x = (∂z/∂y) × (∂y/∂x)
You multiply gradients along the path from output back to input. If multiple paths lead to a variable, you sum the gradients from all paths (multivariate chain rule).
Why This Matters
This systematic application of the chain rule is the engine of backpropagation. Every layer, every activation, every operation contributes its local partial derivative, and the chain rule stitches them together to compute gradients for all parameters efficiently.
Key Takeaway: The chain rule in computational graphs means computing gradients by multiplying partial derivatives backward along paths from output to inputs, enabling efficient gradient computation for complex nested functions.