The Computational Graph
What you'll learn: How PyTorch dynamically builds and uses a computation graph to track operations and enable automatic differentiation.
What Is a Computational Graph?
A computational graph is PyTorch's behind-the-scenes record of every operation you perform on tensors with requires_grad=True. Think of it as a detailed receipt that tracks not just your final result, but every mathematical step taken to get there.
When you write y = x * 2 + 3, PyTorch doesn't just compute the answer—it builds a graph structure that records "multiply by 2" and "add 3" as connected nodes. Each node remembers what operation happened and which tensors were involved.
Dynamic Graph Construction
Unlike some frameworks that require you to define the entire graph upfront, PyTorch builds its graph dynamically as your code executes. This means:
- The graph is created on-the-fly, operation by operation
- Each forward pass can be different (loops, conditionals work naturally)
- The graph exists only for tensors where
requires_grad=True
Why Does This Matter?
This graph is what makes backpropagation possible. When you call .backward() later, PyTorch traverses this graph in reverse, computing gradients at each node using the chain rule. Without the graph storing operation history, automatic differentiation would be impossible.
Analogy
Imagine baking a cake while a friend writes down every action: "mixed flour and sugar," "added eggs," "baked at 350°F." If the cake turns out wrong, you can trace back through those notes to figure out where things went sideways. The computation graph is PyTorch's set of notes.
Key Takeaway: PyTorch builds a dynamic computational graph as operations execute, storing the history needed to compute gradients during backpropagation—this is the foundation of autograd.