Gradient Function and the grad_fn Attribute
What you'll learn: How to inspect the grad_fn attribute to understand what operation created a tensor and trace the backward pass.
The grad_fn Attribute
Every tensor that's created through an operation on another tensor with requires_grad=True automatically gets a .grad_fn attribute. This attribute stores a reference to the function that created the tensor, forming a link in the computational graph.
Think of grad_fn as a breadcrumb trail: it tells you "I was created by this operation, and here's how to compute my gradient during the backward pass."
Inspecting grad_fn
import torch
x = torch.tensor([2.0], requires_grad=True)
y = x * 3
z = y + 5
print(y.grad_fn) # <MulBackward0 object>
print(z.grad_fn) # <AddBackward0 object>
print(x.grad_fn) # None (leaf tensor)
Here, y.grad_fn points to MulBackward0 (multiplication), and z.grad_fn points to AddBackward0 (addition). The original tensor x has grad_fn=None because it's a leaf tensor — it wasn't created by any operation.
Why This Matters
The grad_fn attribute is how PyTorch knows what chain rule to apply during .backward(). When you call z.backward(), PyTorch traverses backwards through these grad_fn references, computing gradients for each operation in reverse order.
You can use grad_fn for debugging: if gradients aren't flowing as expected, checking grad_fn helps you verify that your computational graph is built correctly.
Key Takeaway: The grad_fn attribute records the operation that created a tensor, enabling PyTorch to automatically compute gradients by tracing the computational graph backward.