Course contentsShow
← Machine Learning and Deep Learning
Lesson 798 of 3,53819. PyTorch FundamentalsFree lesson

Gradient Function and the grad_fn Attribute

Inspecting the grad_fn to see what operation created a tensor and understand the backward pass.

Gradient Function and the grad_fn Attribute

What you'll learn: How to inspect the grad_fn attribute to understand what operation created a tensor and trace the backward pass.


The grad_fn Attribute

Every tensor that's created through an operation on another tensor with requires_grad=True automatically gets a .grad_fn attribute. This attribute stores a reference to the function that created the tensor, forming a link in the computational graph.

Think of grad_fn as a breadcrumb trail: it tells you "I was created by this operation, and here's how to compute my gradient during the backward pass."

Inspecting grad_fn

import torch

x = torch.tensor([2.0], requires_grad=True)
y = x * 3
z = y + 5

print(y.grad_fn)  # <MulBackward0 object>
print(z.grad_fn)  # <AddBackward0 object>
print(x.grad_fn)  # None (leaf tensor)

Here, y.grad_fn points to MulBackward0 (multiplication), and z.grad_fn points to AddBackward0 (addition). The original tensor x has grad_fn=None because it's a leaf tensor — it wasn't created by any operation.

Why This Matters

The grad_fn attribute is how PyTorch knows what chain rule to apply during .backward(). When you call z.backward(), PyTorch traverses backwards through these grad_fn references, computing gradients for each operation in reverse order.

You can use grad_fn for debugging: if gradients aren't flowing as expected, checking grad_fn helps you verify that your computational graph is built correctly.

Key Takeaway: The grad_fn attribute records the operation that created a tensor, enabling PyTorch to automatically compute gradients by tracing the computational graph backward.