Course contentsShow
← Machine Learning and Deep Learning
Lesson 792 of 3,53819. PyTorch FundamentalsFree lesson

Computing Gradients with .backward()

Calling backward() on a scalar loss to compute gradients for all tensors with requires_grad=True.

Computing Gradients with .backward()

What you'll learn: How to trigger automatic gradient computation in PyTorch using the .backward() method on your loss value.

The Core Idea

Once you've built a computational graph by performing operations on tensors with requires_grad=True, you need to actually compute the gradients. That's where .backward() comes in.

Think of .backward() like pressing "calculate" on a calculator after entering an equation. You've set up all the operations (the computational graph), but nothing happens until you explicitly ask PyTorch to traverse that graph backward and compute all the derivatives.

How It Works

You call .backward() on a scalar tensor—typically your loss value. PyTorch then:

  1. Starts at that scalar
  2. Walks backward through the computational graph
  3. Applies the chain rule at each node
  4. Accumulates gradients in the .grad attribute of every tensor that had requires_grad=True
import torch

# Set up tensors
x = torch.tensor([2.0], requires_grad=True)
y = torch.tensor([3.0], requires_grad=True)

# Forward pass: build the graph
z = x * y + y**2
loss = z.sum()  # Scalar loss

# Backward pass: compute gradients
loss.backward()

# Gradients are now available
print(x.grad)  # Gradient of loss w.r.t. x
print(y.grad)  # Gradient of loss w.r.t. y

After calling .backward(), each parameter "knows" how the loss would change if that parameter changed—exactly what you need for gradient descent!

Key Takeaway: Call .backward() on your scalar loss to automatically compute and store gradients for all tensors with requires_grad=True in their .grad attributes.