Course contentsShow
← Machine Learning and Deep Learning
Lesson 790 of 3,53819. PyTorch FundamentalsFree lesson

The requires_grad Flag

How to mark tensors for gradient tracking and when to enable or disable gradient computation.

The requires_grad Flag

What you'll learn: How to tell PyTorch which tensors need gradient tracking so you can train your models efficiently.

The Core Idea

In PyTorch, not every tensor needs to have its gradients computed. The requires_grad flag is a boolean attribute that marks whether a tensor should track operations performed on it for automatic differentiation. When requires_grad=True, PyTorch builds a computational graph behind the scenes, recording every operation so it can later compute gradients via backpropagation.

Think of requires_grad as a "breadcrumb trail" switch. When turned on, PyTorch drops breadcrumbs at each operation, allowing it to retrace steps backward and calculate how much each input contributed to the final output.

When to Enable It

You typically enable requires_grad for:

  • Model parameters (weights and biases) – these are what you're training
  • Inputs you want to analyze – for visualization or adversarial examples

You keep it disabled (the default) for:

  • Training data inputs – you don't update your dataset
  • Constants and hyperparameters – these aren't learned
  • Inference/evaluation – saves memory and speeds up computation

How It Works

When you create a tensor, it defaults to requires_grad=False. To enable tracking:

x = torch.tensor([2.0, 3.0], requires_grad=True)

Or enable it after creation:

x = torch.tensor([2.0, 3.0])
x.requires_grad = True

Any tensor resulting from operations on a requires_grad=True tensor will also track gradients automatically.

Key Takeaway: The requires_grad flag tells PyTorch which tensors need gradient computation—enable it for parameters you want to train, disable it everywhere else to save memory and computation.