Moving Tensors to GPU with .to() and .cuda()
You'll learn how to transfer tensors from CPU memory to GPU memory so your model can leverage accelerated computation.
The Core Idea
By default, PyTorch creates tensors in CPU memory. To take advantage of GPU acceleration, you must explicitly move tensors to GPU memory using either the .to() method or the .cuda() method. Think of this like moving your workspace from a regular desk (CPU) to a specialized workbench with power tools (GPU) — the work happens faster, but you need to physically transport your materials there.
Two Methods: .to() and .cuda()
The .to(device) method is the modern, flexible approach:
import torch
# Create a tensor on CPU
x = torch.tensor([1.0, 2.0, 3.0])
# Move to GPU
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
x_gpu = x.to(device)
The .cuda() method directly moves tensors to GPU:
# Explicit GPU transfer
x_gpu = x.cuda()
Which Should You Use?
Prefer .to(device) because it's device-agnostic — your code works whether CUDA is available or not. The .cuda() method will crash if no GPU exists, while .to(device) gracefully falls back to CPU when you use the conditional device assignment shown above.
Important Notes
- Moving tensors creates a copy — the original remains on CPU unless you reassign:
x = x.to(device) - All tensors in an operation must be on the same device
- Moving data between CPU and GPU has overhead, so minimize transfers
Key Takeaway: Use .to(device) to move tensors between CPU and GPU memory, ensuring device-agnostic code that leverages GPU acceleration when available.