What is a Tensor?
You'll learn: What tensors are and why they serve as the fundamental building block for all PyTorch operations.
The Core Idea
A tensor is simply a multidimensional array—a container for numbers arranged in a grid with one or more dimensions. Think of it as a generalization of familiar mathematical objects:
- A scalar (single number) is a 0-dimensional tensor
- A vector (list of numbers) is a 1-dimensional tensor
- A matrix (table of numbers) is a 2-dimensional tensor
- Anything with 3+ dimensions is still a tensor
Why Tensors Matter in PyTorch
In PyTorch (and deep learning generally), everything is a tensor. Your input images? Tensors. Model weights? Tensors. Gradients during backpropagation? Also tensors.
Tensors are designed for efficient computation on GPUs and support all the mathematical operations neural networks need—matrix multiplication, element-wise operations, reshaping, and automatic differentiation.
A Simple Analogy
Think of a tensor like a spreadsheet that can have multiple dimensions:
- 1D tensor = a single row of data
- 2D tensor = a complete spreadsheet with rows and columns
- 3D tensor = a stack of spreadsheets (like monthly reports stacked together)
- 4D tensor = multiple stacks (like yearly collections of monthly reports)
For images, a color photo is typically stored as a 3D tensor: height × width × color channels (RGB). A batch of photos becomes a 4D tensor: batch_size × height × width × channels.
Key Takeaway: Tensors are multidimensional arrays that serve as PyTorch's universal data structure, enabling efficient computation and automatic differentiation for neural networks.