Tensor Initialization Functions
What you'll learn: How to create tensors filled with specific patterns using PyTorch's built-in initialization functions.
The Core Idea
Now that you know how to create tensors from existing data, you often need tensors with predefined values or patterns. PyTorch provides convenient functions to generate tensors filled with zeros, ones, random numbers, or sequences—essential for initializing weights, creating masks, or generating test data.
Think of these functions as "tensor factories" that stamp out tensors with specific filling patterns, saving you from manually typing values.
Key Initialization Functions
torch.zeros()
Creates a tensor filled entirely with zeros. Perfect for initializing bias terms or creating blank canvases.
import torch
zeros_tensor = torch.zeros(3, 4) # 3x4 tensor of zeros
torch.ones()
Creates a tensor filled with ones. Useful for masks or initializing certain parameters.
ones_tensor = torch.ones(2, 3) # 2x3 tensor of ones
torch.rand()
Generates random values uniformly distributed between 0 and 1. Great for random initialization.
random_tensor = torch.rand(2, 2) # Values between 0 and 1
torch.randn()
Creates random values from a normal distribution (mean=0, std=1). The standard choice for weight initialization.
normal_tensor = torch.randn(3, 3) # Standard normal distribution
torch.arange()
Creates a sequence of numbers, like Python's range() but returns a tensor.
sequence = torch.arange(0, 10, 2) # [0, 2, 4, 6, 8]
When to Use Each
- zeros/ones: Known initialization patterns, biases, masks
- rand: Uniform random initialization (less common)
- randn: Weight initialization (most common for neural networks)
- arange: Index tensors, creating coordinate grids
Key Takeaway: PyTorch's initialization functions let you quickly create tensors with zeros, ones, uniform random values, normal random values, or sequences—building blocks for everything from weight initialization to data generation.