Element-wise Operations
What you'll learn: How to perform operations on tensors that apply to each element individually, forming the foundation of most mathematical transformations in deep learning.
The Core Idea
Element-wise operations apply the same calculation to every element in a tensor independently. Think of it like having a magic wand that touches each number in your tensor with the same spell—if you add 5, every single element gets 5 added to it. If you multiply by 2, every element doubles.
These operations are fundamental because neural networks constantly transform data through mathematical functions applied to every value in your tensors.
Basic Arithmetic Operations
The four basic operations work element-by-element:
import torch
x = torch.tensor([1.0, 2.0, 3.0])
y = torch.tensor([4.0, 5.0, 6.0])
# Element-wise addition
z = x + y # tensor([5., 7., 9.])
# Element-wise subtraction
z = x - y # tensor([-3., -3., -3.])
# Element-wise multiplication
z = x * y # tensor([4., 10., 18.])
# Element-wise division
z = x / y # tensor([0.25, 0.4, 0.5])
Element-wise Functions
PyTorch provides mathematical functions that operate element-wise:
x = torch.tensor([1.0, 2.0, 3.0])
# Exponential
torch.exp(x) # tensor([2.7183, 7.3891, 20.0855])
# Natural logarithm
torch.log(x) # tensor([0.0000, 0.6931, 1.0986])
# Square root
torch.sqrt(x) # tensor([1.0000, 1.4142, 1.7321])
These functions are crucial for activation functions, probability calculations, and various transformations throughout your network.
Key Takeaway: Element-wise operations apply the same calculation independently to every element in a tensor, enabling efficient parallel computation and forming the building blocks of neural network transformations.