Course contentsShow
← Machine Learning and Deep Learning
Lesson 783 of 3,53819. PyTorch FundamentalsFree lesson

Matrix Multiplication and Linear Algebra

Use torch.mm(), torch.matmul(), and @ operator for matrix products, plus torch.linalg for decompositions.

Matrix Multiplication and Linear Algebra

What you'll learn: How to perform matrix multiplication in PyTorch using multiple methods and access linear algebra operations that power neural networks.

Why Matrix Multiplication Matters

In deep learning, nearly every operation is a matrix multiplication at its core. When data flows through a neural network layer, it's being matrix-multiplied by weights. Understanding how to perform these operations efficiently in PyTorch is fundamental.

Three Ways to Multiply Matrices

PyTorch gives you three equivalent ways to multiply matrices:

  1. torch.mm() – The explicit matrix multiplication function (works only for 2D tensors)
  2. torch.matmul() – A more flexible function that handles batches and broadcasting
  3. @ operator – Python's matrix multiplication operator (cleanest syntax)

Think of torch.mm() as the strict mathematician who only works with 2D matrices, while torch.matmul() and @ are more flexible, handling various tensor dimensions intelligently.

import torch

A = torch.tensor([[1, 2], [3, 4]])
B = torch.tensor([[5, 6], [7, 8]])

# All three produce the same result for 2D tensors:
result1 = torch.mm(A, B)
result2 = torch.matmul(A, B)
result3 = A @ B  # Most Pythonic!

Linear Algebra Toolkit

For advanced operations like decompositions (breaking matrices into simpler components), PyTorch provides torch.linalg. This module contains functions for eigenvalue decomposition, matrix inversion, solving linear systems, and more – essential tools for understanding what's happening inside optimization algorithms.

# Compute the inverse of a matrix
A_inv = torch.linalg.inv(A)

Key Takeaway: Use @ for clean matrix multiplication, torch.matmul() for batch operations, and torch.linalg for decompositions and advanced linear algebra operations that underpin deep learning mathematics.