Course contentsShow
← Machine Learning and Deep Learning
Lesson 156 of 3,5382. Machine Learning Mindset and Core ConceptsFree lesson

Broadcasting Rules

How NumPy automatically expands dimensions for operations between arrays of different shapes in ML operations.

Broadcasting Rules

What You'll Learn

You'll learn how NumPy automatically stretches arrays of different shapes so they can work together in vectorized operations—a fundamental trick that makes ML code fast and concise.

The Core Idea

Broadcasting is NumPy's clever system for handling operations between arrays of different shapes without making wasteful copies. Instead of forcing you to manually expand arrays to matching sizes, NumPy follows specific rules to "virtually" stretch smaller arrays to match larger ones.

Think of it like a photocopy machine that automatically tiles a small pattern to fill a larger page—but without actually using extra paper.

The Broadcasting Rules

NumPy checks array shapes from right to left and applies these rules:

  1. If arrays have different numbers of dimensions, pad the smaller shape with ones on the left until both have the same length
  2. For each dimension, the sizes must either:
    • Be equal, OR
    • One of them must be 1 (which gets "stretched")
  3. If any dimension fails these conditions, broadcasting fails

Example Walkthrough

# Shape (3, 4) array + shape (4,) array
A.shape = (3, 4)
b.shape = (4,)

# Step 1: Pad b's shape on left: (4,) becomes (1, 4)
# Step 2: Compare dimensions right-to-left:
#   - Last dim: 4 == 4 ✓
#   - Second-to-last: 3 vs 1 → 1 stretches to 3 ✓
# Result shape: (3, 4)

This is why you can add a bias vector to every row of a weight matrix without loops—broadcasting handles the repetition automatically during computation.

Key Takeaway: Broadcasting lets NumPy perform operations on arrays with compatible (not identical) shapes by automatically expanding dimensions of size 1, enabling fast, memory-efficient vectorized ML operations without explicit loops or array copying.