Vector Operations: Addition and Scalar Multiplication

What you'll learn: How to combine vectors through addition and how to stretch or shrink them using scalar multiplication—two fundamental operations you'll use constantly in machine learning.

Understanding Vector Addition

Think of vectors as arrows pointing in space. When you add two vectors, you're placing them tip-to-tail and drawing a new arrow from the start of the first to the end of the second.

Mathematically, you add vectors element-wise—meaning you add corresponding positions:

Vector A = [2, 3]
Vector B = [1, 4]
A + B = [2+1, 3+4] = [3, 7]

Real-world analogy: Imagine walking 2 blocks east and 3 blocks north (vector A), then continuing 1 block east and 4 blocks north (vector B). Your total displacement is 3 blocks east and 7 blocks north—that's vector addition!

Understanding Scalar Multiplication

When you multiply a vector by a scalar (a single number), you scale every element in the vector by that amount. This stretches or shrinks the vector while keeping its direction the same (or reversing it if the scalar is negative).

Scalar = 3
Vector A = [2, 3]
3 × A = [3×2, 3×3] = [6, 9]

Geometric intuition: If a scalar is 2, you double the vector's length. If it's 0.5, you halve it. If it's -1, you flip its direction entirely.

Why This Matters

In machine learning, you'll constantly adjust model parameters (vectors) by adding gradients to them or scaling learning rates. These simple operations are the building blocks of how neural networks learn.

Key Takeaway: Vector addition combines vectors element-by-element, while scalar multiplication scales every element uniformly—master these and you've unlocked the basic arithmetic of machine learning.