Basis and Dimension
What you'll learn: How a basis provides the minimal "building blocks" for a vector space, and how dimension counts those blocks.
The Core Idea
Imagine you're in a city where every address can be uniquely described by just two numbers: blocks east/west and blocks north/south. Those two directions form a basis for navigating the city—you don't need a third direction because any location is a combination of just those two.
In linear algebra, a basis is a set of linearly independent vectors that span the entire vector space. "Linearly independent" (from our previous lesson) means none of these vectors can be written as a combination of the others. "Span" means you can reach any vector in the space by combining the basis vectors.
Why It Matters
The dimension of a vector space is simply the number of vectors in a basis. Our city example has dimension 2 (east/west, north/south). Three-dimensional space needs three basis vectors (think x, y, z axes).
In machine learning, dimension tells you how many features or variables you're working with. A dataset with 100 features lives in a 100-dimensional space—you'd need 100 basis vectors to describe all possible data points.
Example Without Code
Consider vectors in a plane. The vectors [1, 0] and [0, 1] form a basis because:
- They're linearly independent (neither is a multiple of the other)
- Any vector [a, b] can be written as a·[1, 0] + b·[0, 1]
- There are 2 of them, so the dimension is 2
You could also use [1, 1] and [1, -1] as a basis—same dimension, different "building blocks."
Key Takeaway: A basis is the minimal set of independent vectors needed to represent all vectors in a space; the dimension counts how many basis vectors you need.