Course contentsShow
← Machine Learning and Deep Learning
Lesson 11 of 3,5381. Mathematical Foundations for Machine LearningFree lesson

Basis and Dimension

Learn what constitutes a basis, how dimension is defined, and their role in representing data.

Basis and Dimension

What you'll learn: How a basis provides the minimal "building blocks" for a vector space, and how dimension counts those blocks.

The Core Idea

Imagine you're in a city where every address can be uniquely described by just two numbers: blocks east/west and blocks north/south. Those two directions form a basis for navigating the city—you don't need a third direction because any location is a combination of just those two.

In linear algebra, a basis is a set of linearly independent vectors that span the entire vector space. "Linearly independent" (from our previous lesson) means none of these vectors can be written as a combination of the others. "Span" means you can reach any vector in the space by combining the basis vectors.

Why It Matters

The dimension of a vector space is simply the number of vectors in a basis. Our city example has dimension 2 (east/west, north/south). Three-dimensional space needs three basis vectors (think x, y, z axes).

In machine learning, dimension tells you how many features or variables you're working with. A dataset with 100 features lives in a 100-dimensional space—you'd need 100 basis vectors to describe all possible data points.

Example Without Code

Consider vectors in a plane. The vectors [1, 0] and [0, 1] form a basis because:

  • They're linearly independent (neither is a multiple of the other)
  • Any vector [a, b] can be written as a·[1, 0] + b·[0, 1]
  • There are 2 of them, so the dimension is 2

You could also use [1, 1] and [1, -1] as a basis—same dimension, different "building blocks."

Key Takeaway: A basis is the minimal set of independent vectors needed to represent all vectors in a space; the dimension counts how many basis vectors you need.