Course contentsShow
← Machine Learning and Deep Learning
Lesson 16 of 3,5381. Mathematical Foundations for Machine LearningFree lesson

Eigenvalues and Eigenvectors: Definitions

Understand eigenvectors as special directions and eigenvalues as scaling factors under linear transformations.

Eigenvalues and Eigenvectors: Definitions

What you'll learn: You'll discover what makes certain vectors "special" when transformed by a matrix, and how eigenvalues measure their behavior.

The Core Idea

Imagine you have a matrix A that transforms vectors (through matrix-vector multiplication, which you've already learned). Most vectors, when multiplied by A, change both their direction and length.

But some special vectors only get stretched or shrunk—their direction stays the same (or exactly flips). These are eigenvectors.

The amount by which an eigenvector gets scaled is its eigenvalue.

Mathematically: If Av = λv, then v is an eigenvector and λ (lambda) is the eigenvalue.

Plain English Analogy

Think of a matrix as a transformation machine. When you feed most vectors through it, they come out pointing in completely new directions. But eigenvectors are like "stubborn arrows"—they refuse to rotate. The transformation can only make them longer, shorter, or flip them backward. The eigenvalue tells you how much longer or shorter (2 means doubled, 0.5 means halved, -1 means same length but flipped).

Why This Matters

Eigenvalues and eigenvectors reveal the "natural directions" of a transformation. In a symmetric matrix (remember: Aᵀ = A), eigenvectors form a basis—meaning any vector can be expressed as combinations of these special directions. This becomes incredibly powerful for understanding data patterns in machine learning.

Key Takeaway: Eigenvectors are directions that remain unchanged (except for scaling) under matrix transformation; eigenvalues are the scaling factors that tell you how much stretching or shrinking occurs.