The Gram-Schmidt Algorithm: Overview
What you'll learn: How to systematically convert any linearly independent set of vectors into an orthogonal (or orthonormal) set that spans the same subspace.
The Core Idea
Imagine you have a set of linearly independent vectors that point in different directions, but they're not perpendicular to each other. The Gram-Schmidt algorithm is a step-by-step procedure that transforms these vectors into a new set where each vector is orthogonal (perpendicular) to all the others, while preserving the span—the subspace they generate remains exactly the same.
Think of it like remodeling a room: you're replacing a crooked coordinate system with one that has nice, perpendicular axes, but the "room" itself (the span) doesn't change.
The Algorithm in Plain English
Starting with linearly independent vectors {v₁, v₂, v₃, ...}:
-
Keep the first vector as-is: u₁ = v₁
-
For each subsequent vector, remove the parts that point in directions already covered:
- Take v₂ and subtract away its projection onto u₁
- Take v₃ and subtract away its projections onto both u₁ and u₂
- Continue this pattern...
-
Each new u vector is orthogonal to all previous ones
-
(Optional) Normalize each u to unit length to create an orthonormal set
The beauty is that each output vector uᵢ is built only from the input vectors v₁ through vᵢ, ensuring span{u₁, ..., uₖ} = span{v₁, ..., vₖ} at every stage.
Key Takeaway: The Gram-Schmidt algorithm systematically orthogonalizes a linearly independent set by iteratively removing components that aren't perpendicular, creating an orthogonal basis for the same subspace.