The Projection Matrix
You'll learn how to build a single matrix P that projects any vector onto a subspace, given the subspace's basis.
The Core Idea
We've learned how to project a vector onto a subspace using the formula p = A(AᵀA)⁻¹Aᵀb, where A has the subspace's basis vectors as columns and b is the vector we're projecting. Notice something interesting: the part P = A(AᵀA)⁻¹Aᵀ depends only on the subspace (through A), not on the specific vector b.
This matrix P is called the projection matrix. Once you compute it, projecting any vector b onto your subspace becomes just a simple matrix multiplication: p = Pb.
How It Works
Think of P as a "projection machine" customized for your specific subspace:
- Start with your basis: Put the basis vectors of your subspace as columns of matrix A
- Compute the projection matrix: P = A(AᵀA)⁻¹Aᵀ
- Project any vector: For any vector b, just compute Pb
Key Properties
The projection matrix has remarkable properties:
- P² = P (projecting twice is the same as projecting once)
- Pᵀ = P (it's symmetric)
- Vectors already in the subspace stay unchanged: if b is in the column space of A, then Pb = b
- The matrix (I - P) projects onto the orthogonal complement
Example Structure
If your subspace is spanned by columns of A = [a₁ a₂], compute:
- AᵀA (a 2×2 matrix of inner products)
- (AᵀA)⁻¹ (invert that small matrix)
- P = A(AᵀA)⁻¹Aᵀ (the final projection matrix)
Key Takeaway: The projection matrix P = A(AᵀA)⁻¹Aᵀ encodes orthogonal projection onto a subspace as a single reusable matrix transformation.