Course contentsShow
← Mathematics
Lesson 2227 of 3,12146. Linear Algebra — Inner Products and OrthogonalityFree lesson

Numerical Stability and Modified Gram-Schmidt

Recognize limitations of classical Gram-Schmidt in finite precision and learn the modified algorithm for better accuracy.

Numerical Stability and Modified Gram-Schmidt

You'll learn why the classical Gram-Schmidt algorithm can fail in practice and how a simple modification fixes it.

The Problem with Classical Gram-Schmidt

When you learned the Gram-Schmidt algorithm, you saw how to convert a basis into an orthonormal one by subtracting projections. In theory, it's perfect. In practice, on computers with finite-precision arithmetic (like floating-point numbers), classical Gram-Schmidt can produce vectors that aren't nearly as orthogonal as they should be.

Why? As you process each vector, you subtract off projections onto previously computed vectors. But those earlier vectors may already contain small rounding errors. When you use them to compute projections, those errors accumulate and amplify. By the time you're orthogonalizing the last vector against many others, the loss of orthogonality can be severe.

Think of it like a game of telephone: each person introduces a tiny error, and by the end, the message is garbled.

The Modified Gram-Schmidt Algorithm

The fix is surprisingly simple: instead of computing all projections at once using the original vectors, subtract each projection immediately and work with the updated vector for the next projection.

In classical GS, you compute v₂ = u₂ - proj₁(u₂), subtracting all projections in one go. In modified GS, you do v₂⁽¹⁾ = u₂ - proj₁(u₂), then immediately use v₂⁽¹⁾ for the next step—not u₂.

This incremental correction prevents error accumulation. Each subtraction uses the most current, "cleaned up" version of the vector, maintaining better numerical orthogonality.

When It Matters

For small, well-conditioned problems, classical Gram-Schmidt often works fine. But for large matrices, nearly dependent vectors, or high-precision requirements, modified Gram-Schmidt is essential.

Key Takeaway: Classical Gram-Schmidt suffers from numerical instability due to rounding error accumulation; Modified Gram-Schmidt corrects this by updating vectors incrementally after each projection, preserving orthogonality in finite-precision arithmetic.