Course contentsShow
← Machine Learning and Deep Learning
Lesson 21 of 3,5381. Mathematical Foundations for Machine LearningFree lesson

Orthogonal Matrices and Their Properties

Learn properties of orthogonal matrices including preservation of norms and angles.

Orthogonal Matrices and Their Properties

What you'll learn: How orthogonal matrices preserve lengths and angles, making them powerful tools for transformations in machine learning.

What Is an Orthogonal Matrix?

An orthogonal matrix is a square matrix whose columns (and rows) are orthonormal vectors—meaning each column has length 1 and is perpendicular to all other columns. Mathematically, a matrix Q is orthogonal if:

Q^T Q = Q Q^T = I

where Q^T is the transpose and I is the identity matrix. This also means Q^T = Q^(-1)—the transpose equals the inverse!

Why They Matter: Preservation Properties

Orthogonal matrices have two magical properties:

1. They Preserve Norms (Lengths)

When you multiply a vector v by an orthogonal matrix Q, the result Qv has the exact same length as v. Think of rotating a stick—it doesn't stretch or shrink, just changes direction.

2. They Preserve Angles

The angle between any two vectors stays identical after transformation by an orthogonal matrix. If two vectors meet at 60°, they'll still meet at 60° after applying Q.

The Intuition

Imagine orthogonal matrices as rigid rotations and reflections. They move things around in space but never distort shapes—no stretching, squashing, or shearing. A circle remains a circle, a square remains a square (though possibly rotated or flipped).

In machine learning, this property is crucial for techniques like Principal Component Analysis (PCA) where we want to rotate data to find new axes without changing the intrinsic relationships between data points.

Key Takeaway: Orthogonal matrices are special transformations where Q^T = Q^(-1), and they preserve both vector lengths and angles between vectors—acting like pure rotations or reflections in space.