Covariance Matrix
What You'll Learn
You'll learn how to organize covariances among multiple random variables into a matrix structure that captures all pairwise relationships at once.
The Core Idea
When working with just two random variables, we compute a single covariance. But what if you're analyzing three, four, or even dozens of variables simultaneously? The covariance matrix elegantly packages all the variance and covariance information into one mathematical object.
For random variables , the covariance matrix (capital sigma) is an matrix where:
- The diagonal entries contain the variances:
- The off-diagonal entries contain the covariances:
For example, with three random variables , , and :
Since , the covariance matrix is symmetric.
Why It Matters
Think of the covariance matrix as a complete "relationship map" for your variables. It tells you:
- How spread out each variable is (the diagonal)
- How each pair of variables moves together (the off-diagonals)
This matrix is fundamental in multivariate statistics, appearing in principal component analysis, portfolio optimization, multivariate normal distributions, and regression with multiple predictors.
Key Takeaway: The covariance matrix compactly stores all variances and pairwise covariances for a set of random variables, providing a complete picture of their joint variability and relationships.