Course contentsShow
← Mathematics
Lesson 2955 of 3,12158. Joint and Limit BehaviorFree lesson

Covariance Matrix

Introduce the covariance matrix for multiple random variables and its role in multivariate analysis.

Covariance Matrix

What You'll Learn

You'll learn how to organize covariances among multiple random variables into a matrix structure that captures all pairwise relationships at once.

The Core Idea

When working with just two random variables, we compute a single covariance. But what if you're analyzing three, four, or even dozens of variables simultaneously? The covariance matrix elegantly packages all the variance and covariance information into one mathematical object.

For random variables X1,X2,…,XnX_1, X_2, \ldots, X_n, the covariance matrix Σ\Sigma (capital sigma) is an n×nn \times n matrix where:

  • The diagonal entries contain the variances: Σii=Var(Xi)\Sigma_{ii} = \text{Var}(X_i)
  • The off-diagonal entries contain the covariances: Σij=Cov(Xi,Xj)\Sigma_{ij} = \text{Cov}(X_i, X_j)

For example, with three random variables XX, YY, and ZZ:

Σ=[Var(X)Cov(X,Y)Cov(X,Z)Cov(Y,X)Var(Y)Cov(Y,Z)Cov(Z,X)Cov(Z,Y)Var(Z)]\Sigma = \begin{bmatrix} \text{Var}(X) & \text{Cov}(X,Y) & \text{Cov}(X,Z) \\ \text{Cov}(Y,X) & \text{Var}(Y) & \text{Cov}(Y,Z) \\ \text{Cov}(Z,X) & \text{Cov}(Z,Y) & \text{Var}(Z) \end{bmatrix}

Since Cov(Xi,Xj)=Cov(Xj,Xi)\text{Cov}(X_i, X_j) = \text{Cov}(X_j, X_i), the covariance matrix is symmetric.

Why It Matters

Think of the covariance matrix as a complete "relationship map" for your variables. It tells you:

  • How spread out each variable is (the diagonal)
  • How each pair of variables moves together (the off-diagonals)

This matrix is fundamental in multivariate statistics, appearing in principal component analysis, portfolio optimization, multivariate normal distributions, and regression with multiple predictors.

Key Takeaway: The covariance matrix compactly stores all variances and pairwise covariances for a set of random variables, providing a complete picture of their joint variability and relationships.