Trace of a Matrix
What you'll learn: You'll discover what the trace of a matrix is, how to compute it, and why it shows up everywhere in machine learning formulas.
What Is the Trace?
The trace of a square matrix is simply the sum of all elements along its main diagonal (top-left to bottom-right). That's it! If you have a matrix A, its trace is written as tr(A) or trace(A).
Think of the diagonal as the "spine" of your matrix—the trace tells you the sum of values running down that spine.
How to Calculate It
For a matrix like this:
A = [ 2 5 1 ]
[ 0 3 7 ]
[ 4 6 -1 ]
The diagonal elements are 2, 3, and -1. So:
tr(A) = 2 + 3 + (-1) = 4
You completely ignore all off-diagonal elements!
Key Properties
- Only square matrices have a trace (you need the same number of rows and columns)
- tr(A + B) = tr(A) + tr(B) — traces add nicely
- tr(cA) = c · tr(A) — you can pull out scalar multiples
- tr(AB) = tr(BA) — order doesn't matter for trace of products!
Why It Matters in Machine Learning
The trace appears frequently in ML loss functions and optimization. For example:
- When computing regularization penalties on weight matrices
- In principal component analysis (PCA) when measuring variance
- When working with covariance matrices to understand data spread
The trace gives you a single number that summarizes important information about a matrix—perfect for optimization algorithms that need scalar values to minimize.
Key Takeaway: The trace is the sum of a square matrix's diagonal elements, and it's a convenient scalar summary that appears throughout ML mathematics, especially in loss functions and regularization terms.