Testing Significance in Correlation Matrices
You'll learn how to determine which correlations in a matrix are statistically meaningful and how to avoid false discoveries when testing many pairs at once.
The Core Challenge
When you compute a correlation matrix for multiple variables, you get dozens or even hundreds of correlation coefficients. Some might look impressive (say, r = 0.35), but are they real patterns or just random noise?
Testing significance in correlation matrices means applying hypothesis tests to each correlation coefficient to determine which ones are statistically significant. However, there's a catch: the multiple comparisons problem.
Why Multiple Comparisons Matter
Imagine testing 100 correlations at α = 0.05. Even if no true correlations exist, you'd expect about 5 false positives purely by chance (5% of 100). The more tests you run, the more likely you are to find "significant" results that are actually flukes.
Analogy: It's like flipping 100 coins and being surprised when 5 land on heads. That's not special—it's expected randomness.
Adjustment Strategies
To control false discoveries in correlation matrices:
- Bonferroni correction: Divide your significance level by the number of tests (e.g., 0.05/100 = 0.0005)
- Holm-Bonferroni: A less conservative step-down method you've learned
- False Discovery Rate (FDR): Controls the expected proportion of false positives among all significant results
Each correlation is tested using the t-test for Pearson correlation (or appropriate test for Spearman/Kendall), then p-values are adjusted using your chosen method.
Practical Interpretation
After adjustment, you might find that only a few correlations survive the stricter threshold. This is good—it means you're identifying robust relationships rather than chasing statistical noise.
Key Takeaway: When testing significance across an entire correlation matrix, always adjust for multiple comparisons to avoid mistaking random chance for meaningful patterns—unadjusted tests will produce misleading false positives.