Course contentsShow
← Machine Learning and Deep Learning
Lesson 72 of 3,5381. Mathematical Foundations for Machine LearningFree lesson

Independence of Random Variables

Define and test for statistical independence between random variables.

Independence of Random Variables

What you'll learn: How to recognize and test whether two random variables are statistically independent—a crucial concept for understanding data relationships in ML.

Core Idea

Two random variables are independent if knowing the value of one gives you no information about the other. Think of it like flipping a coin and rolling a die: the coin result doesn't affect the die, and vice versa.

Formally, random variables X and Y are independent if:

P(X = x, Y = y) = P(X = x) · P(Y = y)

In words: the joint probability equals the product of their individual (marginal) probabilities for all possible values of x and y.

Testing for Independence

Using Probability Distributions

For discrete variables, check if the joint PMF factorizes:

  • If p(x, y) = p(x) · p(y) everywhere → independent
  • If this fails for even one pair → not independent

For continuous variables, check if the joint PDF factorizes:

  • If f(x, y) = f(x) · f(y) everywhere → independent

Using Covariance

Remember from the previous lesson that covariance measures linear relationship. If X and Y are independent, then Cov(X, Y) = 0.

⚠️ Warning: The reverse isn't always true! Zero covariance doesn't guarantee independence (it only rules out linear dependence).

Intuition

Independence is stronger than zero correlation. Independent variables have no relationship whatsoever—not linear, not nonlinear, nothing. Zero covariance just means no linear pattern.

Key Takeaway: Random variables X and Y are independent when their joint probability always equals the product of their marginals, meaning knowing one reveals nothing about the other—a foundational assumption in many ML algorithms.