Marginal and Conditional Distributions
What you'll learn: How to extract single-variable information from joint probability distributions and understand probabilities when certain conditions are met.
The Core Idea
When you have a joint probability distribution for two or more random variables (like height and weight together), you often need to focus on just one variable or ask "what happens when one variable takes a specific value?"
- Marginal distribution answers: "What's the probability distribution of X alone, ignoring Y entirely?"
- Conditional distribution answers: "What's the probability distribution of X given that Y equals some specific value?"
Marginal Distributions
Think of a table showing probabilities for all combinations of X and Y. To get the marginal distribution of X, you sum across all possible values of Y for each value of X.
Analogy: Imagine a cafeteria tracking lunch choices (sandwich type) and drink choices. The marginal distribution of sandwich types tells you how popular each sandwich is overall, regardless of what drink people chose.
For discrete variables: P(X = x) = Σ_y P(X = x, Y = y)
For continuous variables, replace the sum with an integral over Y.
Conditional Distributions
A conditional distribution restricts your view to cases where one variable has a specific value. Using Bayes' theorem from earlier:
P(X = x | Y = y) = P(X = x, Y = y) / P(Y = y)
This tells you how X behaves when you know Y = y.
Analogy: In the cafeteria, the conditional distribution of sandwiches given someone chose soda shows different preferences than for those who chose water.
Key Takeaway: Marginal distributions extract single-variable probabilities by summing over others; conditional distributions describe probabilities when fixing one variable to a known value. Both are essential for understanding relationships between variables in ML models.