Course contentsShow
← Machine Learning and Deep Learning
Lesson 69 of 3,5381. Mathematical Foundations for Machine LearningFree lesson

Joint Probability Distributions

Extend probability to multiple random variables using joint distributions.

Joint Probability Distributions

You'll learn how to describe the probability of two or more random variables occurring together.

What Is a Joint Probability Distribution?

So far, you've worked with single random variables—one coin flip, one measurement, one outcome. But in machine learning, you often need to understand how multiple random variables behave together. A joint probability distribution tells you the probability that each combination of values occurs simultaneously.

Think of it like weather forecasting: you might want to know not just the probability of rain, but the probability of rain and cold temperatures together—that's a joint probability.

The Core Idea

For two random variables X and Y:

  • Discrete case: The joint probability mass function is written as P(X = x, Y = y) or simply P(x, y). It gives the probability that X takes value x and Y takes value y at the same time.

  • Continuous case: The joint probability density function f(x, y) works similarly, where probabilities come from integrating over regions.

Simple Example

Imagine rolling a die (X) and flipping a coin (Y):

  • X can be 1, 2, 3, 4, 5, or 6
  • Y can be Heads or Tails

The joint distribution P(X, Y) tells you the probability of every pair, like P(X=3, Y=Heads) = 1/12 (assuming independence).

Why It Matters

Joint distributions let you model relationships between variables—critical for understanding feature interactions in datasets. You can also derive marginal distributions (single-variable probabilities) and conditional distributions (one variable given another) from the joint distribution.

Key Takeaway: Joint probability distributions extend probability to multiple random variables, capturing how they occur together—essential for modeling real-world data with many interacting features.