Expectation and Mean
You'll learn: How to calculate the expected value of a random variable as a weighted average of its possible outcomes.
What Is Expectation?
The expectation (or mean) of a random variable is the long-run average value you'd expect if you repeated an experiment infinitely many times. Think of it as the "center of mass" of a probability distribution—where the data balances out.
How to Calculate It
The formula depends on whether your random variable is discrete or continuous:
For discrete random variables: The expectation E[X] is calculated by multiplying each possible value by its probability, then summing:
E[X] = Σ x · P(X = x)
Where you sum over all possible values x.
For continuous random variables: You integrate the value times its probability density function:
E[X] = ∫ x · f(x) dx
Intuitive Example
Imagine rolling a fair six-sided die. Each outcome (1, 2, 3, 4, 5, 6) has probability 1/6. The expected value is:
E[X] = 1·(1/6) + 2·(1/6) + 3·(1/6) + 4·(1/6) + 5·(1/6) + 6·(1/6) = 3.5
You'll never actually roll 3.5, but over many rolls, the average converges to 3.5. It's the weighted average where weights are the probabilities.
Why It Matters in ML
Expectations form the foundation for understanding model behavior, loss functions, and predictions. When you train a model, you're often minimizing the expected loss across all possible data points.
Key Takeaway: The expectation is a probability-weighted average that tells you the central tendency of a random variable—it's where the distribution "balances" on average.