Probability Axioms and Basic Rules
What You'll Learn
You'll discover Kolmogorov's three fundamental axioms that define probability and how to derive essential rules for combining probabilities.
The Foundation: Kolmogorov's Axioms
After defining sample spaces and events (our previous lesson), we need formal rules to assign probabilities. Russian mathematician Andrey Kolmogorov established three axioms that form the bedrock of probability theory:
Axiom 1: Non-negativity
The probability of any event A is always non-negative: P(A) ≥ 0
Think of it like this: you can't have "negative chance" of something happening.
Axiom 2: Certainty
The probability that something in the sample space occurs is 1: P(S) = 1
If you roll a die, you're absolutely certain to get some number from 1 to 6.
Axiom 3: Additivity
For mutually exclusive events (events that cannot happen simultaneously), probabilities add: P(A or B) = P(A) + P(B) when A and B cannot both occur.
Rolling a 2 or a 5 on one die roll: P(2 or 5) = P(2) + P(5) = 1/6 + 1/6 = 1/3.
Derived Rules
From these axioms, we derive critical rules:
Complement Rule: P(not A) = 1 - P(A)
- If the probability of rain is 0.3, the probability of no rain is 0.7.
Addition Rule (General): P(A or B) = P(A) + P(B) - P(A and B)
- Accounts for overlap when events can co-occur.
Multiplication Rule: For independent events: P(A and B) = P(A) × P(B)
- Flipping heads twice: P(H and H) = 0.5 × 0.5 = 0.25
Key Takeaway: Kolmogorov's three axioms (non-negativity, certainty, additivity) form the mathematical foundation from which all probability rules emerge, enabling us to rigorously calculate chances of events in machine learning models.