Course contentsShow
← Machine Learning and Deep Learning
Lesson 54 of 3,5381. Mathematical Foundations for Machine LearningFree lesson

Probability Axioms and Basic Rules

Learn Kolmogorov's axioms and derive fundamental probability rules including addition and multiplication.

Probability Axioms and Basic Rules

What You'll Learn

You'll discover Kolmogorov's three fundamental axioms that define probability and how to derive essential rules for combining probabilities.

The Foundation: Kolmogorov's Axioms

After defining sample spaces and events (our previous lesson), we need formal rules to assign probabilities. Russian mathematician Andrey Kolmogorov established three axioms that form the bedrock of probability theory:

Axiom 1: Non-negativity

The probability of any event A is always non-negative: P(A) ≥ 0

Think of it like this: you can't have "negative chance" of something happening.

Axiom 2: Certainty

The probability that something in the sample space occurs is 1: P(S) = 1

If you roll a die, you're absolutely certain to get some number from 1 to 6.

Axiom 3: Additivity

For mutually exclusive events (events that cannot happen simultaneously), probabilities add: P(A or B) = P(A) + P(B) when A and B cannot both occur.

Rolling a 2 or a 5 on one die roll: P(2 or 5) = P(2) + P(5) = 1/6 + 1/6 = 1/3.

Derived Rules

From these axioms, we derive critical rules:

Complement Rule: P(not A) = 1 - P(A)

  • If the probability of rain is 0.3, the probability of no rain is 0.7.

Addition Rule (General): P(A or B) = P(A) + P(B) - P(A and B)

  • Accounts for overlap when events can co-occur.

Multiplication Rule: For independent events: P(A and B) = P(A) × P(B)

  • Flipping heads twice: P(H and H) = 0.5 × 0.5 = 0.25

Key Takeaway: Kolmogorov's three axioms (non-negativity, certainty, additivity) form the mathematical foundation from which all probability rules emerge, enabling us to rigorously calculate chances of events in machine learning models.