Bayes' Theorem
You'll learn: How to update your beliefs about an event's probability when you receive new evidence.
What Is Bayes' Theorem?
Bayes' Theorem is a formula that flips conditional probabilities around. Remember that conditional probability P(A|B) tells us "the probability of A given B happened." Bayes' Theorem lets us compute P(B|A) when we know P(A|B), using information we already have about the individual probabilities.
The Formula
Starting from the definition of conditional probability you learned:
- P(A|B) = P(A ∩ B) / P(B)
- P(B|A) = P(A ∩ B) / P(A)
Since both equal P(A ∩ B), we can rearrange to get Bayes' Theorem:
P(B|A) = P(A|B) × P(B) / P(A)
Or in words: the probability of B given A equals the probability of A given B, times the base probability of B, divided by the base probability of A.
Why It Matters
Think of a medical test: you know P(positive test | disease) from lab studies, but what you really want is P(disease | positive test). Bayes' Theorem makes this possible by incorporating how common the disease is in the population—P(disease)—and how often the test is positive overall—P(positive test).
Example Intuition
Suppose 1% of people have a disease. A test is 95% accurate. If you test positive, are you 95% likely to have the disease? No! Bayes' Theorem shows the true probability is much lower because false positives from the 99% healthy population dominate.
Key Takeaway: Bayes' Theorem lets you reverse conditional probabilities and update beliefs with evidence—essential for reasoning under uncertainty in machine learning.