Bayesian Inference via Bayes' Rule
What You'll Learn
You'll learn how to apply Bayes' theorem to update your beliefs about model weights after observing training data, combining prior knowledge with evidence.
The Core Idea
In Bayesian machine learning, we don't just find a single "best" set of weights—we maintain a distribution over possible weights. Bayes' rule tells us how to update this distribution when we see data:
P(weights | data) = P(data | weights) × P(weights) / P(data)
Breaking this down in plain English:
- P(weights): Your prior distribution—what you believed about the weights before seeing any data
- P(data | weights): The likelihood function—how probable the observed data is for each possible weight configuration
- P(data): A normalizing constant (often called the evidence or marginal likelihood)
- P(weights | data): The posterior distribution—your updated beliefs about the weights after seeing the data
An Intuitive Analogy
Imagine you're a chef learning how much salt to add to a recipe:
- Prior: Your initial guess based on experience with similar dishes
- Likelihood: How well each salt amount explains the taste feedback you're getting
- Posterior: Your refined belief about the ideal salt amount, combining both your initial intuition and the taste evidence
How It Works
When you observe training data, Bayes' rule mathematically combines your prior beliefs with the likelihood of that data to produce a posterior distribution. This posterior becomes your new, more informed belief about which weight values are plausible.
Key Takeaway: Bayesian inference via Bayes' rule transforms prior beliefs about model weights into posterior beliefs by incorporating the evidence from observed data, giving you a full distribution rather than a single point estimate.