Course contentsShow
← Machine Learning and Deep Learning
Lesson 560 of 3,53814. Probabilistic and Bayesian Machine LearningFree lesson

Bayesian Inference via Bayes' Rule

Apply Bayes' theorem to compute the posterior distribution over weights given data and prior beliefs.

Bayesian Inference via Bayes' Rule

What You'll Learn

You'll learn how to apply Bayes' theorem to update your beliefs about model weights after observing training data, combining prior knowledge with evidence.

The Core Idea

In Bayesian machine learning, we don't just find a single "best" set of weights—we maintain a distribution over possible weights. Bayes' rule tells us how to update this distribution when we see data:

P(weights | data) = P(data | weights) × P(weights) / P(data)

Breaking this down in plain English:

  • P(weights): Your prior distribution—what you believed about the weights before seeing any data
  • P(data | weights): The likelihood function—how probable the observed data is for each possible weight configuration
  • P(data): A normalizing constant (often called the evidence or marginal likelihood)
  • P(weights | data): The posterior distribution—your updated beliefs about the weights after seeing the data

An Intuitive Analogy

Imagine you're a chef learning how much salt to add to a recipe:

  • Prior: Your initial guess based on experience with similar dishes
  • Likelihood: How well each salt amount explains the taste feedback you're getting
  • Posterior: Your refined belief about the ideal salt amount, combining both your initial intuition and the taste evidence

How It Works

When you observe training data, Bayes' rule mathematically combines your prior beliefs with the likelihood of that data to produce a posterior distribution. This posterior becomes your new, more informed belief about which weight values are plausible.

Key Takeaway: Bayesian inference via Bayes' rule transforms prior beliefs about model weights into posterior beliefs by incorporating the evidence from observed data, giving you a full distribution rather than a single point estimate.