The Evidence (Marginal Likelihood)
You'll learn: What the "evidence" term in Bayes' Theorem does and why it's crucial for making the posterior distribution mathematically valid.
What Is the Evidence?
In Bayes' Theorem, the evidence (also called the marginal likelihood) is the denominator that normalizes the posterior distribution. It's written as P(Data) or P(D), and ensures that the posterior probabilities sum (or integrate) to 1, making it a proper probability distribution.
The formula for evidence is:
P(Data) = ∫ P(Data | θ) × P(θ) dθ
In plain English: you sum up (or integrate over) all possible parameter values, weighting each by how likely it makes the data (the likelihood) and how plausible that parameter value was before seeing data (the prior).
Why It Matters
Without this normalizing constant, you'd have an unnormalized posterior that looks right proportionally but doesn't add up to 1. Think of it like slicing a pie: the evidence tells you the size of the whole pie so each slice represents a true probability.
The catch: Computing the evidence is often the hardest part of Bayesian analysis. For complex models with many parameters, the integral can be intractable (impossible to solve analytically). That's why many Bayesian methods focus on sampling from the posterior without explicitly computing P(Data).
A Simple Analogy
Imagine you're distributing 100 votes among candidates based on polls (prior) and debate performance (likelihood). The evidence is the total number of votes—it scales everything so percentages are meaningful.
Key Takeaway: The evidence P(Data) is the normalizing constant in Bayes' Theorem that makes the posterior a valid probability distribution, but it's often computationally challenging to calculate directly.