Conjugate Priors: Beta-Binomial Model
You'll learn how combining a beta prior with binomial data produces a beta posterior—making Bayesian updates elegantly simple.
In Bayesian inference, we start with a prior belief about a parameter, observe data via a likelihood function, and obtain a posterior belief. The beta-binomial model is the gold standard example of conjugacy—where the prior and posterior belong to the same distribution family, making computations straightforward.
Why Beta and Binomial?
Suppose you're estimating a success probability θ (like a coin's bias or a conversion rate). Your data are binary outcomes: x successes in n trials follow a binomial likelihood.
The beta distribution Beta(α, β) is the natural prior for θ because it:
- Lives on the interval [0, 1] (appropriate for probabilities)
- Has flexible shapes controlled by α and β (past "successes" and "failures")
- Is conjugate to the binomial—after observing data, your posterior is also beta!
The Magic Formula
Prior: θ ~ Beta(α, β)
Likelihood: x | θ ~ Binomial(n, θ) — observe x successes in n trials
Posterior: θ | x ~ Beta(α + x, β + n − x)
That's it! You simply add observed successes to α and observed failures to β. No integrals, no messy normalizations.
Intuition
Think of α and β as "prior pseudo-counts" of successes and failures. Observing real data just adds to these counts. If α = β = 1 (uniform prior, expressing ignorance), after 7 successes in 10 trials, your posterior is Beta(1+7, 1+3) = Beta(8, 4)—a tidy summary of all you've learned.
Why This Matters
Conjugacy means:
- Sequential updating: Process data batch-by-batch; today's posterior becomes tomorrow's prior.
- Closed-form results: No computational burden.
- Interpretability: Parameters have natural meanings.
Key Takeaway: The beta-binomial model exemplifies conjugate priors—where a beta prior and binomial likelihood yield a beta posterior by simply adding observed counts to prior parameters, enabling efficient, intuitive Bayesian inference.