Course contentsShow
← Mathematics
Lesson 3113 of 3,12160. Statistics — ModelingFree lesson

Discrete Parameter Spaces and Bayesian Updating

Apply Bayesian updating when the parameter takes on a finite set of values, computing posteriors via weighted likelihoods.

Discrete Parameter Spaces and Bayesian Updating

You'll learn how to update beliefs about a parameter when it can only take a few specific values, using weighted likelihoods.

The Core Idea

In many real-world scenarios, your parameter of interest doesn't vary continuously—it takes on just a handful of distinct values. Maybe a coin is either fair (p = 0.5), slightly biased (p = 0.6), or heavily biased (p = 0.8). When your parameter space is discrete like this, Bayesian updating becomes remarkably straightforward: you multiply each possible parameter value's prior probability by its likelihood given the data, then normalize to get posteriors.

How It Works

Think of it like a racing competition where each parameter value is a horse. Before the race (before seeing data), each horse has odds—your prior probabilities. When the race happens (you observe data), each horse's performance is its likelihood: how probable the data would be if that parameter were true. The finishing positions—your posteriors—are determined by multiplying each horse's starting odds by its performance, then rescaling so all probabilities sum to 1.

Formula:

Posterior(θᵢ) = [Prior(θᵢ) × Likelihood(data | θᵢ)] / Σ[Prior(θⱼ) × Likelihood(data | θⱼ)]

The denominator ensures your posteriors are valid probabilities (summing to 1).

Example Structure

Suppose θ can be {0.3, 0.5, 0.7} with priors {0.2, 0.5, 0.3}. After observing 7 successes in 10 trials, compute each likelihood using the binomial formula, multiply by the priors, then divide each product by their sum. The result: updated beliefs about which θ is most plausible.

Key Takeaway: In discrete parameter spaces, Bayesian updating reduces to computing weighted likelihoods—multiply priors by likelihoods for each possible parameter value, then normalize.