Defining Prior Distributions
You'll learn how to encode your initial beliefs about parameters into probability distributions before seeing any data.
What Are Prior Distributions?
In Bayesian inference, the prior distribution represents what you believe about a parameter before observing new data. It's the starting point for your analysis. Unlike frequentist methods that treat parameters as fixed unknowns, Bayesian analysis treats them as random variables with probability distributions.
Your prior can range along a spectrum:
Informative Priors
When you have strong beliefs or previous knowledge, you use an informative prior. For example, if you're estimating the click-through rate for a button on your website and past tests showed rates between 2-4%, you might use a Beta(20, 480) distribution centered around 4% with relatively low variance.
Analogy: Like starting a new cooking recipe when you already know how much salt typically works in similar dishes.
Weakly Informative Priors
These gently constrain parameters to reasonable ranges without being too specific. They prevent absurd values (like negative standard deviations or 150% conversion rates) while letting the data speak.
Uninformative (Flat) Priors
When you truly know nothing, you might use a flat or uninformative prior that assigns equal probability across all plausible values. A uniform distribution is the classic example.
Analogy: Like approaching a completely new cuisine with no preconceptions about flavoring.
The Balance
Choosing priors is part science, part art. Strong priors make your conclusions more stable with small samples but can bias results if wrong. Weak priors let data dominate but may give unrealistic estimates with limited data.
Key Takeaway: Prior distributions mathematically encode your initial beliefs about parameters—ranging from highly informative (based on strong evidence) to flat and uninformative (minimal assumptions)—and directly influence how much your conclusions will update when new data arrives.