MLE for Common Distributions
You'll learn: How to derive the maximum likelihood estimators for four fundamental probability distributions used throughout statistics.
Why This Matters
Now that you know how to find MLEs by differentiation, let's apply that method to the distributions you'll encounter most often in practice. Each distribution has its own likelihood function, but the process remains the same: write the likelihood, take the log, differentiate, and solve.
The Core Derivations
Binomial Distribution
For n trials with k successes, the MLE for probability p is simply: p̂ = k/n (the sample proportion)
The likelihood involves p^k(1−p)^(n−k), and maximizing gives us the intuitive result: your best estimate is the observed success rate.
Poisson Distribution
Given observations x₁, x₂, ..., xₙ from Poisson(λ), the MLE is: λ̂ = x̄ (the sample mean)
This makes sense: the Poisson parameter λ represents the mean number of events, so estimate it with your observed average.
Normal Distribution
For observations from N(μ, σ²), we get two MLEs:
- μ̂ = x̄ (sample mean)
- σ̂² = (1/n)Σ(xᵢ − x̄)² (note: divides by n, not n−1)
The MLE for variance uses n in the denominator, making it slightly different from the unbiased sample variance s².
Exponential Distribution
For Exponential(λ), the MLE is: λ̂ = 1/x̄ (reciprocal of the sample mean)
Since the exponential mean is 1/λ, we invert the sample mean to estimate the rate parameter.
Key Takeaway: MLEs for common distributions often have intuitive forms—sample mean, sample proportion, or simple functions thereof—reflecting how each parameter relates to observable quantities.