Maximum Likelihood Estimation
What you'll learn: How to find the "best" parameter values for a probability model by choosing those that make your observed data most likely.
The Core Idea
Imagine you have a bag of coins, and you suspect some are weighted. You flip one coin 100 times and get 70 heads. What's your best guess for this coin's probability of heads?
Maximum Likelihood Estimation (MLE) answers: "Choose the parameter value that makes what you actually observed most probable." In this case, 70/100 = 0.7 would be the maximum likelihood estimate.
How It Works
You start with:
- A probability model with unknown parameters (like a Normal distribution with unknown mean μ and variance σ²)
- Your observed data (actual samples you collected)
MLE finds parameter values by:
- Writing down the likelihood function: the probability (or probability density) of observing your specific data, as a function of the parameters
- Finding the parameter values that maximize this likelihood
Think of it as asking: "Which parameter settings would make my observed data the least surprising?"
Connection to What You Know
Remember point estimation fundamentals? MLE is one specific method for creating point estimators. It uses the probability distributions and density functions you've learned to mathematically formalize "best guess."
The maximization often involves taking derivatives (using your calculus knowledge) and setting them to zero to find critical points—just like finding extrema of functions.
Key Takeaway: Maximum Likelihood Estimation finds parameter values that maximize the probability of observing your actual data, providing a principled, mathematically rigorous way to estimate unknown parameters from samples.