Likelihood Ratio Tests
You'll learn how to compare nested models using the ratio of their likelihoods—a powerful alternative to the F-test.
What Is a Likelihood Ratio Test?
A likelihood ratio test (LRT) compares two nested models by examining how well each explains the data. Instead of using sums of squares (like the F-test), it uses log-likelihoods—measures of how probable the observed data is under each model.
The test statistic is:
LR = -2 × (log-likelihood of simpler model - log-likelihood of complex model)
Under certain conditions, this statistic follows a chi-squared distribution with degrees of freedom equal to the difference in the number of parameters between models.
Why Use Likelihood Ratios?
The F-test assumes normally distributed errors and uses least squares. The LRT is based on maximum likelihood estimation, which:
- Works even when assumptions are relaxed
- Applies to generalized linear models (logistic regression, Poisson, etc.)
- Provides a unified framework across different model types
Think of it like comparing two explanations for a mystery: the likelihood ratio tells you how much more convincing the detailed explanation is compared to the simpler one.
How It Works
- Fit your simpler (nested) model and compute its log-likelihood
- Fit your more complex model and compute its log-likelihood
- Calculate the test statistic: LR = -2 × (ℓ_simple - ℓ_complex)
- Compare LR to a chi-squared distribution with df = difference in parameters
- If p-value < α, the complex model fits significantly better
Key Takeaway: Likelihood ratio tests compare nested models by examining the ratio of their log-likelihoods, offering a flexible alternative to F-tests that extends beyond normal linear regression.