Course contentsShow
← Data Science
Lesson 693 of 2,14518. Generalized Linear ModelsFree lesson

Overdispersion in Count Data

Detecting when variance exceeds the mean, violating Poisson assumptions, and checking goodness-of-fit.

Overdispersion in Count Data

You'll learn how to detect when your count data violates Poisson assumptions by having more spread than expected.

What is Overdispersion?

The Poisson distribution has a special property: its mean equals its variance. When modeling count data with Poisson regression, we assume this holds true. Overdispersion occurs when the actual variance in your data significantly exceeds the mean—violating this core Poisson assumption.

Think of it like a traffic counter: if cars arrive truly randomly and independently, Poisson works great. But if cars travel in clumps (maybe following traffic lights upstream), you'll see more variability than pure randomness predicts—that's overdispersion.

Why Does It Matter?

When overdispersion exists but you ignore it, your standard errors will be too small. This makes your coefficients appear more statistically significant than they really are, leading to overconfident conclusions and poor inference.

Detecting Overdispersion

Simple ratio check: Calculate the ratio of residual deviance to degrees of freedom from your fitted Poisson model. If this ratio is substantially greater than 1 (often >1.5 or 2), overdispersion is likely present.

Goodness-of-fit tests: Formal tests compare observed versus expected frequencies. Large chi-square statistics or small p-values suggest the Poisson model doesn't fit well—often due to overdispersion.

Variance inspection: Directly compare the empirical variance and mean of your count variable across groups. If variance consistently exceeds the mean (especially by large margins), suspect overdispersion.

What Causes It?

Common sources include unobserved heterogeneity (missing predictors), clustering in data, or excess zeros beyond what Poisson expects.

Key Takeaway: Overdispersion means your count data has more variability than the Poisson distribution allows; detect it by checking if variance exceeds the mean and examining goodness-of-fit statistics, because ignoring it produces unreliable standard errors and significance tests.