Mean and Variance in the Exponential Family
What You'll Learn
You'll discover how the mean and variance of any exponential family distribution can be elegantly derived directly from its cumulant function using simple derivatives.
The Core Idea
Remember the canonical form of exponential family distributions? It includes a function b(θ) called the cumulant function or log-partition function. Here's the beautiful mathematical shortcut: you can extract the mean and variance of Y by taking derivatives of b(θ).
The formulas:
- E(Y) = b'(θ) — the first derivative of the cumulant function gives you the expected value
- Var(Y) = b''(θ) · a(φ) — the second derivative times the dispersion function gives you the variance
Think of b(θ) as a "information capsule" about your distribution. The first derivative tells you where the distribution centers (its mean), and the second derivative tells you how spread out it is (its variance). The dispersion term a(φ) acts as a scale factor for the variance.
Why This Matters
This elegant relationship means you don't need to memorize separate formulas for the mean and variance of each distribution in the exponential family (Normal, Binomial, Poisson, Gamma, etc.). Once you identify the cumulant function b(θ), two quick derivatives give you everything you need.
For example, in the Poisson distribution, θ = log(λ) and b(θ) = exp(θ) = λ. Taking derivatives: b'(θ) = λ (the mean) and b''(θ) = λ (which equals the variance, since a(φ) = 1 for Poisson).
Key Takeaway: The cumulant function b(θ) encodes both mean and variance through its first and second derivatives—a unifying property that makes exponential family distributions mathematically elegant and computationally efficient in GLMs.