Course contentsShow
← Data Science
Lesson 665 of 2,14518. Generalized Linear ModelsFree lesson

Canonical Form of Exponential Family Distributions

The mathematical expression f(y; θ, φ) = exp[(yθ - b(θ))/a(φ) + c(y, φ)] and what each component represents.

Canonical Form of Exponential Family Distributions

You'll learn the mathematical structure that unifies many common probability distributions into one elegant formula.

The Big Picture

In the previous lesson, you learned that many familiar distributions (Normal, Binomial, Poisson, etc.) belong to the "exponential family." Now we're looking at the canonical form — a standardized mathematical expression that reveals what all these distributions have in common.

The Formula

The canonical form is written as:

f(y; θ, φ) = exp[(yθ - b(θ))/a(φ) + c(y, φ)]

Let's decode each piece:

y

The observed data value (the response variable).

θ (theta)

The natural (canonical) parameter — connects directly to the mean of the distribution. This is what changes when your predictors change in GLMs.

φ (phi)

The dispersion parameter — controls the spread or variability. For many distributions (like Poisson or Binomial), this is fixed at 1.

b(θ)

A specific function of θ that determines the distribution's mean and variance. Different distributions have different b(θ) functions.

a(φ)

A function of the dispersion parameter, often just φ itself or φ/n.

c(y, φ)

A normalizing term that depends on the data and dispersion but not on θ.

Why This Matters

This form lets us treat diverse distributions uniformly in Generalized Linear Models. Once you express a distribution in canonical form, you automatically know how to fit it using the same algorithmic machinery.

Key Takeaway: The canonical form f(y; θ, φ) = exp[(yθ - b(θ))/a(φ) + c(y, φ)] provides a unified mathematical structure for exponential family distributions, where θ links to the mean, φ controls dispersion, and b(θ) determines the distribution's specific properties.