Characteristic Functions (Optional)
What you'll learn: You'll discover an alternative tool to MGFs that works for every probability distribution, no exceptions.
Why Characteristic Functions Exist
You've learned that moment generating functions (MGFs) are powerful tools for finding moments and analyzing sums of independent random variables. But there's a catch: MGFs don't always exist! Some distributions (like Cauchy) don't have finite moments or an MGF that converges.
Characteristic functions solve this problem elegantly. They always exist for any probability distribution.
The Core Idea
A characteristic function is defined using complex exponentials instead of real exponentials:
φ_X(t) = E[e^{itX}]
where i is the imaginary unit (√−1) and t is a real number.
Think of it this way: an MGF asks "what's the expected value of e^{tX}?" which can blow up to infinity. A characteristic function asks "what's the expected value of e^{itX}?" Since |e^{itX}| = 1 always (complex exponentials just rotate on the unit circle), the expectation always exists—it's bounded.
Key Properties
- Always exists for any random variable
- Uniquely determines the distribution (like MGFs)
- For sums of independent RVs: φ_{X+Y}(t) = φ_X(t) · φ_Y(t)
- Relates to MGF: if M_X(t) exists, then φ_X(t) = M_X(it)
- Connects to Fourier transforms in signal processing
When to Use
Use characteristic functions when dealing with distributions that lack MGFs or when theoretical guarantees of existence matter. For most standard distributions you work with daily, MGFs are simpler—but characteristic functions are the more universal tool.
Key Takeaway: Characteristic functions are the "always-work" alternative to MGFs, using complex exponentials to guarantee existence for every probability distribution.