Order Statistics
What you'll learn: You'll learn how to find the probability distributions of the minimum, maximum, and median when you draw a random sample from a population.
The Core Idea
Imagine you draw n random values from a population and then arrange them from smallest to largest. These sorted values are called order statistics. The smallest is the 1st order statistic, the largest is the nth order statistic, and so on.
Order statistics are themselves random variables—their values depend on which random sample you happened to draw. The key question: what are their distributions?
Deriving the Distributions
Minimum of a Sample
Let X₍₁₎ denote the minimum of n independent random variables from a population with CDF F(x).
Key insight: The minimum exceeds a value x only if all sampled values exceed x.
For independent samples:
- P(X₍₁₎ > x) = [1 - F(x)]ⁿ
- Therefore: F₍₁₎(x) = 1 - [1 - F(x)]ⁿ
Maximum of a Sample
Let X₍ₙ₎ denote the maximum.
Key insight: The maximum is at most x only if all sampled values are at most x.
- F₍ₙ₎(x) = [F(x)]ⁿ
Median and General Order Statistics
For the k-th smallest value X₍ₖ₎, the CDF involves counting how many of the n values fall below x—this uses binomial probabilities:
F₍ₖ₎(x) = Σᵢ₌ₖⁿ (ⁿᵢ)[F(x)]ⁱ[1 - F(x)]ⁿ⁻ⁱ
The median corresponds to k = ⌈n/2⌉ for odd n.
Analogy
Think of n runners finishing a race at random times. The order statistics are the finish times ranked 1st place, 2nd place, ..., last place. Each rank has its own distribution.
Key Takeaway: Order statistics transform a random sample into sorted random variables; their distributions depend on the parent CDF and sample size through powers and binomial sums.