In this lesson, you will learn how to compute fundamental descriptive statistics for numeric datasets using Python’s built-in statistics module while mastering the distinction between population and sample variance.
When working with data, we often need to understand the "center" of a distribution and how much that data spreads out. Python makes this straightforward with the statistics module.
Computing Central Tendency
The mean and median represent the center of your data. The mean() function calculates the arithmetic average, while median() finds the middle value of a sorted dataset.
import statistics
data = [10, 2, 38, 23, 38, 23, 21]
print(f"Mean: {statistics.mean(data)}")
print(f"Median: {statistics.median(data)}")
Distinguishing Variance
Variance measures the "spread" of your data points relative to the mean. It is crucial to choose the right function based on your data source:
- Sample Variance (
variance): Use this when your dataset is a subset or sample of a larger population. It uses Bessel’s correction (dividing by ) to provide an unbiased estimate. - Population Variance (
pvariance): Use this when your dataset represents the entire population. It divides the sum of squared deviations by .
# Assuming 'data' is just a sample of a larger group
sample_var = statistics.variance(data)
# Assuming 'data' contains every possible observation
pop_var = statistics.pvariance(data)
print(f"Sample Variance: {sample_var:.2f}")
print(f"Population Variance: {pop_var:.2f}")
Notice that the sample variance is slightly higher than the population variance. This is mathematically intentional; by dividing by , we account for the fact that a sample is likely to have less variance than the full population, thereby correcting the estimation bias. If you are ever unsure, always default to variance() for real-world data samples.
Key Takeaway: Use statistics.mean and median for central tendencies, and always choose variance for samples versus pvariance for complete population datasets to ensure your statistical analysis is mathematically sound.