Common Statistical Tests
What you'll learn: You'll learn when and how to use t-tests, chi-square tests, and ANOVA to compare groups and test relationships in your data.
Why Statistical Tests Matter
Now that you understand hypothesis testing and error types, you need practical tools to apply these concepts. Statistical tests are standardized procedures that help you decide whether differences between groups are real or just random chance.
The Three Workhorses
t-tests
Use t-tests when comparing means between groups with continuous data. Think of comparing average heights between two groups of people.
- One-sample t-test: Does your sample mean differ from a known value?
- Two-sample t-test: Are two group means different (e.g., treatment vs. control)?
- Paired t-test: Are before/after measurements different for the same subjects?
Chi-Square Test
Use chi-square tests for categorical data to test independence or goodness-of-fit. Imagine testing whether customer preferences depend on their age group—both variables are categories, not numbers.
- Tests whether two categorical variables are related
- Example: Is model prediction accuracy independent of data source?
ANOVA (Analysis of Variance)
Use ANOVA when comparing means across three or more groups. It's like a t-test's bigger sibling. Think of comparing average model performance across four different hyperparameter settings.
- Tests if at least one group mean differs from the others
- Avoids multiple t-test comparisons (which inflate Type I errors)
Choosing Your Test
Ask yourself: What type of data do I have, and how many groups am I comparing? Continuous data with two groups → t-test. Categorical data → chi-square. Continuous data with 3+ groups → ANOVA.
Key Takeaway: t-tests compare means between groups, chi-square tests examine relationships between categorical variables, and ANOVA compares means across multiple groups—choose based on your data type and comparison structure.