Choosing α Before Analysis
You'll learn why committing to your significance level before you look at the data is essential for honest, reproducible science.
The Pre-Registration Principle
Imagine a basketball player who takes a shot, watches where the ball lands, then paints the target around it afterward. They'd hit "bullseye" every time! Similarly, if you set your significance level (α) after seeing your p-value, you can always make your result look significant.
Choosing α before analysis means deciding your threshold for rejecting the null hypothesis—typically 0.05, 0.01, or 0.10—before you collect or analyze your data. This pre-commitment prevents you from unconsciously (or deliberately) moving the goalposts to get the answer you want.
Why This Matters: Avoiding P-Hacking
When researchers peek at results and then adjust their α, they engage in p-hacking—manipulating the analysis until they get p < 0.05. This might mean:
- Lowering α to 0.10 when p = 0.08 ("close enough!")
- Raising α requirements when results are too significant ("let's be more conservative")
- Running multiple tests and reporting only those that "work"
Pre-specifying α keeps you honest. You set your risk tolerance for Type I errors (false positives) based on the context—medical trials might use α = 0.01 for safety, while exploratory marketing tests might accept α = 0.10—not based on what makes your current data look good.
Valid Inference Requires Pre-Commitment
The entire logic of hypothesis testing assumes you defined your decision rule before observing the outcome. Your p-value only has its stated meaning if α was fixed in advance. Otherwise, you're invalidating the probability calculations that make inference possible.
Key Takeaway: Always choose and document your significance level α before analyzing data—it's the cornerstone of valid statistical inference and protects against p-hacking.