Course contentsShow
← Data Science
Lesson 321 of 2,1459. Hypothesis Testing FoundationsFree lesson

P-Values and the Strength of Evidence

Understand p-values as a continuous measure of evidence against the null, not a binary decision threshold.

P-Values and the Strength of Evidence

What you'll learn: How to interpret p-values as a continuous measure of evidence strength, not just a pass/fail test.

The Continuous Nature of P-Values

You've learned that a p-value tells you the probability of observing your data (or more extreme) if the null hypothesis were true. But here's what many beginners miss: a p-value isn't just about crossing the 0.05 threshold—it's a spectrum of evidence strength.

Think of p-values like a dimmer switch, not a light switch. A p-value of 0.049 and 0.051 tell nearly identical stories about your data, yet one "rejects" and one "fails to reject" if you use α = 0.05 as a rigid cutoff. This binary thinking misses the nuance.

Understanding the Spectrum

  • p = 0.001: Very strong evidence against H₀ (highly unlikely to see this if H₀ were true)
  • p = 0.04: Moderate evidence against H₀ (somewhat unlikely under H₀)
  • p = 0.08: Weak evidence against H₀ (reasonably likely under H₀)
  • p = 0.40: Little to no evidence against H₀ (quite compatible with H₀)

Instead of asking "Did I reject?", ask "How much evidence do I have?" A p-value of 0.001 provides much stronger evidence than 0.049, even though both might lead to the same binary decision at α = 0.05.

Why This Matters

Using p-values as continuous measures helps you avoid arbitrary decision boundaries and better communicate uncertainty. You might write: "The data provide strong evidence against the null (p = 0.003)" rather than just "We reject the null hypothesis."

Key Takeaway: P-values are a continuous measure of how surprising your data are under the null hypothesis—smaller values mean stronger evidence against H₀, but the magic of 0.05 is just a convention, not a natural boundary.