Power Analysis and Study Design
What You'll Learn
You'll learn how to use power analysis practically—before collecting data—to design experiments that can actually detect real effects, and how to interpret results when tests aren't significant.
The Core Idea
Power analysis isn't just a theoretical exercise—it's your planning tool. Before you run an experiment, you use power analysis to answer: "How many subjects do I need to have a reasonable chance of detecting the effect I care about?"
Think of it like planning a fishing trip. You wouldn't bring a tiny net if you're trying to catch enough fish to feed your family. Similarly, you need enough sample size to "catch" a real effect if it exists.
Designing Studies with Power Analysis
Step 1: Specify what matters
- What's the smallest effect size you care about detecting? (clinical significance)
- What Type I error rate (α) will you use? (typically 0.05)
- What power do you want? (typically 0.80 or 0.90)
Step 2: Calculate required sample size Using the formulas and methods from the previous lesson, determine how many observations you need. This happens before data collection.
Step 3: Account for real-world factors
- Expected dropout rates? Add 10-20% more participants
- Unequal group sizes? May need larger total sample
- Multiple comparisons planned? Adjust α accordingly
Interpreting Non-Significant Results
When p > α, you have two possible explanations:
- There's truly no effect (null is true)
- There is an effect, but your study lacked power to detect it
Post-hoc power analysis tells you: "Given the effect size we observed, what was our actual power?" If power was low (< 0.50), you can't conclude much—your study was underpowered.
Key Takeaway: Power analysis transforms study design from guesswork into principled planning, ensuring your experiments are adequately sized to detect meaningful effects and helping you interpret negative results responsibly.