Formulating Hypotheses
You'll learn how to write clear, testable hypotheses that set your A/B test up for success.
Why Hypotheses Matter
In A/B testing, a hypothesis is your explicit prediction about what will happen when you make a change. Without one, you're just throwing spaghetti at the wall. A proper hypothesis forces you to think clearly about what you expect to change, how much, and in which direction.
Think of it like a scientist planning an experiment: "If I water this plant daily instead of weekly, I predict it will grow 20% taller in a month." Not vague—specific, testable, and falsifiable.
The Anatomy of a Good Hypothesis
A testable hypothesis for A/B testing has three essential components:
- The change you're making (your treatment or variant)
- The metric you're measuring (your success metric from the previous lesson)
- The direction and magnitude of expected effect
Bad hypothesis: "The new button will improve engagement."
Good hypothesis: "Changing the checkout button from blue to green will increase conversion rate by at least 2 percentage points."
Directional vs. Non-Directional
- Directional (one-tailed): "The new landing page will increase sign-ups by 5%"
- Non-directional (two-tailed): "The new landing page will change sign-ups"
Directional hypotheses are stronger when you have a clear expectation based on prior research or domain knowledge. They also require smaller sample sizes—but be careful: if the effect goes the opposite direction, you might miss it.
Key Takeaway: A well-formulated hypothesis specifies exactly what you're testing, which metric matters, and what change you expect—making your A/B test both focused and interpretable.