Intent-to-Treat Analysis
What you'll learn: How to measure true incrementality by analyzing all users based on their assigned treatment, not who actually received it.
The Core Problem
In incrementality tests (like holdout groups or geo-lift experiments), not everyone assigned to see your marketing actually sees it. Some users use ad blockers, skip ads, or simply ignore them. Conversely, some control group members might be exposed through spillover effects.
The naive approach would be to analyze only users who actually received the treatment. But this introduces selection bias—users who engage with ads might already be more likely to convert, inflating your results artificially.
What Intent-to-Treat Means
Intent-to-Treat (ITT) analysis compares outcomes based on the assignment to treatment or control, regardless of whether users actually received or engaged with the treatment.
Think of it like a clinical drug trial: if a patient is assigned to take a medication but stops halfway through, researchers still count them in the treatment group. This preserves the randomization and gives you the true causal effect of offering the treatment.
Why It Matters for Incrementality
When measuring marketing incrementality, ITT tells you: "If we run this campaign, what's the expected lift across everyone we intended to reach?" This is your true business impact.
If you only analyzed users who saw the ad, you'd overestimate incrementality because:
- You're comparing engaged viewers (treatment) vs. random users (control)
- The randomization is broken
- Results won't generalize to future campaigns
ITT preserves the integrity of your experiment and gives conservative, reliable estimates of actual incremental value.
Key Takeaway: Intent-to-Treat analysis measures incrementality by comparing all users based on their random assignment, not actual exposure—preserving randomization and providing unbiased causal estimates of true marketing impact.