Attribution Pitfalls and Common Errors
You'll learn the three most dangerous mistakes that invalidate metric attribution: double-counting, confounders, and false attribution.
Why Attribution Goes Wrong
Even with control groups and careful time windows, attribution can fail spectacularly. Three errors are especially common and costly.
Double-Counting
This happens when multiple teams claim credit for the same metric lift. For example, Marketing launches a campaign and Product ships a new feature in the same week. Both teams attribute the 15% revenue increase to their initiative. The company thinks it gained 30% when it only gained 15%.
Analogy: Two people pushing a door open together, each claiming "I opened it 100%." The door only opened once.
Prevention: Use additive decomposition or agreed-upon attribution rules before initiatives launch.
Confounders
A third factor drives both your "cause" and "effect," creating spurious attribution. You launch a feature on Black Friday and attribute sales lift to the feature—ignoring that sales always spike then.
Analogy: Ice cream sales and drowning both increase in summer. Ice cream doesn't cause drowning; warm weather is the confounder.
Prevention: Compare against seasonal baselines and year-over-year controls, not just last week. Use time-series decomposition to isolate trends from your intervention.
False Attribution (Selection Bias)
Assigning credit based on correlation alone. Users who adopted your new feature also had higher retention—but they were already your most engaged users who would've stayed anyway.
Analogy: Umbrellas don't cause rain just because they appear together.
Prevention: This is why you learned about control groups in the previous lesson. Randomized assignment breaks selection bias. If randomization isn't possible, use propensity score matching or regression adjustment to account for pre-existing differences.
Key Takeaway: Attribution fails when you double-count wins, ignore confounding variables that drive both cause and effect, or mistake correlation for causation without proper controls—always verify your attribution logic against these three errors.