Comparing Cohorts Over Time
What you'll learn: How to analyze differences between cohorts to spot improving or degrading retention trends and uncover their root causes.
Now that you can visualize cohort data with heatmaps, the real power emerges when you compare cohorts side-by-side. The question shifts from "what does one cohort look like?" to "are newer cohorts performing better or worse than older ones?"
The Core Idea
Think of cohorts as graduating classes from a school. If the 2023 class has a 90% job placement rate after one year but the 2024 class only reaches 75%, you'd want to investigate: Did curriculum change? Was the job market different? Did student quality shift?
Similarly, compare cohort retention curves at the same elapsed time point. For example:
- Week 4 retention for January cohort: 45%
- Week 4 retention for February cohort: 52%
- Week 4 retention for March cohort: 55%
This shows improving retention over time — a positive signal.
What to Look For
Improving trends: Later cohorts retain better than earlier ones. Investigate what changed: Did you improve onboarding? Launch a sticky feature? Fix a critical bug?
Degrading trends: Newer cohorts drop off faster. Red flag. Possible causes: product changes that confused users, increased competition, quality of acquisition channels declined, or seasonal effects.
Stable patterns: All cohorts behave similarly. Your product experience is consistent, for better or worse.
Practical Investigation
When you spot divergence, cross-reference with:
- Product release timelines
- Marketing campaign dates
- Changes in acquisition channels
- Seasonality or external events
This turns cohort comparison from descriptive ("retention changed") to diagnostic ("retention improved because we redesigned the first-time user experience in February").
Key Takeaway: Comparing cohorts over time reveals whether your product is getting better or worse at retaining users, and directs you toward the changes responsible for those shifts.