Why is my retention curve lying to me?
Almost always because it pools cohorts from different periods. A blended curve mixes people who joined under an old product with people who joined under a new one, and the average describes nobody. Split by joining month and the single curve usually turns into two that disagree.
The grandmother version
Someone asks how tall children are. You measure a toddler and a teenager, average them, and announce that children are four feet tall.
Perfectly accurate. Describes nobody.
And if more toddlers arrive next month, the "average height of children" falls — without a single child having shrunk.
Vintage is the usual culprit
Retention is only comparable within a cohort of the same joining period. Pool several and you are averaging groups who met different onboarding, different pricing and different marketing.
The giveaway: your blended curve moves and nobody changed anything. What changed was the mix — you acquired more of a weaker cohort, so the average fell while every individual cohort held steady.
Mix effects run the other way too
Simpson's paradox is not exotic here, it is routine. Retention can improve in every single cohort while the pooled number falls, purely because the newest and largest cohort is the weakest.
So run every headline cut by at least two segments before writing it down. If the finding reverses inside segments, the aggregate claim is wrong.
The other three traps
Averages. Show the distribution — percentiles and buckets — not the mean. Retention in a business with a heavy tail is not well described by its average.
Missing counts. Every rate ships with its absolute: "41% (312 of 761)". A dramatic curve on n=23 is a story about 23 people.
Undeclared windows. Decide what counts as "still here" before measuring. Roughly two weeks of silence is a reasonable threshold for most consumer products — state the one you used, because it changes the answer.
The short version
Never pool retention across cohorts of different vintages. Never report a curve without its counts. Never let an average stand in for a distribution.
Do those three and the curve stops lying.
Cohort retention and LTV by source, never pooled — Where the Money Comes From, €4,500.
See prices →