The Grandmother Test

Nobody can agree what "active user" means

There is no correct definition, and hunting for one is the mistake. A definition is a business judgment call, so write it down with seven fields and test it: hand it to two people separately and have them compute it. If they get different numbers, the definition failed and you fix it before anyone reports it.

Gergana Tyaneva · 21 September 2026 · 12 years in product and marketing analytics

The grandmother version

Ask three people in a shop who counts as a "regular". One says anyone who's been in twice. One says anyone who knows her name. One says anyone who's spent over fifty pounds.

None of them is wrong. They're answering different questions — restocking, staffing, and the Christmas card list.

The trouble only starts when all three write "regulars" on a piece of paper and put it in front of the owner, who thinks it's one number.

The seven fields

Every metric gets all seven, and a definition missing any one of them is not finished: name (specific enough that nobody reaches for it by accident), numerator (exactly what is counted), denominator (the population, stated as a population), time window (calendar month versus trailing 30 days versus a snapshot), inclusions, exclusions (bots, internal accounts, test users, guests, refunds), and rationale — why this choice and not the reasonable alternative.

The rationale field is the one people skip and the one that saves the argument six months later.

One population. Three defensible definitions of “active”.logged in once this month41,200did a core action this month12,840did a core action 3+ days3,910All three are correct. The trouble starts when all three are called “active users”.

The two-person test

Send the written definition to two people separately, with no contact between them, and ask each to compute it. If the numbers differ, the definition failed — find the field that let them diverge and close it.

Run this before delivery, not after. It takes an hour and it is the difference between a definition and an opinion.

Ship more than one

Where several definitions are defensible, publish each one, labelled with the decision it serves and the person who uses it. One number for the board, one for the product team, both written down.

A written spec settles in advance what would otherwise be settled in a hallway by whoever is more senior.

The short version

Before you adopt any headline metric, ask your team how they would game it. If the answer comes easily, the metric is broken.

And be careful with anything reported publicly. Once a number is published it stops being a measurement and becomes a promise — after which people start protecting the number instead of the thing it was measuring.

The Measurement Spec — €900, one week, two-person tested before you get it.

See prices →

More answers

Attribution — Which channel actually pays?Plumbing — Our numbers live in eight tools and no two agreeTime — Month-end reporting eats three days and nobody reads itMoney — We know people churn. We don't know who, when, or what it's worth

One question before you read on. I'd like to count visits with Google Analytics, which sets cookies and sends your IP to Google. Nothing has loaded yet, and the site works exactly the same either way. What's collected.