Put a clickable thing in front of a human within a day

The argument for moving fast here has two halves, and only one of them survives checking. It matters which, because the half that fails is the half people quote in meetings.

The half that holds: fidelity barely matters

You do not need to build a good version to learn what a good version needs.

Virzi, Sokolov and Karis ran two experiments and found “substantially the same sets of usability problems were found in the low- and high-fidelity conditions.” Walker, Takayama and Landay replicated it for the web: “low- and high-fidelity prototypes are equally good at uncovering usability issues. Usability testing results were also found to be independent of medium.” Sauro’s review of eleven studies reaches the same place.

Same problems, same numbers, roughly the same severity. Which means the rough clickable thing you could have shown somebody yesterday would have taught you almost exactly what the polished one will teach you in three weeks.

That is the entire case for speed, and it is well supported. The cost of waiting is not better findings. It is three weeks.

The half that fails: five users

Here is where the usual version goes wrong, and it goes wrong at the source.

The five-users rule is everywhere. Nielsen’s own article says “The best results come from testing no more than 5 users and running as many small tests as you can afford.” But the model underneath it, from Nielsen and Landauer, actually puts the maximum benefit-to-cost ratio at four evaluations, and finds sixteen evaluators still worth their cost for a medium project. That is an economic argument about diminishing returns, not a claim that five people find everything.

And the variance is brutal. Faulkner drew random sets of five from a larger pool: “Some of the randomly selected sets of 5 participants found 99% of the problems; other sets found only 55%.” With ten, the worst set found 80%. With twenty, 95%.

Spool and Schroeder ran the same task across four production websites with 49 users and titled the paper “Testing web sites: five users is nowhere near enough.” Schmettow’s peer-reviewed verdict is blunter: “usability researchers often resort to either magic numbers or the geometric series formula; inaccurate for making predictions, both underestimate required sample size.” A meta-analysis lands on ten plus or minus two.

So do not say five users finds 85% of problems. It is not what the source says, and somebody will know.

The move

Show a rough clickable thing to a real person within a day, then keep doing it, because the number of sessions is what compounds and the fidelity is not.

The honest reconciliation of those two literatures is Nielsen’s actual advice, which is better than his headline: “running as many small tests as you can afford.” Five is not enough to be confident. But five today, five next week and five the week after is twenty, spread across three versions, which beats twenty on one version of something you already finished.

WHAT YOU LEARN                  WHAT IT COSTS

 rough clickable, day 1          one day
   -> substantially the same
      problems, per three
      separate studies

 polished build, week 3          three weeks
   -> substantially the same
      problems

 5 users, once                   half a day
   -> somewhere between 55%
      and 99% of problems.
      you will not know which

 5 users, three times,           a day and a half, spread out
 on three versions
   -> the variance stops
      mattering, because you
      are sampling repeatedly

What it costs

Rough things get judged as products, and this is the recurring hazard of the whole part. Say what it is before they touch it: this is a day old and mostly fake, I want to know if the shape is right.

Repeated small sessions need repeated recruiting, which is the actual friction. Five people once is a project. Five people every week is a habit somebody has to own, and if that is nobody, it quietly becomes five people once.

And none of this covers rare-path problems. Small samples find the problems most people hit. They will not find the thing that breaks for the customer with forty thousand records, which is what move 27’s real-data prototype is for. Use both.

Try this week

Take whatever you are working on and make it clickable by tomorrow. Hardcoded, ugly, three screens, no backend. It does not need to work, it needs to be pressable.

Then get one person who is not on your team in front of it for fifteen minutes. One. Not a study, not a recruited panel: somebody from support, or sales, or the person the ticket came from.

Watch, do not explain. Move 3 has the discipline for that part.

Then do it again next week with whatever it has become. The compounding is the whole move, and it starts with one person tomorrow rather than five people in a fortnight.