Usability Testing With 5 Users: Is It Enough?

Great things come in 5’s. Five Guys burgers and fries. 5 star ratings on Google Reviews. We even use 5 testers in our design sprint!

24 years ago, Jacob Nielsen, of the world-leading research group the Nielsen Norman Group, agreed, making the bold claim that you only needed to test the usability of your product with 5 users.

It was game-changing.

Product managers at huge global companies and entrepreneurs with the next big thing could test concepts, fast and at incredibly low cost.

But over the years, it’s been misinterpreted to be seen as a silver bullet for all testing purposes, resulting in products that go to market prematurely and cost more to get right after the fact.

Read on to find out where people are getting the 5 users concept right, and where people are getting it wrong.

Nielsen’s Argument: Why usability test with only 5 users?

In an era before the smartphone and full of dial up internet connections, Nielsen and his research partner Landauer found that for computing systems, they could actually estimate the number of insights from a given number of users.

In fact, they estimated that the first 2 testers could come across over 50% of usability issues, a staggering amount of feedback, that could ultimately be the difference between a costly re-development post-live and a quick tweak in design.

For every person they added past 3, the same themes would begin to appear, with fewer fresh insights, finally landing on 5 (85% of usability issues found) users as the sweet spot for identifying high priority issues.

Key Takeaways From Nielsen

The premise of 5 users tries to counter the belief that user testing needs to be elaborate, expensive and on a huge scale
Over 85% of usability issues can be found through 5 users testing the product/experience
Adding more users than 5 results in fewer new findings being made and you seeing the same things over and over
3 users is a critical number in the testing phase, verifying the first two users’ issues and likely producing a small amount of their own
Emphasises the need for tests to be small scale (so not involving huge scopes)

Elaborate usability tests are a waste of resources. The best results come from testing no more than 5 users and running as many small tests as you can afford.

Jakob Nielsen

Is 5 Users Still The Usability Sweet Spot?

In an era where smartphones are now the norm and computing has come along way from the 2000’s. However, a general Google search will tell you that the broad consensus is that 5 users is still enough for usability testing, staying true in the new technology heavy environment.

There are some caveats and misunderstandings about how the 5 users principle should be used though that you need to keep in mind:

Great for qualitative, but not quantitative

Raluca Budiu, Director of Research at NNG, for example argues that 5 users isn’t a silver bullet for all testing.

She argues that although 5 is great for testing usability issues from a qualitative standpoint (where anything users identify is objectively an issue for everyone), there are issues that come with using it for quantitative purposes.

Where you’re looking for any of the following, 5 just doesn’t end up being a representative number for your user base:

How many people could complete the task?
How quick could they do it?
How satisfied are users?

Testing on 5 users ends up not being a big enough sample for that purpose, leaving issues like bias and individual circumstances to skew results far greater in a smaller sample than a big sample.

Iteration will inevitably demand more than 5 users

As Nielsen even mentions in his own post, you need 15 people to get 100% of usability issues, suggesting that you split your budget across 3 different 5 person usability tests.

If you’ve been paying attention at home, then you’ll realise that what Nielsen really means is that 5 is a great starting point before you iterate.

And even 15 may feel too few in the end due to the possibility that you may introduce more issues with each iteration.

In short, 5 is an amazing rule of thumb for a starting point, but don’t budget your whole project on 5 users.

Considerations: What Makes 5 Users Too Few?

Your goal is to test quantitatively to gathering statistical evidence of product/experience success
Your searching for something other than usability issues
Your process might be too complex
Your test relies on having input from different types of users
You don’t plan on iterating your designs/product/experience

As a team that wants to help our partners deliver the best possible user experience, we operate 5 person user tests during our design sprints and product design sprints.

Not only is it great for producing great insights fast, but Nielsen's premise of smaller tests works hand-in-hand with the sprint concept of experimentation and iteration.

James Hamilton

Want to prototype and usability test a new concept in just 4 days?

Fill in the form below and let’s have a chat!

Continue Learning…

Company Updates