Back to blog

To test or not to test?

We’ve become increasingly disillusioned with user testing. User testing involves putting screens or prototypes in front of users in a test lab and watching the decisions they make or collecting the opinions they have about proposed site structures or designs.

When companies started to crash in 1999-2000, user testing became popular with clients and agencies. In order to avoid an expensive failure like those of the dot com era user testing attempted to prognosticate user acceptance of new sites and features.

We quite enjoyed user testing as part of our process early on but then something happened. User testing became a political tool. As soon as clients and agencies saw that data from tests could steer development, user testing became a way to manage risk, manage political agendas and kill innovation.

User testing kills innovation because the fact is, people don’t know what they want. I was reminded of this while reading Malcolm Gladwell’s book Blink. He tells the story of the Aeron chair and how it almost never came to be. The Aeron chair is the most popular chair in the history of office furniture. But during testing it bombed. Customers hated the way it looked. If Herman Miller (the manufacturer) had listened to users, they would have missed the largest business opportunity in furniture history. More than that, listening to customers would have killed one of the most innovative designs ever.

Instead of user testing, try:

  1. Working with a design team that employs user centered thinking not user centered testing as their basis.
  2. See if features and designs work in the real world by releasing early and releasing often.
  3. Build sites with an eye to continually optimize them. This way decisions aren’t permanent. Something isn’t working, you change it.

Remember no data exists about the future. User testing may give you insight but it doesn’t predict success or failure.

Geoff Teehan More posts by Geoff Teehan