Best Practices

“Usability” Myths

There are common misconceptions about how to design a product with a good user experience and even why it’s important. Here’s a look at some of the myths and impediments to a compelling user experience.

Some of the more common research-related myths we've encountered are:

• Testing or other research methods are not scientifically based
• We already do user interaction testing, but it goes by another name
• We haven't heard of any user interaction problems with our products, so the whole product experience of our products is good
• We asked our users for design input and implemented what they asked for, so we have good design

 

Myth 1 – Testing Isn't Scientifically Based

This myth seems to come up for a number of reasons. The objection is often based on the fact that only 8 users are utilized in a typical test. While this is a small number if you intend to use the results to represent a larger population, if your intent is to uncover potential problems with the product, it is not.

A number of studies have shown that 6-8 users will typically uncover 80% of the problems with a given interface. That's a powerful diagnostic effect! While this rule does not guarantee a foolproof interface and more users may be needed depending on the range of uses (variability) of the product, it is a valuable method for improving the user experience.

Myth 2 - We Do Usability Testing Already

“Usability” and “usability testing” are popular buzzwords in business today particularly amongst the technology companies we tend to work with.

We've heard the term “usability testing” used to describe verification tests, market trials, product release with a feedback form and most often focus groups where potential users are asked how they 'think' they would interact with the product. While these are valuable methods to gather user data, they are not a means of real user interaction testing.

In a scientific test users are asked to complete tasks, without being told how to, that a typical user would be expected to perform within the product. The product is then rated based on how successful the users are in completing those tasks. Obstacles to completion are identified and recommendations to improve the product are made. This can be an important distinction if the developers or business management believe they are getting data on a certain aspect of user interaction when they may be receiving user preferences, user feed-back or suggestions for new functionality, or focus group feedback on the potential "receptivity" of a given product or feature.

Here are some signs to watch for to indicate that the type of data gathering being done is not user interaction testing:

• The test is very late in the development cycle - this is usually a form of user validation to make sure the desired functionality is available.
• The tasks are described or demonstrated to users and they are asked for their opinions, rather than being given a specific task to perform.
• The test facilitator is not experienced or aware of many other aspects of user interaction or human factors research.
• There's little or no interaction with a prototype (i.e. a paper or software application mock up of the product).

Myth 3 - we haven't heard of any problems, so we don't have any.

Here is a classic case of the old saying "absence of evidence is not evidence of absence". While it may be that there are no significant user interaction problems reported with a product, we've found in many instances the problems just have not been. This happens for several reasons:

• users simply don't report problems.
• users report problems, but there's no systematic means of gathering the data.
• problems are misinterpreted or misattributed.

Quite often user interaction problems get reported in other categories because users don't directly report it as a user interaction problem and customer support doesn't recognize a user interaction problem underlying the stated problem. The report comes in as ".is not working" or an intermittent problem with the operation of the product is reported.

Whatever the reason, unless you do scientifically based research to gather user interaction data it's not possible to know with any reasonable degree of certainty what problems are out there and what problems are not.

Food for thought; of the problems coming into your Support Group, how many are real, test-case supported bugs? It's usually less than 10%. Can it then be considered that, of the rest, the majority can be attributed to "user interaction"... if you could fix even 20% of the remainder therefore... how much would that save your company?

Myth 4 - we asked our users for design input and we implemented what they asked for, so we'll have a good design

While it is crucial to get input from users, asking them for design suggestions can often lead to more user interaction problems. For the simple reason they are users, not designers.

Users can identify the problems with the product but won't typically know the solutions. For instance, I can tell my mechanic that my car won't start in cold weather, but don't expect me to tell him how to fix it.

Improving user interaction means, identifying and making design trade-offs which still achieve a set goal or outcome for the users, seeking out negative interactions between design decisions and accurately predicting user behaviour.

For all these reasons its important not to expect your users to give you the design, keeping in mind that it is essential you listen to the underlying problem they are identifying.

In summary, the key to good user interaction research is similar to other types of research. Is the research methodology appropriate for the type of information you need? Are you aware of the potential contributors to error in the data you are gathering? Does your test facilitator have an understanding of human factors and user interaction principles to correctly interpret results of the test?

To be able to answer these questions effectively is to have the basis for good user interaction research.