Macadamian Blog

Product Management Research Pitfalls 6

In case you missed the introduction to this series, I’ll be identifying one product management user research pitfall per post. Missed the previous mistakes? Catch up!
Mistake #1     Mistake #2     Mistake #3     Mistake #4     Mistake #5

Mistake #6: Confusing Focus Groups and Usability Testing

FOCUS GROUPS ARE BEST USED IN THE EARLY STAGES of developing user requirements to provide rich information on the opinions and attitudes of the target audience. They are suited to an exploratory situation — getting a feel for the range of opinions, understanding the reasons underlying prefer¬ences, developing a basic list of user requirements — information that pro¬vides direction in the early stages of development.

Focus groups are not a good source of behavioral data. What people say they will do and what they actually do are two different things.

Our intentions and actions often vary because we can’t predict contextual factors that may alter our behavior in a given situation. We aren’t even fully aware of all our behaviors. Some everyday actions are so commonplace that we can’t remember them. For instance, if you ask someone what features they use on their telephone and how often, and then observe them using the phone, the results are usually staggeringly different.

The more complex the behavior pattern you ask people to predict or recall, the worse people are at predicting or recalling. So how do you get around this problem? That’s where observational techniques like user testing come in.

User testing provides behavioral information by assessing users’ performance on pre-determined tasks critical to the successful use of an application or website. Testing collects performance measures such as time to complete task, number of errors, and success rate, along with ease of use ratings for a series of tasks within a typical usage scenario. Using this information, you can identify the roadblocks and lesser obstacles to the successful use of the application or website.

The bottom line is that you use focus groups in a development program to help define user requirements. Once development has begun, you can employ user testing at set milestones — as early as a wireframe design — to test that the product being designed and built matches the original expectations from the focus group.

Both research methods play a useful role in supporting design decisions in any development project. To get the most value for your research budget, it’s important to know which method to use and for what purpose.