Macadamian Blog

Product Management Research Pitfalls

This is the first in a series of 10 posts.

Today’s product managers know it’s their job to be in tune with the market and with real-world users. More and more, they turn to user research techniques like customer interviews, focus groups, analytics, personas and prototyping to discover what customers want next from their products. Managers from companies like Apple, SalesForce.com, and SAP, who incorporate user research into their development cycles have consistently released software that sets a new bar in the industry, so won’t these techniques benefit everyone?

UNFORTUNATELY, USER RESEARCH IS RIFE WITH PITFALLS. Often what customers say they want and what they really need are very different. Surveys, focus groups, and analytics can be misleading. And even if you do glean the correct insights from your research, incorporating the results into the next product release can be even more of a challenge. In this series, I'll identify ten of the most common errors that product managers and business analysts make when gathering user research and customer feedback, and explain how to avoid them.

Mistake #1

But We Asked Users What They Want!

HAVE YOU SEEN THE EPISODE OF THE SIMPSONS where Homer’s long-lost brother, who owns a car company, instructs his engineers to design a car that average people want by getting input from Homer — the typical middle American customer? Homer demands extra large cup-holders, tail fins, a bubble dome, and horns that play La Cucaracha. The car is an expensive flop, and his brother’s company goes bankrupt.

In the end, customers aren’t product designers. They will happily give you ideas for your product, but they love to jump straight to the solution as they see it without understanding interactions or the context in which people use the software.

Customers might even come up with a wish list of features or changes such as, “This button should be blue” and “I want to print with one click.” But that doesn’t give you insight into why they need that feature, or why it might (or might not) help other customers. When a customer asks for a feature, your first job is to understand the underlying problem — Why did they request that particular feature or fix? By getting out of the solution space, and staying in the problem space, you understand the motivations and context behind the feedback. (We’ll talk more about that later.)

Here’s an example. Imagine you are creating an application for border security. It scans passports and displays a warning if the traveler has a criminal record. Your customer, the owner of the new system, wants to be sure that the cus¬toms agent sees the warnings, so they tell you to make the screen blink red whenever the passport scan brings up a suspicious person. Now imagine a land border crossing at midnight, where the border guard sits in a glass booth. It’s dark, and the person crossing comes up as having a criminal record. The screen flashes red. It reflects off the glass, lighting up the whole booth — and is visible to the person in the car outside. They know they’ve been flagged. If the traveler is armed, you have the potential for a deadly situation.

Maybe lives don’t rest on your application, but blindly following user feedback without understanding the problem or context of use can have consequences for your software, too.