Check yourself before you wreck yourself

User research: ask anyone, and you’ll find that the consensus is that it is very important. Whether it’s actually happening or not is another story.

For the sake of discussion, let’s just say it IS happening. If you are one of the people who make user research happen in your organization, we applaud you. You are striving for a user-centered eXperience, which we, as UX professionals, strive for as well.

Here’s where we introduce the predictable plot twist:

Just because you are doing user research does NOT mean your product will succeed.

**Gasp**, but why? The best way to make the right product is to talk to the user, right?

Well, right, but also wrong. I’ll explain.

Back to basics

While user research affords great flexibility in methodology, time commitment, and sample size, it still must follow a rigorous process to ensure the reliability of data.

Think back to grade school when you first learned about the scientific method which went something like:

The process of experimentation was established as the rigorous process that it is today to deliver the most reliable, unbiased findings possible. UX design, when done right, is nothing more than the scientific method on loop applied in a different context. As so:

And now, without further ado, I’ll finally get to the point:

5 ways you can screw up user research

As a process, user research has multiple potential points of failure:

1. Wrong User

Talking to the wrong user leads to finding the wrong insights, which leads to building the wrong product.

And, even if you have the right user in mind, here’s a word of wisdom: you can never replace your end user, no matter how well your SME, Support Rep, Sales Rep, PM, or CEO may know the user.

User insights must come straight from the actual user.

2. Wrong Questions

Let’s say you got the right user. Yay! But wait, you are asking the wrong questions…

Wrong questions lead to (1) missed insights which lead to (2) bad requirements which lead to (3) a product that the user may not need.

As a general rule of thumb, good research questions should find the what and the why. If you are finding out what users are doing (asking for a ¼” drill) but not why (they need a ¼” hole), chances are, you are not asking the right types of questions.

The questions you ask will define the problems you solve later in design.

3. Wrong Method

So the user you chose and the questions you want to ask are a-OK. Why does the method matter?

Wrong methods will not allow you to get the answers you need. This means time and money being spent on research but no actual insights as a result.

For example: Let’s say you’re building a new e-commerce site. You run a usability test to see if users are able to find specific products through the navigation, only to realize that there is a 75% failure rate. In this case, the better method to have used would have been a card sorting study earlier in the design process which would have given more insights into users’ natural ways of grouping items.

4. Poor Synthesis

Okay, assuming you’re fine up until this point: you identified the right user, asked the right questions, and chose the right method for research. What can go wrong in synthesis? Good data is good, right?

Right, good data is good, but good data interpreted the wrong way leads to bad insights. And, you know the drill by now… bad insights lead to bad product decisions which lead to sad customers.

Here are some warning signs to watch out for during synthesis:

  1. Changes being made based on the feedback of 1 user
  2. Failure to differentiate between new hypothesis vs facts
  3. Conclusions drawn without enough data corroborating the claim

5. Poor Presentation

And now, we’re at the final stretch. You’ve done everything right up to this point, so how do we take this to the finish line?

As the research champion — whether you’re a researcher, designer, PM, or anyone else — the final stretch all comes down to how effectively you can present these insights in such a way where people walk away with what’s important. Because the reality is:

  1. Most people don’t read the research reports
    Fully detailed research reports, as valuable their contents may be, are typically skimmed at best as cross-functional stakeholders seek a faster, more consumable summary of insights.
  2. People hear what they want to hear
    Confirmation bias is a real thing, and there is always the possibility of someone not fully understanding what you’re saying the way you intended.
  3. Data and insights get lost in an abyss
    More often than not, reports end up stored in a file on someone’s computer where the data can not have a lasting impact on future design endeavors.

The key here is to make sure the insights are delivered in a way where they are correctly heard and registered. Get creative! Here are some suggestions to get you started:

  1. For leadership stakeholders: use visual artifacts like journey maps and video highlight reels
  2. For development stakeholders: Create a trackable “to-do” list of prioritized fixes. This can be as simple as a spreadsheet with severity ratings per issue
  3. For PM and design stakeholders: Start a central insights repository

The moral of the story

Long story short: “check yourself before you wreck yourself.”

All jokes aside, the moral of the story is that the act of doing user research is not enough to guarantee a great product. User Research is a process which, while flexible, must be done right for it to drive value for your product and to effectively build a portfolio of insights.