By: Jamie Yoo | Mon, Aug 12, 2019

Blog Tags User Experience; Pragmatic UX; UX Strategy; Business Strategy; UX Design

Why usability testing alone won’t get your business far

I’ll get straight to the point: most companies do not get the value they should be getting out of user research.
Not for lack of trying though. The problem lies in most people’s perception of user research: show my product to users and see if it’s user-friendly (usually in the form of a usability study).
For the record, you SHOULD be doing usability studies. But usability studies only offer a fraction of the value you could be getting out of user research.
Usability testing tells you whether or not you built the product right, but does it tell you if your product has market fit? Or where your new opportunities lie? Does usability testing make your company “user-centric?”
The answers are: no, no, and no.
Back in 2014, Satyam Kantamneni of UXReactor wrote about the “SFV Research Framework” which we now call the “Portfolio of Insights,” an integral part of our PragmaticUX framework.
Essentially, the Portfolio of Insights describes the 3 different types of research every company should be doing at any given time to learn:

1. What are the unsolved needs / what should my business focus on?
2. Did I build my product the right way?
3. How is my product performing with users?

We call these 3 different types:

1. Formative research:

Research that provides insights which are foundational in nature and inform where businesses should focus their priorities through methods like ethnographic studies.
Formative research is best used at the beginning stages of product development when you are trying to understand what user needs to solve.
It tells you what the right thing to build is.

2. Summative research:

Research that provides insights which validate the UX or UI of a product or service achieved methods like usability studies.
Summative research is best used when you have a design stimuli users can react to — whether it be sketches, wireframes, visuals, or a first build — during the design or development stage.
It tells you if you built the right thing right.

3. Sensorial research:

Insights which come from data that gives an indication or a “sense” of how a product is doing, using data collected from instrumentation.
Sensorial research is best used on launched products as it will give you data on how users are using and feeling about the product.
It tells you if the product is working the right way with users.

Each type of research answers different questions appropriate at different stages of product development.

The Portfolio of Insights is the cornerstone between user-centricity and business value.
Unfortunately, here’s the typical breakdown we see today:

Most companies focus most of their research efforts on usability studies.

Now, this wouldn’t be a bad thing if you are a product or design team trying to take a product to launch as Summative research can provide immediate answers about a product’s usability.
However, this construct alone does not give businesses the full picture they need to evolve their product or find new opportunities to innovate on.
Satyam summarizes this nicely in his original article:
I would recommend the design leaders to invest proportionally in all three parts of the portfolio, this way they would never get caught in the trap of just solving for today’s need, and then get undercut by a competitor who was able to identify an unmet customer need and build a whole business of it.

In conclusion

1. You need a complete portfolio of insights to be truly user-centric
2. A portfolio of insights contains 3 different types of research: Formative, Summative, Sensorial
3. Formative research is good for identifying what to build.
4. Summative research (usability testing) is good for understanding how usable your pre-launched design is.
5. Sensorial research is good for understanding how your live product is doing.
6. Do this, and you will have bridged the gap between user-centricity and business value.

About the Author and Firm:

Jamie Yoo is a UX Consultant at UXReactor.
UXReactor is a full-service strategic eXperience consulting and design firm that helps teams and organizations of all maturity optimize their user-centered focus to deliver useful, usable and desirable products.
UXReactor’s partnerships are grounded in a deep understanding of our partner’s goals and intent, allowing us to work and focus on driving meaningful outcomes.

The PragmaticUX framework is our unique collection of proprietary design and research methods which allow us to quickly identify, prioritize, and solve complex problems.

UXReactor Scroll To Top