In my experience leading User Research teams at UXReactor, UserZoom (now UserTesting), and PayPal, I’ve seen firsthand the importance of demonstrating the impact of user research on business outcomes.
It’s easy to get lost in the day-to-day and lose track of the team’s achievements and broader engagement. That’s why I make it a point to reflect and jot down key accomplishments and their impact at the end of each month, quarter, or year.
Let me introduce you to the ‘5-V Framework’. It’s designed to help you collect metrics, document them effectively, and narrate compelling success stories, thus demonstrating impactful outcomes.
In the business world, when the value and impact of user research are recognized, it fosters opportunities for increasing investments.
Velocity Metrics: Keeping Pace with Organizational Demand
In today’s fast-paced business environment, the ability to execute research at a high velocity is essential.
How Do We Measure Velocity?
- Number of studies, participants, and workshops facilitated: Per month or quarter; Quarter over quarter.
- Quantity of research requests that could not be supported: This shows unmet demand and is a great way to discuss additional investments in user research.
- Total count of researchers on the team (end of quarter).
At UXReactor, we certainly deliver Rapid Usability Testing for our customers. This is a great way to demonstrate value under time pressure. Over time, it helps our customers integrate research as a program. But it all starts with one agile small project in most cases.
Validity: The Cornerstone of Reliable Research
Research should not be a box-ticking exercise; its actual value lies in the validity of its methods and insights. This requires a meticulous approach to selecting methodologies, ensuring data integrity, and rigorously validating findings.
How Do We Measure Validity?
- Sample size and method selection appropriateness.
- Frequency and depth of peer reviews.
- Alignment of research with business objectives.
- The extent of research democratization (Measured through training and project reviews).
- Number of research templates created, updated, or reviewed.
Variety: A Range of Methods and Engagement Strategies
The richness of user research lies in its diverse methodologies and approaches. From qualitative deep dives to quantitative analyses, variety in research broadens the scope of insights and caters to different stakeholder needs.
How Do We Measure Variety?
- Diversity in types of studies and participant segments. Examples:
- Moderated vs. Unmoderated
- In-person vs. Remote vs. Field Visits
- Consumers vs. B2B or Patients vs. Providers
- Percentage breakdown of projects across Discovery, Design Validation, and Benchmarking stages.
- Use this as a nomenclature guide: When to use which user research methods.
- Helpful Reading: The 3 types of user research your company needs to be doing.
Visibility: Research Engagements Need To Be Seen and Heard
Elevating the profile of the research department, particularly among executives and key stakeholders, is crucial for long-term success. At UXReactor and in my previous roles, prioritizing the visibility of research efforts and outcomes has always been a game changer.
Visibility involves engaging with stakeholders before, during, and after each research initiative. This simple gesture can go far in demonstrating the impact of research.
User Research is a team sport, so research execution alone cannot drive impact. You need the awareness and involvement of the larger team to take it to the finish line.
How Do We Measure Visibility?
- Number of product managers, designers, business units, and product areas supported.
- Research coverage percentage = Business Units supported vs. Total Business Units at the company. Replace Business Units with Product Areas when appropriate.
- Leadership mention frequency (of User Research or Insights from User Research).
- Level of engagement with the research repository.
- Results of stakeholder survey for perceived value.
Value: Impact made on moving the needle
In my two decades of leading research teams, a recurring question I pose is: “Where is our ‘Show me the money’ slide?”. This question isn’t just rhetorical – it’s a prompt to delve deeper into the tangible impact of our work.
A user research project does not end when the report is presented. It ends when we understand and document the impact of the research engagement.
I know it sounds easy, but in practice, it’s not.
How Do We Measure Value?
Often, it’s hard to connect the dots from a single research study with the entire company’s financial outcomes. This is where the four levels of the KPI model come in handy.
- Level 1: Company Financial Metrics:
Revenue, Bookings, Margin, EBITDA, Recurring Revenue, Cash Flow Figures. - Level 2: Acquisition, Usage Metrics:
Metrics on Customer Acquisition, Conversion, Activation, Adoption, Usage, Lifetime Value, and Churn Rate. - Level 3: Voice of the Customer (VOC) Metrics:
Net Promoter Score (NPS), Customer Satisfaction (CSAT), Social Media Analytics, App Store Ratings, Customer Support Ratings, Number of Support Tickets and Calls. - Level 4: Product or Project UX Metrics:
Task Success Rate, Error Frequency, NPS, System Usability Scale (SUS), SUPR-Q, QXScore; Count of Usability Issues Identified, Prioritized, or Accepted; Number of New Ideas Generated.
Strategies for Connecting Metrics Across Levels
Connecting Level 4 to Levels 3, 2, and 1: In bridging the different levels of metrics, it’s essential to start each research project with a clear goal or hypothesis. First, gather Level 4 metrics directly related to the specific research engagement. Then, collaborate closely with teams across Product Management, Analytics, Data Engineering, Customer Support, Finance, Customer Success, Marketing, and Sales. This collaboration is critical to accessing and interpreting Level 2 and 3 metrics and even further Level 4 insights.
Sometimes, it does not work: Acknowledging that not every user research effort will directly boost revenue is crucial. In my experience, it’s often challenging to directly link a particular research activity to a specific increase in revenue. However, a comprehensive and convincing narrative of impact can be constructed by methodically connecting a series of research initiatives to Levels 3 and 2 and, ultimately, even to Level 1.
Laddering up to get to a vetted ROI model: This process of ‘laddering up’ the impact story must be thoroughly vetted with stakeholders to ensure its credibility. Developing robust Return on Investment (ROI) models and engaging stakeholders in scrutinizing and refining these models is vital. Welcome stakeholder challenges to the assumptions in your ROI models as a positive sign of engagement. Such discussions are opportunities for collaboration and refinement, indicating a vested interest in the research outcomes.
Yes, it takes significant effort: This approach demands considerable effort and cross-functional collaboration, but the payoff is substantial. The more you practice and refine this approach in your research projects, the more proficient and impactful it becomes.
Another way to demonstrate impact is to collect regular feedback from stakeholders such as product managers, marketing executives, or executives.
The Five Vs – Velocity, Validity, Variety, Visibility & Value metrics are the pillars upon which successful user research departments are built. Embracing these can transform your research function.
As you reflect on the future of your user research organization, remember that I, alongside my team, am here to partner with you. Let’s make a meaningful impact together.
What to Do Next?
Do this to demonstrate the impact of user research:
- Determine Meaningful Measures Along With Your Team: Identify which metrics for Velocity, Validity, Variety, Visibility, and Value are significant in your context.
- Consistently Track these Metrics At Least Every 90 days: Engage your entire user research team in collecting these metrics. Use reasonable approximations when exact data isn’t available. Focus on measuring Velocity, Validity, and Variety and ensure a record of Visibility Metrics. Demonstrate Value by connecting efforts to outcomes at both project and department levels. Start small, but start now!
- Visualize Data and Craft a Quarterly User Research Impact Statement: Utilize decks, graphs, charts, newsletters, infographics, or short videos to present these 5-V metrics and success stories. An example of a quarterly user research impact statement is provided below.
- Share with Stakeholders and Celebrate Success: Regularly share metrics from the 5-V Framework with the broader organization to ensure widespread awareness and engagement and publicize wins and success stories to foster a culture of recognition and continuous improvement.
Example Quarterly User Research Impact Statement
I encourage you to craft a similar statement reflecting your team’s achievements:
“This quarter, our team conducted 30 studies, engaged with 600 participants, and used 4 different research methods. We actively supported all top 3 company initiatives, collaborated with 60% of Product Managers, identified 50 critical usability issues before development began, and generated 3 big new ideas that were adopted. The product groups we supported released Product X, receiving stellar customer feedback and meeting expected acquisition metrics. Business Unit Y is on track to achieve its quarterly goal. However, the 40% of product managers we could not support this quarter are asking for help. So we’re proposing a 20% increase in investment in the next budget cycle.”
→ We can help brainstorm & collate these metrics for your user research organization. Reach out, talk to us. ←
Other Related Article:
12 Ways to Gain Influence Within Your Organization and Webinar (YouTube)