Do you know how your product is doing? Let data help you improve your product’s UX.

This article is part of the Founder’s Guide to UX Research, where we break down methods that will help you and your product team understand your users, improve your product eXperience, and exceed your business goals. See our previous article on Usability Testing and the eXperience Audit for more.

Right now, companies are buzzing about data as the sexy tool to solve their products’ problems. It’s easy to find and capture data, but also all too easy to fall into a rabbit hole where you have too much data and don’t know what to do with it.

Big data, machine learning, AI, it’s all the rage!

Hold your horses. There are easier, more accessible ways to make use of data, and we’re going to explore how to use that data to improve your product’s web presence.

Today’s topic: product/website optimization.

What is product/website optimization?

Product/website optimization is the practice of tracking metrics from the usage of your product — such as behavioural data — and implementing improvements to your product based on insights gathered from those metrics.

You will need to instrument your product/website to collect metrics.

Instrumentation, while vastly popular for DevOps, is a very powerful UX tool as well. It is a fast and easy way to collect data from a large sample size of users. Our recommendation is to pair quantitative data (via tools like Google Analytics) with qualitative data (via tools like Inspectlet) for the best outcomes.

Alternatively, you can also use tools that do both, like Hotjar or Fullstory.

What outcomes can you expect?

Because of its ability to collect large amounts of data, instrumentation provides organizations many avenues to analyze and optimize their products. When undergoing product optimization, it’s important to focus on specific goals to help the business; these include evaluating user engagement or tracking metrics such as conversions, acquisition, and retention.

Instrumentation tools in the market provide three methods of collecting data:

  1. Web analytics
  2. Session analysis
  3. Intercept surveys

Using any number of these methods or a combination of methods will help you understand how users are using your product.

Benefits and Constraints

Instrumentation is easy to set up but may require an expert to analyze and synthesize the data. Because of the low initial cost and longevity of the tool, instrumenting your product has great potential for ROI.

Web Analytics

Web Analytics (via products like Google Analytics, Adobe Analytics, and Pendo) is an effective method for gathering quantitative data on users, ranging from their demographic information and behavior to the devices they use to keywords, click metrics, and time on page.

In short: Who is coming to my site and what are they doing?

Benefits:

  1. Objective user data — instrumentation collects concrete data on things like new versus returning users, where they are coming from, and what keywords they use to reach your site.
  2. Usage data — data around the behaviour of users on your site which reveals where they drop off, where they click on the page, and how long they spend on different areas of your site.

Constraints:

  1. Only tells who and what, but not why — although Web Analytics provides a detailed picture of who is using your site and what they are doing, the quantitative data doesn’t provide insight into why users behave the way they do

Session Analysis

Session analysis is an extremely powerful addition to Web Analytics. Web Analytics tells you who is doing what on your site, and Session Analysis tells you how people use your site through session recordings via tools such as Inspectlet and Fullstory. This, in turn, allows you to quickly identify areas of friction to improve on.

Benefits:

  1. First-hand user behaviour observation — you can see what users are actually doing on your site which becomes extremely valuable when trying to triangulate issues and areas for improvement.
  2. Identifying specific friction points — since you can follow a user’s flow on your site, it will become apparent where the drop-off points and areas of confusion are. When you triangulate the drop-off point with other data, such as time on page and click analysis, you should be able to pinpoint what caused them to drop off.

Constraints:

  1. How, but not why — although session analysis can show pain points and drop-offs, it lacks direct feedback from the user. Because of this, insight to why users behaved the way they did can only be speculated via deductive reasoning.

Intercept Surveys

Intercept surveys are a well-known method of collecting user feedback and impressions of your site to gauge overall satisfaction and sentiment. Methods such as NPS, CSI, and Leikert scale questions yield recognizable and reliable metrics for comparison. Integration into your site makes this a low cost option to implement.

Benefits:

  1. Easy distribution — because of the standardization of surveys, they are easy to distribute to large sample sizes. Larger sample sizes yield more confidence in the data.
  2. Standardization — the structure of surveys makes the data collected easy to compare. Thus, it is effective for baseline and follow-up purposes for a comprehensive understanding of user impressions in an experience.
  3. Contextual — intercept surveys are designed to gather user feedback while they are using your site or product. Therefore, the responses are given in real-time, giving you a more accurate and contextual snapshot.

Constraints:

  1. Confounding questions — although NPS and CSI are standard for surveys, custom questions have the potential to introduce bias in the data. Try to use neutral terms when creating survey questions to avoid confounds.
  2. Static feedback — there are many influences on user impressions and perceptions throughout an experience. Surveys can only capture one snapshot of the user’s feedback during their journey, which has the potential to give an inaccurate representation of their sentiment.