Thinking Outside the Task: Leveraging Evaluative Research for Deeper User Insights

By Helen King ▪ July 7, 2023

When conducting user research, it’s easy to pick a lane based on where you are in your product’s development cycle: you either do evaluative research to improve the current product or discovery research to look for new product opportunities. 

Your priority then points you to possible methodologies for either type of research, like a usability study for evaluative research or in-depth interviewing for discovery research. 

While choosing a lane could be most effective for your goals and timing, an “either/or” mentality around evaluative and discovery research could also be causing you to miss out on valuable user insights to drive your product strategy. 

In other words, the two aren’t mutually exclusive: you either learn about your product through evaluative research or learn about your users to uncover new opportunities through discovery research. 

User research is a critical endeavor for product teams that takes your and your customers’ time – resources no one wants to waste. To make the most out of your efforts, let’s look at some productive ways to integrate evaluative and discovery research.

Why an Either/Or Mentality About User Research Could be Holding You Back

Evaluative and discovery research have complementary roles within product design that are often treated as separate and distinct activities.

Evaluative research is a relatively quick and directional form of user research, asking users to interact with your product to uncover issues and needs to inform immediate design implications for the product. Discovery research is generally a more time-intensive and exploratory form of research, diving deeply into topics and processes with your users to uncover insights to inform strategic product opportunities.

While the structure and goals seem at odds, these two forms of research can borrow from each other to create opportunities to learn more about your customers and your product.

Using Discovery Methods in Evaluative Research 

It should come as no surprise that valuable learnings about your users can pop up in evaluative research endeavors like usability test sessions. Outside of whether or not they can complete a particular task, participants often speak to their more significant motivations or needs as they interact with your product.

You can find these gems of customer learning anywhere from unprompted descriptions of how they like to shop the product category, to responses to guided prompts around how they understand your product positioning. Hearing and observing these moments causes you to stop and think, “That’s different from what we thought…” or “We’ve heard that from other customers!”

The key is often looking beyond a binary “pass / fail” approach to designing and analyzing your usability study to leave room for new learnings about your users.

When & How to Blend Discovery into Your Evaluative Research

While not a replacement for discovery research, using evaluative research to learn about your users can be helpful when more exploratory research is either not possible or not the appropriate time for your team.

In situations where you may already have a full product pipeline, different planning priorities, or time and budget constraints, the critical thing to remember is that you don’t need to put gathering learnings about your users on hold.

Here are three ways to gather deeper user insight from your evaluative research:

1. Design Learning Moments into Your Protocol

Whether conducting moderated or unmoderated testing, you can build in tasks, and questions focused on learning about your users by using methods reflective of discovery-oriented interviewing. 

Moderated testing is flexible to include questions and follow-ups from the moderator about the participant’s preferences and behavior. Keep your questions and follow-ups focused to avoid turning your usability test into a full-blown interview. Here are a couple of techniques to try in a moderated test:

• Include one question in the introduction about the participant’s general habits, product feedback, etc. These introductory questions are great to keep consistent over multiple tests to build up an understanding across users.
• As participants interact with the product, probe with behavioral follow-up questions like, “Is that typical for you?”, “Can you remember the last time you did X?”

Unmoderated testing is less fluid since participants have pre-set tasks and questions they respond to independently. Writing clear and focused tasks are essential to ensure you meet the study goals with an unmoderated test. When building your protocol, try including one more exploratory prompt, like: 

• A verbal task or written follow-up that has participants reflect on the most significant benefits of the product or feature. This kind of prompt often reveals what they value personally, or other needs the product may or may not fulfill.

2. Look Beyond the Tasks During the Analysis

While assessing the clarity and usability of the design will be your main focus for an evaluative test, analyzing the rich data outside of participants’ task success rate will be your key to unlocking the potential of your evaluative study. 

When reviewing the test sessions, keep an eye out for these unexpected learning “gems” about your users:

• Expectations & mental models. How do they describe the product? What do they compare it to? These observations can help you understand their mental models and preconceived expectations about how the product might work or how they want it to work.
• Benefits & use cases. What are they most excited about the product? How do they describe they would use it? These learnings can help you better understand their values, which may or may not align with the product or feature you are testing.
• Unmet needs. Do their suggestions have a common theme? Do they have a need that a different solution could better serve? If they are non-plussed by the particular feature you’re testing, you may have the opportunity to learn about a more significant problem they need to solve.

3. Maintain a Repository of Research Data and Learnings

To prevent your learnings and observations from becoming forgotten anecdotes, document them! While an observation from one usability session may be thought-provoking, continuing to gather data in other studies to see where themes emerge or learnings become more nuanced will provide more relevant and meaningful insight for your team.

Input your data and learnings into a shared spreadsheet or a repository tool like Dovetail or Aurelius. Doing so allows you to use tags to code the entries for sorting and filtering. Then, as you add to your repository over time, you can revisit it to conduct analysis and determine implications. Discuss any exciting findings with your team and capture your hypotheses and unanswered questions to explore in future user research.

Finding Blended Approaches to Expand Your Research Toolkit 

While purely evaluative or discovery research will always have their place, considering ways to blend these approaches can help open up your options for informing your team’s decision-making and driving your product forward.

By being intentional about framing your usability protocol, looking carefully for user learnings in analysis, and creating systems for maintaining the data and insights, your team’s way of thinking about your customers can grow and evolve through evaluative research.

Make Decisions Grounded in Customer Insights & Design Better Products

Everyday Industries is a UX strategy and digital product design firm. See how our UX research services can give you an in-depth understanding of your customer’s needs and motivations.