How to Use Qualitative Research to Tell You Everything Quantitative Data Can’t

By Allison Sall
October 14, 2021

Quantitative data gets a lot of hype, for good reason. Without it, you don’t get nuts-and-bolts metrics like your conversion rate, daily active users, or bounce rate. But there’s another approach to collecting insights that tell you precisely what you need to make a more engaging product.

We’re talking about qualitative data, or the why and how behind the numbers. It tells you why your users converted at that particular moment in time. It explains how your users feel when they use your product day after day, and even why they decide to leave.

Getting those insights isn’t as elusive or daunting as you may think. It just takes knowing what to look for and asking the right questions along the way. In doing so, you can unlock a deeper understanding of your users and their actions, leading to a new level of possibility for your product.

Uncovering these a-ha! moments will tell you a great deal more than numbers alone. In building your best product, you need to get the full picture.

Why Qualitative Usability Testing for Your Product Rocks

Quantitative data is like the melody of a song. Without it, your product is just noise. But, admit it: Those data points don’t tell you everything you need to know. And despite your best efforts, developing your team’s collective clairvoyance has been on the back burner.

Sure, you can assume you know what makes your users interact with your product the way they do. You can assume the value they’re going to get out of your latest feature. But we all know assuming takes us down a dead-end road.

Collecting qualitative insights leads us away from assumptions and widens our previously narrow perspectives. What’s more, firsthand user feedback allows you to create and perfect your product to standards that exceed what your users expect. It’s not clairvoyance, but hey, it’s pretty darn close.

Gathering these insights means you’re observing real, unbiased people as they use your product, asking them questions along the way.

It’s beneficial because you can ask about the specific situations that make your user want to keep using your product — or want to stop. Can quantitative datas answer that? At any number of touchpoints, you have the opportunity to clear up areas of confusion. This ultimately leads to a definitive answer of why your user is dropping off.

These kinds of first-person accounts are rich — and invaluable. And they’re the underutilized keys to unlocking why your users keep leaving before checking out, why they’re stopping before completing their onboarding session, or what UX discrepancies are causing them frustration.

In listening to real people (not our assumptions, not the stats alone), you can make improvements that truly resonate.

Simply put: If quantitative data is the melody, qualitative insights are the rhythm. In concert, they can make your product rock.

Qualitative Testing Translates Users’ Head-Scratching Actions

Observing someone using your product on their own is a weird feeling — especially without you stepping in to help (as much as it may pain you to refrain). You cringe if they can’t find a very obvious button. You raise your eyebrows in surprise when they find a very creative way to navigate to your product’s information (that, note to self, you definitely need to prioritize).

But in watching someone explore your product, you’ll quickly realize that it’s not just what they do. It’s also what they say. Every hmmm, oh!, and uhhh speaks volumes.

Hmmm… can be a cue to ask what they might expect.

Oh! is a clue to ask what was surprising and why.

Uhhh… is a dead giveaway that a step isn’t designed as intuitively as you thought.

These spoken (or mumbled) moments can prompt you to dig deeper and find out more.

  • What points of the flow are cumbersome?
  • What questions do they have in different scenarios?
  • What do they need to know to feel comfortable hitting that ‘Purchase’ button?
  • Why don’t they feel connected to the product?

Qualitative user testing is your ticket to making sense of those confounding moments — for both of you.

Moderated or Unmoderated: Which Qualitative Research Method is Right for You?

You may think that getting into the why’s and how’s of your users’ actions is going to be…a lot. While it’s true there’s no set-it-and-forget-it way to conduct qualitative research, there is good news. That is, there is more than one way to get the information you need, based on the amount of resources at your disposal and your objectives.

For those who have time and personnel on their side, moderated testing should be the first consideration. For those looking to get some quick answers in a less-hands-on way, unmoderated testing will be the way to go. Here’s the difference.

Moderated Usability Testing — a.k.a. Someone From Your Team Leads the Test

When you go the moderated testing route, you’ll sit down with a user for at least 45 minutes (usually over Zoom these days). You’ll ask questions about their background, what other products they use, and other specific behaviors it might be helpful for you to know. You can then use that information to contextualize their motivation for using your product.

So for example, if your user is on the hunt for the perfect sofa, you’ll have them shop for a sofa using your product. You’ll ask follow-up questions as they swipe and scroll to learn what they’re thinking of your product and its design. You’ll ask specific questions in the moment to clarify anything they’ve said (remember the hmms, oohs and ahhs).

Unmoderated Usability Testing — a.k.a. The User’s in Control, No Questions Asked

There are a few good reasons to use unmoderated testing: if the test is simple, you lack the appropriate person to conduct the test, or want to hone in on a specific issue. With this type of testing, you come up with prompts and questions that guide the user down a specific path. Users are recorded as they explore your product, answering your prompts and questions along the way.

As long as your test is designed with intention and detail, you can get great data and feedback. But since no one is present to moderate the test, asking questions in context is out of the question. For example, you can’t ask follow-up questions that could clarify a user’s answer. Nor can you personalize the test to match a user’s specific context or motivation. But there are about a zillion tools to conduct unmoderated testing these days, so getting started is pretty quick and easy.

What Can You Do With the Data Once the Results Are In?

Without qualitative research you can’t zero in on why you are seeing a perpetual drop off at a specific point in your conversion funnel,. You just have to sit there wondering, or throw a few educated guesses against the wall to see what sticks. That’s not really productive. And it definitely doesn’t help you solve a design issue.

Qualitative research allows you to understand what your user understands about your product (or doesn’t). You can’t know to ask yourself: What do we need to adjust after the user does this? What information can we add (or remove) to make them confident in hitting the “Purchase Now” button?

Numbers only tell you so much. But discovering the why and how can give you the rest of the story so your team can forge ahead with informed, user insight-driven decisions.

Your user’s experience with your product is more than just a moment on a page. It’s how they explore it, from screen to screen. It’s how it makes them feel, how it makes their life better, or easier. The inevitable little points of friction that bubble up, the critical information they are looking for — all these moments build up questions in your user’s mind. Is this product right for me? These little things add up. And unaddressed by qualitative research, they can cause real harm.

Qualitative research can examine those seemingly insignificant moments in a way that quantitative data can’t — and give you a path forward. Just short of clairvoyance, it’s the only way to build a product you’re confident is satisfying your customer’s wants and needs. And that is definitely worth the hype.