Usability Research for Retention. How to Move Beyond Your Product’s First Impressions.

By Helen King ▪ October 14, 2021

Usability research to understand the experience people first have with your digital product provides great insight into users’ initial friction points and perceptions. It can help identify immediate usability issues, information gaps and value proposition disconnects that act as barriers to building trust and getting your customers started off on the right foot. 

The trouble is, many of today’s digital products and services are designed for user engagement beyond the initial moment of conversion or onboarding. As a result, these products need to consider a much bigger time frame of the user’s experience. Your product is likely one of them.

Sticking strictly to one-time, first-impression usability testing may be why you haven’t discovered specific issues that cause users to abandon your product over time.

It’s time for a different approach to understanding ongoing product use. One that gives you a holistic picture of how your users use your product long-term. The result will be a more comprehensive view of the user experience, allowing you to make design decisions that build meaningful engagement with your customers over the long haul. 

What One-and-Done Usability Testing Doesn’t Tell You About Your Product

First-impression usability testing is undoubtedly essential, but for products designed to support users beyond one-time interactions, this approach to usability research doesn’t tell you everything.

Specifically, first-impression usability testing misses the time frame and context complexities inherent in common digital products:

• Subscription services. Users of subscription services need the ability to manage their subscription details, access digital offerings, or make adjustments to their account over the course of their subscription – returning over time for different reasons and with different goals in mind.
• Tracking apps. Tracking apps need to support use at different times of day and in different contexts, which can vary widely. For example, a product like the Nara Baby Tracker app helps parents log and track feedings, naps, diapers, and pumping, which can happen at any time of day and in time-pressed circumstances.
• Dashboards. Dashboards help users access information in easily digestible ways. These digital tools need to adapt to data and users’ needs that shift over time, helping them understand both current snapshots and long-term patterns.
• Shared connected products. Smart devices, like household thermostats, speakers or security systems, need to support multiple users whose interactions with the product may differ based on frequency of use, goal, or age.

These product time frames and unique contexts influence how your customers use and value your product over time. As a result, they have critical implications on how they come to love — or not love — your product. Not factoring longer-term time frames into your research approach can leave significant insights out of the picture.

3 Key Factors To Shape Your Usability Research Plan for Long-Term Product Use

Just like your product is one-of-a-kind, your approach for assessing long-term usability will be, too. With that in mind, consider these specific details about your product before crafting your approach.

1. Product modes of use. If your customers have different ways they might use your product based on their context, you’ll want to account for these various modes of use in your usability approach. Do you have one core scenario (like a user logging in to pay a bill when prompted)? Or multiple scenarios to understand (like a user re-ordering a prescription, as well as making a telehealth appointment)?

2. User types. The differences in your users’ motivations and behaviors inform who you need to recruit for your research activities, whether it’s distinct age cohorts or user groups with shared goals. You can scale the number of participants in your sample based on both the percentage of users who fall in different groups, and who is most important for you to learn from.

3. User availability and time constraints. You won’t gain any insights if you don’t have willing participants. Therefore, your level of access to users and a realistic understanding of their time constraints should influence your research method. The key is finding a balance between your ideal data and the “lift” you are asking of your research participants. Then, right-size the research activity accordingly.

Aside from your product, be mindful of your process and timing for sharing insights and brainstorming implications with your product team along the way. Based on what you learn from users, you’ll adjust your research plan to explore different questions and needs your product team is exploring. You’ll discover the areas where you want to dig deeper and when to zero in on a certain feature.

Determining the Right Methods to Capture Long-Term Product Use

With your specific factors in mind, you can determine how you’ll gather data on long-term use of your product. Cast a wide net of methods you consider and get creative about how they can be used in different ways.

Scenario 1. Research with one set of users over time. One approach could be exploring research methods that focus on gathering data from a single set of existing users across a dedicated time frame. Methods could include a longitudinal diary study or lighter-lift usability “snapshot” surveys sent at different intervals in their health journey. Following one set of users you’ll be able to observe how their specific needs and product usage change over time.

Scenario 2. Research with a time-based cross-section of users. Alternatively, you could segment your users by the different stages of diagnoses and run shorter research activities with users from each group. Examples of methods include a product usage and feedback survey of users at all stages, or more focused, moderated “show and tell” usability interviews with a representative sample at each stage. Conducting research with time-based cross sections of users will help paint a picture of the broader user experience across stages.

There will always be trade-offs to one data-gathering approach versus another. Brainstorming options will help you determine the best approach with your product considerations and project constraints in mind.

How to Know Your Product’s Long-Term Usability Research Approach Works

There’s no one-size-fits-all long-term usability research approach. Of course, your table stakes for actionable insights include involving real users in real contexts. Figuring out just the right approach will take work to hone in on what is best for your product.

• First and foremost, the best long-term usability research approach is one that’s rolling and iterative. One that integrates into your design process as a continuous source of insight into user behavior and motivations. Moreover, it can evolve along the way as new understanding develops.
• You’ll also know you’re on the right path when you find an approach that’s efficient to implement. If your team’s bandwidth is hindering your ability to conduct and use the research, you’ll know it’s time to recalibrate.
• On the other hand, if your selected method is not getting much uptake from users, you can go back and modify your approach and/or look at other potential ways to incentivize users to participate. And don’t get discouraged. It’s all great learning for getting you closer to a feasible and effective research approach for product insights.

As you adopt your new approach to long-term usability research, you may encounter setbacks. What matters is you keep going, keep learning, and keep discovering. Making the necessary tweaks now will help you grow a repeatable strategy for gathering insightful user data over time.

Make Decisions Grounded in Customer Insights & Design Better Products

Everyday Industries is a UX strategy and digital product design firm. See how our UX research services can give you an in-depth understanding of how your product can best meet customers’ needs and expectations.