In my email inbox is a message asking me to take a brief survey about my recent experiences with an airline
I have a frequent flyer account, and the airline has my contact info, so it wasn’t a surprise to get the message. But the request seemed like a wasted effort on the part of the airline; taking it would likely be a waste of my time. I hadn’t flown that airline in nearly five years. Any feedback I’d share certainly wasn’t going to be recent.
Instead of immediately hitting delete, I started wondering about how the effort to gain insights from me, the user, could have been improved. (An occupational hazard, I suppose, as I’m always focused on usage analytics.) Shouldn’t the airline have known that I haven’t flown them since 2017 and that I don’t have recent experiences to comment on? Couldn’t they segment their list more thoughtfully, based on the number of flights actually taken in the last 12 months or something more meaningful related to the insights they are seeking?
Software suppliers face similar situations when they try to gather information about how their products are being used. Clarity into product usage is essential: it helps suppliers optimize the quality of their products, while satisfying (and retaining) users.
Qualitative approaches, like surveys and interviews, have long been used to gather insights. These continue to be valuable, offering an opportunity to gather subjective information about a user’s thoughts. But the reality is that qualitative approaches are usually time consuming (for the company and for the customer), costly, and may have low participation rates. Perhaps even worse, there may be responses from well-intentioned but unqualified respondents.
Today product managers (PMs) have new, quantitative methods that they can combine with older, qualitative methods to maximize and scale their efforts. By holistically pairing software usage data and analytics, PMs can rely on objective data to better inform their more manual processes, strengthening the overall initiatives.
Harness Insights to Drive Innovation
Software usage analytics—the process of tracking and analyzing how users engage with software—provides software suppliers with tracking (by collecting raw data about user actions and computing environments) and analysis (through visualization dashboards that show data and trends, user behavior, and differentiators across user segments). These findings yield actionable insights, which can facilitate context-relevant engagement with the software.
Teams across an enterprise—compliance, customer success, marketing, product management, sales, software engineering, and senior management teams—can all turn to usage data to support varied initiatives. Tangible use cases include beta testing, decisions about features (prioritization and/or deprecation), pricing decisions, roadmap development, software version decisions, UI/UX design, and unlicensed/pirated usage. Usage data is also key to optimizing a software company’s monetization models (perpetual, subscription, usage-based, value- or outcome-based) and deployment models (embedded, on-premises, SaaS).
Complement, Don’t Substitute
Qualitative and quantitative feedback aren’t substitutes. Instead, by using them together, they can deliver more robust insights into customers’ preferences.
· Qualitative methods are usually manual, subjective processes that require significant time commitments. These typically include support calls (often initiated by a customer with a complaint or with a question about how to use a product feature), customer interviews, feedback from sales teams, and email surveys.
· Quantitative methods, including basic telemetry from the product and advanced product usage analytics, are automated, objective, and scalable processes that can be completed quickly on an ongoing basis. Advanced product usage analytics can help overcome challenges often associated with manual collection methods or with interpreting raw data.
As found in the Revenera Monetization Monitor: Software Usage Analytics 2021 report, product managers identify customer interviews as the most effective qualitative approach to collecting software usage data (reported by 75%). PMs rank advanced product usage analytics as the most effective quantitative approach (reported by 59%). Among all survey respondents, 62% identified customer interviews as being effective, but efficacy of the interviews jumped among those who use advanced usage analytics to 80%, highlighting the value of combining the approaches. Adding this quantitative data helps suppliers make the qualitative interactions more targeted and valuable based on factors including logins, features that are or aren’t used, or upgrade status of users of a freemium or try-before-you-buy version.
Streamline & Strengthen
So, what are the practical ways to combine qualitative and quantitative approaches to collecting software usage data? Consider these common use cases:
· Surveys: Software usage analytics can help identify users who demonstrate the type of engagement the company wants to analyze further. This may mean identifying users who have the greatest engagement with a particular feature, who last logged on within a particular timeframe, or who recently upgraded product versions. This facilitates better designed, more targeted, shorter surveys. By getting the right people to take the right survey, response rates are likely to go up and the results are likely to be more valuable.
· Customer interviews: Even more time-intensive than surveys, customer interviews benefit from usage analytics in a similar manner. When an interviewer is prepared with clear insights into customer interviewee’s product and/or feature adoption, the interview is more efficient and engaging for everyone involved. Rather than investing time into setting the stage, this approach allows the conversation to delve into more meaningful questions—quickly—about why the customer chose to use a feature (or not), unexpected challenges the user faced, etc. This approach gives the interviewee a greater opportunity to truly be heard.
· Sales feedback: Product usage analytics tools can analyze and visualize data, reporting on it efficiently, without manual processes or engineering work. When this is easily accessible throughout the organization (without having to be interpreted by data scientists), sales teams easily gain greater clarity into how their accounts are using the products. This facilitates more specific, tailored conversations that can strengthen relationships—and retain accounts.
Qualitative data benefits when filtered through measurable quantitative data. Quantitative data gains context when augmented by qualitative feedback. Combining these methods combines their value and delivers the deepest possible insights into users’ experiences with an application.
About the Author
Michael Goff is principal, product marketing at Revenera, focused on software monetization and usage analytics solutions. Revenera’s solutions help software and IoT companies build and deliver secure products while protecting their IP. Make a great first impression with your software – with the gold standard for Windows and multi-platform installations.
Featured image:©kenishirotie
