Pendo on Pendo: How Pendo’s product team uses machine learning to find signal in the NPS noise

Net Promoter Score (NPS) is a key metric to help organizations understand how our customers perceive us, and is also a critical input in decisions around where to prioritize resources in order to make the greatest impact for customers. At Pendo, nearly every department—from product, to customer success (CS), to marketing, and up to our executive team—keeps a constant pulse on our NPS data using our product’s integration with Slack. We also use NPS to help our board and investors understand how our customer sentiment is tracking over time. 

This data—both the quantitative NPS scores and qualitative feedback—empowers our teams in a variety of ways. Most notably, teams leverage NPS data to:

• • Understand how customers feel about the product and Pendo itself
  • Understand the impact of recent releases on overall customer sentiment
  • Provide qualitative voice of the customer in product planning to determine which areas need investment
  • Identify customers that may be at risk or need additional hands-on support 
  • Identify champions and successful users across accounts (and celebrate them!)

Here’s a look at how our own practices have evolved as we’ve improved Pendo’s NPS capabilities.

The old way: Desperately searching for the signal in the noise

An important part of our routine involves helping all these stakeholders across the business understand the reasoning behind the NPS numbers. Without context, NPS scores can feel empty—you need to understand why a score has gone up or down. That valuable “why” lies in your qualitative data, more specifically the textual responses that come with the scores. These textual responses are a gold mine of insights into what your customers care about, what they love, and what they are struggling with. 

We recognized this gold mine of data here at Pendo, and we couldn’t let it go to waste. So, for years, our product operations team has run a manual process to comb through each individual textual NPS response to find the signal in the noise. Every quarter, our product ops team would do the following:

1. 1. Pull all NPS scores and verbatims from Pendo into a sheet (along with user role, company, and other valuable visitor information) to help slice customers into segments.
   2. Manually categorize each textual response one-by-one in the sheet. 
   3. Request the data science team to migrate data from the sheet to Looker, to help visualize trends or themes and insights by persona, customer type, etc. 
   4. Create a quarterly and yearly report of all the data to share with cross-functional stakeholders and leadership.

If you’re like us, your product ops team already has a lot to do. While these insights were a crucial piece of the puzzle in understanding customer feedback, the manual, time-consuming process was taking up valuable—not to mention expensive—time and resources.

The new way: A machine learning sidekick in Pendo

Just because qualitative NPS data is crucial doesn’t mean it has to be time consuming. This was the thinking behind Simon, Pendo’s new machine learning capabilities, which now does the manual work for you. This means your team can spend less time digging for insights, and more time acting on them. 

Simon combs through your textual NPS responses and automatically pulls out key themes among your promoters and detractors. These insights are surfaced to you in Pendo from two new widgets on your NPS Overview page. From there, you can drill into each theme to see specific responses, and take action right away by creating a new segment made up of users with responses about the same theme, for example a certain feature in your product. 

With Simon, our own product ops team has unlocked new levels of efficiency and productivity. They conduct analysis with NPS data right away, and use the segments they create from themes to dig into product usage behavior: Do detractors who reference “UI” in their response visit a common area in the product? Are promoters who reference “Guides” power users of a particular feature?

This also makes it easier to share the right insights to stakeholders across the business. Product managers can easily dive into themes that are relevant to their areas of ownership. Product leadership can see emerging or spikes critical trends, and track how those themes are trending over time. CS teams can even pull all responses categorized under themes relating to “support” or “onboarding” to help build a list of accounts in need of intervention. 

Simon is your secret weapon for fueling efficiency, productivity, and increased alignment across departments. By having the most common NPS themes at your fingertips, teams can spend more time hyper-focused on the most impactful takeaways from the data. Many organizations have a love-hate relationship with NPS, and a lot of it is because of the time it takes to get to the most important takeaways to help your teams—and business—improve. If NPS data has felt out of reach for your organization because of time to value, Simon could be the team member you’ve been looking for. 

NPS Insights is currently in open beta. To be the first to get updates on this new product, please join the interest list.