What exactly does it mean to be “data-driven” in today’s product management world? Our colleagues at Pendo sought to answer this question by reaching out to product leaders at a variety of organizations. They asked them to describe their relationship with data, which metrics they considered most important, and the areas of their jobs that are most data-dependent.
A number of themes emerged from their answers, including the areas of the product development and customer life cycles where data plays a particularly important role. Below, we’ll go through each of these and share some of the thoughts and opinions voiced by data-driven PMs.
Discovery
In the discovery process, product managers can leverage data to identify any gaps in the product to inform what changes or updates to pursue. On the flip side, it’s also useful to look at your most used features to see if there are any ways to improve that experience for your users. If you know a particular feature is associated with key customer outcomes, you should always be looking for new ways to make it better.
Roadmapping
Product usage data is valuable for directly informing the product roadmap (i.e. which features are customers using the most, and therefore require more attention?), but Viraj Phanse, product manager at Amazon Web Services, brought up another scenario. He said that sometimes, a product manager will have an idea for the roadmap but no data available to back it up. In this case, PMs still need to make the call one way or the other and take smart risks. Adopting a customer-centric approach can help as well. Focus on your customers’ problems throughout the roadmap journey, and constantly ask yourself what customer problem you’re solving.
Feature launches
For new feature launches, Greg Bayer, SVP of product at Nielsen, says his team always tracks adoption and usage first and foremost, including: how many customers are using the new feature, and is it leading to more overall engagement in the product? He emphasized the importance of linking success criteria with your customers because, at the end of the day, you want them to be successful because they used your product.
You should also analyze a new feature’s success to understand any usage patterns and what customers might still need from the product. Most importantly, make sure you have tracking in place before going into a new feature launch. Travis Turney of Rapid7 believes the metric itself doesn’t matter as much as having some sort of measurement program so you can learn over time what “good” looks like.
Onboarding
When building an onboarding strategy, product data can help you determine which features to include in your onboarding flow (aka those that are most likely to lead to success in the product) as well as help you measure
Travis Turney, senior data strategist at Rapid7, also called out onboarding as an important time to learn about your customers through the data you collect. Since customers are more likely to feel comfortable providing information about themselves when they first start using your product, Travis says it is your best opportunity to capture data (e.g. their role in the company) that can then allow you to provide a more personalized product experience going forward.
Experimentation and testing
The experimentation process hinges on data as you test, analyze, and learn what works and what doesn’t. Manosai Eerabathini, a product manager at Google, sees data as crucial to experimentation, especially for products that are well established and where testing and implementing small tweaks to the user experience can make a large impact.
Andy Browning, UX writer at Mimecast, explained this from the perspective of the UX team: “We found that when we do user testing, we have our own ideas and instincts going in, but then we actually see the users engaging with the product and get the data, and we might be completely wrong. And you can’t really argue with it—you know where you need to make improvements.”
Retention
Retention is always important, but it has become even more critical for businesses to stay ahead of retention by understanding how customers are using the application. Product operations specialist Sam Benson explained how at Firefly Learning, customer success managers (CSMs) need to have an idea of their customers’ user journeys in order to have productive conversations, especially when it comes time for renewals. This way, CSMs can offer the right recommendations tailored to each customer account’s usage.
And in fact, you can learn the most from customers who don’t stick around. In addition to leveraging data to proactively address retention, Bella Renney, head of product at Tray.io, believes the most valuable information comes from talking to the customers who don’t renew their subscription:
“I’m much more interested in the customers who came to the tool, used it, and then decided that it wasn’t valuable. Why did they churn? I think that’s the most important piece of data you can get as a product manager to make prioritization decisions.”
Want to read more about how data impacts the product management role? Check out Pendo’s latest ebook, “How to Become a Data-Driven Product Manager.”
