Pendo’s product managers are in a unique position in that we get to be beta users of our own products, and figure out how to use new features and functionality before our customers do. In the end, our goal is to use this vantage point to give customers a leg up and empower them to get the most out of our product.
As a PM for part of our analytics functionality, my focus has been on Product Engagement Score (PES) and Core Events. For anyone who needs a refresher, PES is a way for PMs to quickly understand how a product is performing, and is calculated by taking the average of a product’s feature adoption, stickiness, and growth. To calculate the adoption metric, you need to know your Core Events, which are your product’s ten most impactful features.
Core Events are based on the idea that when it comes to measuring and improving what matters in your product, less is more. So, Core Events are limited to ten. This is by design; it forces you to be critical about what makes your product great and the behavior you want to optimize for. An important first step is choosing those ten events—something that can be a real challenge and spark a lot of internal debate. If you’re looking for the cliff notes, here’s what we recommend:
If you want the full story, here’s a look at how we chose our own Core Events:
The first attempt at choosing Pendo’s Core Events
To start out, the team chose our initial set of Core Events based on multiple conversations with a group of cross-functional subject matter experts—including our director of product management, director of data science, and representatives from our product operations and marketing teams. Our strategy from the beginning was to pick events (features or track events) that have high retention and high adoption.
Although we did bring retention and adoption data to our original discussions, we ended up mainly optimizing for breadth of coverage across the entire Pendo application to ensure we were measuring product success holistically. Here are the original Core Events we chose:
Missing the mark (and how we knew)
Fast forward six months later, as I started understanding and using Core Events more, I realized that we may not have quite hit the mark with our original ten choices. In examining the Time to First Use widget and pulling reports via Data Explorer, it became clear that some of our Core Events didn’t meet our original criteria of choosing events with high adoption and retention rates.
For example, reviewing the Time to First Use data showed that some of these actions took a very long time for new visitors to achieve. Digging into feature adoption data also revealed that some of our Core Events (e.g. Resource Center | Toggle Production “On”) had incredibly low adoption rates.
Logically, our next questions were, “Why not? Did we just pick the wrong ones?” and “Can something be a Core Event even if it doesn’t have both high adoption and high retention?”
With the upcoming release of PES, there was even more of an onus to get it right since our CEO had been presenting Pendo’s own Product Engagement Score at our biweekly company-wide town hall meetings. Since the adoption pillar of PES is based on Core Event usage, ensuring that our Core Events were relevant was critical to making sure we as a company were measuring the right things.
Bringing in the Squad to get it right
Once I realized that our original ten Core Events weren’t quite right, I formed a group of relevant stakeholders from the product, data science, and product operations teams to reevaluate our choices. Over the course of about a month and a half, the Pendo Core Event Squad met every other week to collaborate on this initiative.
We started by reviewing data on our existing ten Core Events, which included discussions around how we chose them originally, which we had potentially missed the mark on, and features from various product areas that might be a better fit. These conversations were informative, but we knew we needed to level-up the analysis. We asked ourselves, at the end of the day, what are we optimizing for? What does success look like for us and our customers? That’s when we decided that it all came down to onboarding.
Start with onboarding
We decided to align our Core Events with onboarding because of two well-known principles:
-
- Successful customer onboarding is the most critical component of a healthy product.
- Successful onboarding leads to higher retention, more conversions, and lower support costs.
We know users have to find value in a product quickly to keep using it, and if they never get there, nothing you do after that will matter. The final goal of any onboarding experience is achieving “aha” moments when a user internalizes the value of the product, ultimately driving greater user stickiness and growth. So, it’s important to figure out what those “aha” moments are for your customers during onboarding using a variety of sources like Pendo data, user research and interviews, and your own intuition.
Once we knew we were going to focus our Core Events around onboarding, it was time to actually choose them.
The PM vs. data science approach
We realized there were two ways we could approach coming up with our new list of Core Events: we could approach it as PMs or as data scientists. Most teams won’t need to go as deep as we did here, but I wanted to share how we went about balancing data and instinct.
The Pendo Core Event Squad split into two groups to take these two different approaches and run with them, with the end goal of creating a single list of Core Events related to onboarding. The first group used qualitative data based on conversations with (and observations of) customers, along with their own experiences using the platform. The second group used a purely objective, data-driven approach and, led by our director of data science, analyzed retention and adoption data for all of the features within Pendo.
Once both groups had their lists of features in hand, the Squad regrouped and compared lists. Perhaps unsurprisingly, there was a fair amount of overlap and our qualitative and quantitative approaches led to similar overall results. It was easy enough to agree and align on the intersections between the two lists and move these features onto the master list. Then, we discussed and debated the few features that didn’t appear on both lists, with some making it onto the final list and others not.
This is how we ended up determining our final shortlist of Pendo’s new Core Events—but the work doesn’t end there. Since product data isn’t static, we look forward to measuring the impact of our new Core Events in the months to come by examining the time to first use and retention of each of these six features.
Key takeaways: Choosing your 10 Core Events
If you’re tasked with choosing Core Events for your own product, here are three best practices to keep in mind:
-
- It’s a team effort. Like all challenging but rewarding projects, choosing your Core Events is better with friends. Take the time to get in a room with the right people, and have multiple conversations—it’ll pay off.
- Embrace change. Approach your Core Events with curiosity and be prepared to experiment. It’s smart to update and change your Core Events as you gather new information.
- Start today. Identifying Core Events shouldn’t be something big or scary. If you do get them wrong, it’s easy to change them! But the value of adding even just one Core Event is incredibly powerful.
