How Coursera scaled with Pendo to meet surging demand during COVID-19

With schools and universities shifting to online instruction during COVID-19 and workers displaced by economic turbulence eager to re-skill and land new jobs, usage of education technology and online learning platforms has surged since the spring.

As a result, Coursera, a leader in online education that offers virtual courses from hundreds of colleges and universities, found itself welcoming a groundswell of new users onto its platform and facing the challenge of getting all of them up to speed, quickly. And, the company served as a lifeline for universities looking to quickly ramp up virtual instruction, making its course catalog freely available for professors to use in their classrooms. Many of those educators had never used the platform, and some had never delivered instruction online before.

That meant scaling and automating a process that’s historically been a very manual effort, says Terri Czerwinski, a product manager at Coursera. To get the job done, the product team turned to Pendo. Pendo’s in-app guidance and messaging capabilities provided the infrastructure they needed to set up automated onboarding flows and messaging.

“We went from a small number of customers to a very, very large number of customers within the space of a few months,” Czerwinski says. “We relied heavily on Pendo, both to show them how to use the product and to market our online webinars so they could hear us talk about how to use it more effectively.”

Pendo is also helping Coursera professors keep their classes on track. With little to no in-person interaction between students and professors, it’s not always as easy for instructors to keep tabs on each individual student to ensure they’re paying attention and aren’t falling behind in a virtual format.

That’s where Coursera uses data from Pendo and other sources to monitor usage and engagement, then nudge students who lag onto the paths that the most successful students follow. For example, the data might show that students who watch the first video lecture within five minutes of starting the course are significantly more likely to complete it. So, they send a message to students after enrollment to direct them to that first video, says product manager Helen Zou.

Pendo has also historically been a key tool in encouraging product-wide adoption at Coursera, Czerwinski says. In-app guides are routinely directed at administrators as they log in, pointing out new features and walking them through their first use.

In-app surveys are often used to collect user feedback about features that are in development or beta testing to help refine them and maximize their usefulness. That data is helpful in identifying user pain, so that they can be directed to the existing feature that will help alleviate it, or build new ones to solve the problem, says senior product manager Jenny Wolochow. This method, for instance, led the team to realize that administrators felt quizzes took too long to set up, so they built a feature allowing users to import Google Docs with the quiz information directly into the platform.

And, Czerwinski says, Pendo data helps examine the paths users take through the platform, reveal which features are heavily used and which users rarely engage with, and A/B test things like where a feature is located on a page to encourage engagement. “That helps us understand the trends and outcomes that are related to behaviors, she says.

When Pendo data indicated a large number of users were becoming stuck at a certain point on the platform, such as educators who failed to begin inviting students into their course after logging in for the first time, Pendo guides are designed and deployed to demonstrate best practices.

Product planning and refinement

Each year, the team at Coursera hosts its annual conference to showcase the latest trends and best practices in online learning and the newest parts of the platform.

And like each of her last seven years at the company, Wolochow is expected to launch something new. Wolochow manages the part of the platform educators and faculty use to build and administer online courses, and in the past, she’d used a combination of intuition and user feedback to guide product development.

At the last conference, her team took a different approach.

“We used data from Pendo to find out which pages were visited the most by partners and looked at what they interacted with the most,” she said. Those insights prompted the team to invest in UX improvements on the most-used page, changes that educators and partners rated highly on satisfaction surveys also delivered using Pendo.

Data-informed decision making has been a years-long journey for Coursera. Ultimately, Wolochow’s goal is to have enough insight from data and user research to feel confident saying yes to the most fruitful projects and no to the rest.

“We don’t have the time or the resources to do everything,” Wolochow says. “When we prioritize, we decide to solve a use case or problem for one stakeholder over another.”

Coursera uses Pendo primarily for this reason—to vet out the yeses and the nos using product usage data and qualitative research from surveys and experiments. Sometimes, a “no” means removing things that already exist in the platform.

Ultimately, their goal is to feel confident that the few yes’s will make for an incredible product experience for thousands of educators delivering classes and course material on the platform.