[On Demand] Product Management Webinar: Analyzing Feedback

Megan Saker: [00:00:00] Hello everyone, welcome to our ProdPad webinar. This is part two of a two part webinar series that we’ve been doing covering your customer and feedback strategy. We will include in, later on in the webinar, we’ll send, we’ll include a link to the recording of that previous webinar if you missed it.

Then we were talking about the different ways to gather feedback and how to optimize your feedback channel. To ensure you’ve got a sort of consistent flow of user feedback coming into you. What Janna’s going to be talking about today is then what you do with all that feedback, right? So how to analyze your feedback and then importantly how to use that analysis to inform your product decisions.

But just before we get into it, I just want to do a bit of housekeeping. First of all, it’s not just Janna and I from the ProdPad team. We’ve got Andy here. I’m pointing down. Because she’s below me, but that might not make any sense to you. We’ve got Andy with us. So Andy is one of the product experts on our team [00:01:00] who knows ProdPad inside and out.

So for any point you’ve got any questions about ProdPad specifically, or you’d like to follow up and talk to someone about the tools that ProdPad has. Andy can, you can talk to Andy now in the chat and he can reach out and follow up with you afterwards. 

Andy Molina: Amazing. Thank you so much, Megan.

And thank you everyone. Thanks for joining. What I’ll do is I’ll leave my link here below. If you ever want to talk about all things ProdPad, I’m always on hand. Back over to you. 

Megan Saker: Great. The other bit of housekeeping is we’re here for about an hour. We will make sure that there’s time within that at the end to have a question and answer session.

In your Zoom control bar at the bottom there, you’ll see there’s a specific box for Q&A. If at any point during the webinar you come up with a question, just pop it in there and then we will come back to those questions at the end and go through and answer them. Also we’ve got the chat facility that you guys are using now that’s open and available throughout.

So do feel free to, [00:02:00] to chat amongst yourselves in the chat. Right. Just before I hand over to Janna, let me just explain for anyone who’s not a ProdPad customer what ProdPad is. So we are a product management  platform, an all in one platform that allows you to run your strategy and roadmapping, your idea and backlog management and your feedback, your customer feedback management.

The tool gives product teams a Centralized hub for all their product work. A single source of truth. It also includes a whole bunch of collaboration tools and integrations, which means you can bring the rest of the organization in on the ride. So you can let all your stakeholders and colleagues know the information they need to know about what’s happening in the product, but also contribute.

Insights and contributions in an easy way. ProdPad also has the [00:03:00] most advanced AI tools of any other product management tool, all of which are designed to save you time, to take some of the heavy lifting off your hands, to help you stay true to best practice, and to make you a better product manager with more time to spend on the most valuable products.

work. So you can help yourself to a free trial of ProdPad at any time. Visit ProdPad.com or of course you can connect with Andy here and she’ll be able to take you through and show you. ProdPad was created 12 years ago by our very own Janna Bastow, CEO and co-founder, and her co-founder, Simon Kast.

They built ProdPad as a direct solution to the problems they were facing as product managers. So we’re a team of product people here at ProdPad, building a product to solve product problems. So talking of product problems. What are we gonna talk about today? We’re gonna talk about [00:04:00] how you analyze your customer feedback to inform your product strategy.

So without further ado, let me hand over to Janna. 

Janna Bastow: Awesome. Hey, thank you so much for that warm intro. Let’s get this thing started. Good to see everybody here. Good to see that your keyboards are working. And good to see that some folks here who are already checking out ProdPad are seeing some friendly faces as well.

Shout out to them. Those who are already ProdPad users say hi in the chat and for anybody who’s interested in using it give a shout out. Let me know if you want a demo, say demo, and Andy will drop you a message right after this and we’ll hook you up. But in the meantime, let’s kick off and start talking about what it is that we’re going to be talking about today.

So we should be talking about our customer feedback strategy, right? And what it should be looking like. If you joined our last webinar you would have seen this slide. This is our complete customer feedback strategy. And, firstly, with this, you have to gather feedback. You have [00:05:00] to be effective, you have to have effective feedback channels, bringing in a consistent flow of thoughts and feelings and insights from your customer base. You need to understand what they think of your product, what their problems are and what they need help with. And then you need to take that feedback and analyze it.

You need to look for themes and trends and all that. You need to determine what the most common problems are. And once you understand that, you need to look at how it influences your product strategy. How do you take that insight and use it to help validate your product ideas? How do you use it to help prioritize what you’re going to be working on?

How does it inform new initiatives and ideas? And lastly, you need to make sure of that. That part of your customer feedback strategy is a key part of your strategy is responding to that feedback, updating customers and really what we call is closing that loop, making sure customers are in the loop with what you’re doing.[00:06:00] 

So in our last webinar, we covered the first half of this feedback strategy. We looked at a bunch of different feedback channels and talked about how to maximize each one and establish a consistent flow of feedback that’ll give you enough fuel to feed your product strategy on an ongoing basis. Hands up.

Who was here at that last webinar? Who caught that one? All right, I can see some hands throw something into the chat if you were there. If you missed that webinar you can find it at ProdPad.com/webinars, or hit up that QR code or we’ll take you straight to that recording.

You haven’t missed it. It’s all there. You can catch up on it. But today we’re going to be talking about the I think the juicier part of it, right? Which is this is the. Always on feedback strategy. If you follow the advice from the last strategy webinar, you should always have an established flow of feedback coming into you regularly from multiple sources.

That’s what we were helping to establish last time. But really this is about what you do with all that feedback? [00:07:00] And I bet a lot of you, you probably already have lots of feedback. Who here’s got a big pile of feedback already. And so how do you turn it in from like this huge pile of feed, like just stuff that people have said to you to some sort of analytical insight that you can use to make informed product decisions.

So this is what we’re tackling today. All right. Today we’re going to be talking about starting with how to triage your feedback inbox, like how to do it and how often to do it. We’re going to get really tactical. We’re going to talk about how to analyze that massive feedback for themes and trends.

We’re going to talk about how to conduct Quantitative analysis of your product usage data and why that’s really important. And we’re going to talk about how to analyze feedback during discovery work as well. And finally, once that all, once you have all that analysis work done, how to use your analysis to inform your product [00:08:00] decisions.

All right. Are we ready to go? All right. Really before we get started on anything, we need to get all of this feedback into a decent organized state. And this requires triaging all this incoming feedback and doing it regularly. If you don’t do it, it’s going to turn into a problem for you. I bet some of you might already be in this problem where you’ve got this big pile and that’s something we can help with, but all of this incoming feedback is going to completely run away from you and pile up and become an un, overwhelming, unyielding wall of noise.

It becomes unmanageable and terrifying. I’ve been there. I’ve seen lots and lots of product people there. We’ve seen it happen too many times. We actually speak to product teams and PMs who, Let their feedback inbox run away from them. And the job of analyzing it becomes impossible. So they end up walking away, and just ignoring it.

And then the problem gets worse. And a lot of teams in this sort of trouble come to us and they use ProdPad to help untangle this mess and get back on track, but I’m going to show you [00:09:00] more about that later. But my tips on getting your feedback into an organized state ready for analysis really comes down to one taking ownership, right?

If there’s no one person, if no one knows who’s responsible for feedback triaging, no one’s going to do it. And that means that it falls through the cracks. We recommend the product person is the person to take it. If no one else is owning this one that product person needs to be on the pulse for. The decision making around what to do with this feedback and then use a tool.

Don’t rely on that one product person reading the feedback themselves and trying to connect it all up. Right. Product people are really good at connecting the dots, but we’re not machines. Put it all in one tool and get it all pumped into a central place. Otherwise you’re going to have feedback everywhere and you won’t be able to connect those dots.

And it becomes really fragmented and difficult. And then, once it’s your job and where it’s all sitting, you need to triage regularly. Even when you. Do you have a tool? You need to dedicate [00:10:00] time to it, set aside time for it. Here at ProdPad, we do it daily.

We have tools to help this, but it is part of a regular routine.

And on that subject, I want to pause for a second and get a pulse check from you all. How often are you all currently triaging your own backlog? And let me just pull up the poll here. There we are. Would love to hear from you. These are the two questions. How often do you triage your feedback in books?

And are you happy with that current frequency? 

Megan Saker: So hopefully you should all have the poll sitting in a window, a movable window If it’s disappeared somewhere, I can see 

Janna Bastow: the answers coming in now and as expected, we’re seeing all sorts here. Right. A good spread. We’re seeing some people doing it anything from ad hoc through to once a month and a good spread across there.

Not [00:11:00] surprising that people have different different answers to this as well. Yeah. Yeah. All right. I’m going to give just a few more seconds to give people a chance to participate. All right. And I’m going to call an end to this because I think we’ve got some good answers here.

And then I’m going to share these answers as well, because I’d like you all to see what we’ve all said, because I think sharing is really key. I’m going to end this poll and I’m going to share these results with you. And really what we’re seeing here is we’re seeing a spread from some people who are triaging feedback as it comes in.

And, every day, really on top of it, two, three times a week, all the way through to, once a month or less than that, or some people never, they’re just not tackling it. And you can see that there’s an interesting split here. So maybe two thirds of you are looking to find more time to triage more often.

And about a third of you are saying that, yeah, it’s working for you. I wonder, [00:12:00] maybe we’ll be able to do this later. I might be able to correlate as to whether the people who are doing it more often are happy and saying it’s working, or whether that’s the people who are saying that they’re not doing it at all.

I’ll have a look at that later. Maybe we’ll share a little post about that and see if there’s something we can learn here. All right. Thanks for jumping in on that. Always really good to learn about where people are with this. So just as you’ve seen, different teams take different approaches to how often they work through their feedback.

Here at ProdPad, we’re advocates for doing it every day, or, as often as it makes sense for the feedback that’s coming in for you the amount of feedback that’s coming in. But, Here’s the thing: doing it daily establishes it as a habit, something you do over your morning coffee or after your morning stand up.

Without that habitual pattern, it’s really easy to let the regularity slip and get yourself into trouble where it becomes something that doesn’t get done at all. It doesn’t become part of your weekly routine. If it’s not a hard and fast daily slot in your calendar, then it becomes a to-do [00:13:00] list item that ends up getting bumped.

By other more important, more urgent things, because we know that we always have more urgent things to do. Right. And if you look at your feedback daily and fire off quick, thank you emails to the users, as the feedback comes in, you’re really going to impress your customers, they’re going to feel listened to, valued, and they’re going to think you and the team have seriously got your stuff together, right?

This is going to go a long way in helping your customer satisfaction and your retention. So this part should not be underestimated. And the final reason is also the reason that I believe that it should be the product manager doing the triaging. It’s how you keep your finger on the pulse of what your users are thinking about your product and the problems, customer sentiment, pain points, the frustrations, all this stuff can change on the head of a pin.

So it’s crucial you keep up to date with what’s coming in. Now, one part of triaging your feedback is cleaning it all up and getting it all into a structured format that you can actually understand and therefore, begin to do your analysis on. Now, feedback is going to be hitting your inbox [00:14:00] from a whole bunch of different channels and different people.

Some will be really nicely written out comments directly from the customer. If they’ve come in from your feedback portal, for example, might be structured. But others will be called transcripts or, a thread from between support or, some conversations that sales has captured that are messy.

They’re not necessarily going to be messy and so they’re not going to be formatted for you. That might be messy and unstructured. So I can take you through some ways to tidy this up, ready to be analyzed. The first thing you can do is train your customer facing teams, right?

Make sure that they know what useful feedback looks like. So you stand a better chance of, what comes in from them is being what you need, right? So let me explain: a customer has spoken to you sorry, a colleague has spoken to a customer of yours and sent you a message to tell you, the classic the customer wants X.

And a message. That’s all you get. And that’s really what we often get. Right. How useful is that to you? Obviously not very, why did they think they need that feature? [00:15:00] What did they think that feature would do for them? What’s the problem they’re trying to solve? That’s what you want to know.

Because maybe X feature is the best way to solve that problem, but maybe not, maybe you have a better way of solving the problem already on the road map or in your backlog somewhere. If not that, maybe you can at least take the problem and run it through discovery to explore a bunch of other potential solutions to find the best and most efficient one.

So if feedback comes into you like that, like a specific feature request, you have a bunch of work to do to follow up that, ask the questions to get to the heart of the problem, figure out what problem needs to be solved. But if you explain this to your customer team, they can do that work upfront and will save you a lot of time when it comes to triaging this feedback.

Now, we’ve actually talked about this before, and we actually made a slide deck template that you can use to train your customers on how to get the most useful feedback. It contains details on why this is worth their time and [00:16:00] what difference it’ll make for them, what questions to ask their customers, and a bunch of examples of good versus not so good feedback.

If you want a copy of this deck just let us know in the chat now, just say training deck in the chat right now training deck. And we’re going to get Andy Andy’s here on the call now, and she will send that over to you right after this. When they send you the link, you’ll be prompted to make a copy of it, Make a copy and then tweak and edit until you’re ready to take it to your team.

This is something that you can present to your sales and customer teammates so it’s to train up your internal team so that they can get feedback from your wider customer base. Right. Everyone who wants the training deck, get your word in there. These slides are Google slides, so you’ll need a Google account.

But you can use any account to access the account. It doesn’t need to be the email that you’ve signed up with today. If you don’t have a Google account, just reply and let us know, and we’ll get a PDF copy to you. That sound good? [00:17:00] All right, cool. Okay. Another really useful resource is the product feedback and ideas submission guideline.

This is really useful for helping everyone internally to understand what they should be submitting as feedback versus product ideas. Again, this will really help you improve what you’re seeing in your feedback inbox. All too often you hear Colleagues they want a customer.

X feature. So they submit to you that as a product idea, rather than capturing that as a customer feedback, a suggestion and focusing on the problem that the customer has, and again who wants a copy of this? If you want this, just say guidelines and Andy here again, we’ll follow up and get this out to you.

Again, there’s a couple of things here that you can do in advance of feedback coming in. Try to encourage the format and content you ideally want to see. So now let’s [00:18:00] look at how to tidy up feedback that is actually in your inbox and needs formatting. All right. It looks like a ton of people want this training deck and these guidelines.

So stay tuned. We’re going to be getting these out to you. All right. Chances are a bunch of the pieces of feedback in your inbox are going to be super long, especially if you’re pumping in feedback through integrations with the support tool. So you’ll have a lot of ticket threads with long emails and ongoing conversations.

And as I mentioned earlier, you might have customer success or salespeople sending in complete call transcripts, and you might be pulling these in yourself from what sort of calls you have. Now, look, you don’t want to discourage people from sharing the insights they have by telling them they can’t send you entire transcripts.

Copy that stuff, right? You want to keep that stuff. That’s gold. Keep that stuff. Here’s where ProdPad has you. We have an AI summarizer. That’s going to help you get to the really concise. Moments in the [00:19:00] in these transcripts are these big, long blurbs. Let me just quickly flip screens and then show you how that works.

Who wants to see this working in live? Who wants to see a bit of AI working? Let me just do a new share here and then flip over. All right. Yeah. We love seeing a bit of tech, right? Okay. Demo time. All right. I’m also going to take a sip of water and we’re going to stop here at our customer feedback area.

Okay, so I’m in one of my demo accounts but this is all some feedback that we’ve collected. You can see I’ve got a bunch of feedback here, which we’re going to tackle with AI later. But right now we’re tackling individual pieces of feedback which some of these are big. Big, heavy lumps of feedback.

And you can imagine that these conversations could go on and on, right? And I’m not going to read all this. You’re not going to read all this. This is a conversation between somebody on my team and Hannah. And we’re going to just summarize [00:20:00] what this was all about.

And it’s going to pull out this information and tell us what it learned from this, what Hannah’s sentiment was and what some of these key points were. And I can add this to the summary field and now I can take this or any part within here and I can start connecting it to different ideas. In my backlog, I can link this up as I see fit, or I can start processing this feedback in lots of different ways and I can quickly triage.

I’m currently looking at the third piece of feedback, but I can just quickly blast through the different pieces of feedback here and you can see how quickly I can process this feedback now. So much faster to process feedback than it ever was. Right? Bish, bash, bosh. There we are. This is the feedback summarizer.

This is the type of tool that you can be using now to just crunch what’s going on here. And now you only have to read this high level stuff. And if you need to get more information, you can dive in and figure out exactly what it is that they’re saying here. But it’s really helpful here. [00:21:00] All right. Hold that.

I’m going to flip back to my slides here, but we’re going to come back in a little bit and talk about how we’re going to deal with the rest of this feedback as well. Sounds cool? All right. So let me flip back here. All right. And Cool. Alright, so the final thing I want you to do is you triage your feedback and get it ready for analysis is to tag it all up.

We’d always recommend developing a robust tagging structure, and don’t just do this across your feedback. By the way, you should do this across other things in your backlog, your. Like your ideas and your roadmap initiatives, so how you design your tagging structure is going to depend on your product, but and also the way you structure, you segment, your users, your internal team structure, the way that you think about your market and your KPIs, your objectives, that sort of stuff.

Here’s a bunch of suggestions of ways that tagging approaches that you might adopt different things that we’ve seen as best practices. Remember [00:22:00] when tagging. The objective is to be able to easily filter and see all the feedback that relates to, whatever it is that you’re interested in at the time.

So it might be a particular area or feature section of your product or a particular user type or customer segment or a particular functional area like search or navigation or admin. Or something around like wraparound considerations like support or the help center or your pricing. And then also if it relates to something like a bug or an improvement to an existing feature or a new feature idea, these are all things that you might want to tag, use as tags as well.

So here’s an example of a possible tagging structure using ProdPad to, to illustrate it. We’ve got all the different major areas of our product, and then underneath, we’ve got a couple examples of the core features or problem areas related to each, and then we have new features.

improvement or bug, and then the functional areas beyond the core product areas.[00:23:00] 

So those are a few of the steps that you need to take as part of your triaging feedback to get it ready for analysis. Let’s take a look at the process of triaging step by step the whole process. Now, obviously this is just one approach but. It’s the one that we can vote for here. What we recommend doing is setting up a save filter and your ProdPad feedback inbox, or simply, simply you can tag on what we call the favorite product views.

You’re just seeing the product that you’re responsible for working from top to bottom or bottom to top, whatever makes sense for you. Clear it, any of the nonsense, depending on whether there might be some entries in there that aren’t feedback, like tickets closing, or someone spammed your feedback portal or something like that.

It was easy to triage those and get them out and then use the AI summarizer for Any long pieces, as I showed you, like any interview transcripts or support tickets. It works on shorter pieces as well, of course. And if a piece of feedback came directly from a customer, send a thank you email, right?

You can contact them by [00:24:00] clicking on the send email directly from ProdPad. And that opens up a space to contact them. And if it came from a team member. Then you can mention them and ask them additional questions or figure out any additional information about it. And you can use the discussion area in the feedback to have conversations, capture any questions, any communication between you and the person who submitted the feedback.

And if you’re using something like Slack or teams, they won’t even need to log into ProdPad. You can chat on ProdPad. And they can be chatting in Slack or Teams and the conversations all get synced automatically. Same thing with email. So you can actually just log have this conversation in ProdPad.

And if they’re replying to their email notifications, it logs it as well. And if the feedback relates to a bug, you can add mention support or whoever creates bug tickets and then tag it up. Add whatever tags make sense to you as per your tagging structure and then let your AI assistant in [00:25:00] ProdPad link the feedback to any related ideas.

So this is really neat. I didn’t show you this, but this is something that it can do. In ProdPad, your AI assistant will automatically flag up your feedback. pieces of feedback and any related ideas. It’ll group it together. Anything that’s related in your backlog. So it’ll say, here’s five suggestions.

You just have to take a look and confirm and say, yep, those are linked. And then make sure that any contact details are up to date. If you have an integration with your CRM, like, Salesforce this would be automatically taken care of. And so you can just check that information is correct for the customer and it just makes it really quick and make it makes it much more efficient when you’re trying to close the loop with that customer that everything’s in the right place, you’ve got all the information for them and then when ready, move it through to reviewed or archived or whatever sort of next step you need to do with that piece of feedback so that that it’s been processed and you can move on to the next piece of feedback.

So that is a triaging process as you can do it in ProdPad. But [00:26:00] now we’re going to move on to how to analyze it. We’re ready to look at that.

That’s my time to take a sip as well. All right. All right, I’m going to cover off a few different analysis methodologies. So let’s talk about thematic analysis. So there’s two ways to do this. Let’s call it the easy way and the hard way. The latter is the easiest. Manually doing thematic analysis through coding your data as in tagging it, and then going through an affinity mapping exercise.

And for everybody who’s not familiar with affinity mapping, it’s like when you put everything on post it notes and then you group the post it notes by what they’re about and to, to into groups. The former, the easy way is letting the ProdPad AI do it for you. So I alluded to this before.

Let me just jump in and quickly show you how it’s doing this automatically for you. It’s grouping everything up. For you automatically. So again, quick demo, [00:27:00] new share. I’m going to flip over to my other screen here. All right. I’ve got a bunch of feedback here. I’ve got 95 pieces of feedback.

Again, I’m not going to read all this. I’m not going to make you read all this. I’m just going to go to what I call signals. And it’s already done all the grouping for me. I could hit refresh, but it’s already done the job here and it pulled out and grouped things together. So think of this as pulling together three stickies or three pieces of feedback and pulling them into one particular space.

So now I can see that it’s got five pieces of feedback here and I can flip through, I can see the feedback and I can flip through and see the feedback. And if any of these are a bit unwieldy, I can use the AI to summarize it for me and I can add that. There we go to the feedback and I can just quickly blast through and update these as needed, check the information and blast through and triage this feedback as I see fit.

And I can also take this information and I can save these as saved. Grouping saved signals here. And [00:28:00] what that allows me to do is like tag them. So I’ve got 11 under reporting and analytics five under supporting file formats. And this basically creates a tagging system.

So I can just quickly, say of the 95, show me the ones that are related to this sort of thing. So just a way of quickly understanding what’s in my backlog of feedback and then dealing with it.

Cool use of AI? What do we think, everybody? 

Megan Saker: I like it. I know, obviously, I’m not impartial. But having done many Affinity Mapping workshops to just, and it taking hours to just, yeah, just be able to automatically surface the themes. 

Janna Bastow: This is exactly why we’re building it this way, because I used to spend, I don’t know, how many hours a week, just trying to decipher all the stuff people would say.

And just, you would never get it right. You would always miss stuff. You’d always, something would [00:29:00] fall off the end somewhere and you’d miss some signal from a customer and you’d end up building the wrong thing. Whereas what this is doing, it’s basically Pulling it all through and making sure that you’re not missing a beat and just making it so much quicker for you.

So it’s just removing that grunt work. So if anybody’s interested in trying that signals, if you’re not already a ProdPad customer, then let us know just say signals in the chat. How about that, everybody? And we will get back to you and we’ll have Andy set up a time to chat and take you through and show it how, show you how it works either in our demo account, or if you want to, we can get your feedback into ProdPad.

It literally takes moments. And then we can run signals on that and we can just do your affinity mapping grouping and just take it away from you. How’s that sound? So if you want to see a a demo of this, say signals in the chat and we will get you set up with that. All right. If you don’t [00:30:00] have the luxury of signals you can do thematic, a thematic analysis yourself through an affinity mapping exercise.

So to do that wherever you store your feedback you’re going to want to get it into some sort of, whiteboarding sort of space, right? Whether this is, actual post it notes or Miro, FigJam, whatever you use. And what you can do is you can import stuff into a tool like that.

And then start grouping stickies based on commonalities, right? This is essentially what the AI was doing there. You’re basically saying, do these things relate to the same feature? Are they expressing a similar problem? Are there? related to the same function or need. Once you’ve done your first grouping, you want to give each group a title.

You want to find a place where you can find sub themes within each group and then start breaking them out into new groups. And it’s more of a. An art more than a science, right? There might be multiple ways that you can break this stuff up. But here’s an example of what it might look like.

I’ve used [00:31:00] a mobile banking app for the example here. And once you have these groupings of these different problem areas, you want to create Candidates on your road map, candidate roadmap initiatives out of them and start to attach ideas, potential experiments on how you might actually solve these things.

So your goal here is to break down or group together what people are saying into different. Categories or areas, and then start using those to understand what signals are you hearing? Are people complaining more about this area or this area? Does this area represent more of a problem or this area represents more of a problem?

And only then do you want to start coming up with ways to start solving these. Does this look like a familiar exercise? Who here has been doing this for ages as a product team?

All right. So far we’ve only analyzed your incoming feedback, the stuff that’s sitting in your feedback inbox and has come [00:32:00] in from proactively from your customers or what was shared with your customer teams during calls or conversations. So this is a bit of a generalization, but in most cases for a customer to be compelled enough to proactively give product feedback, they’re one of two things.

They’re either super happy. Or they’re delighted with the customer, the product of the feature that you’re building, or they’re frustrated. They have a problem that is painful enough that they’re bothered to, get out there and tell you about it. So you’ve got the two poles of customer sentiment here, and that makes it really tough.

So what about. Average Joe, right? Most of our users are going to be average Joe. They’re going to be lukewarm on a lot of the stuff that you do. They’re not feeling frustrated enough to tell you about the product. They’re not feeling delighted enough to go and shout about it out of nowhere.

What is their experience of your product? What problems do they have that could be solved or solved better? I’m [00:33:00] So even if Joe here is feeling something towards either end of the sentiment spectrum, he might not even be a feedback kind of guy. Some people just stay quiet. A lot of people aren’t.

So if he’s frustrated, he might just walk away. He might be annoyed and just give up. So you need to proactively look at the behaviors of your non feedback submitters to get a sense of their problems. So here’s how to get into the data. All right. So let’s talk about quantitative analysis. This is where your product analytics tool comes in.

This is where you’re pumping your event data. You need to make sure that you understand that data like the back of your hand, right? So you can approach font analysis of your product. usage data in two different ways. One, having a specific question in mind that you want to interrogate and find an answer to, like, what has adoption been like on feature X?

Or general analysis of [00:34:00] behavior. And this is actually something we recommend that you do on a regular basis, get a pulse check from your usage data as well as your incoming feedback. Together you can be confident that you’ve really got a finger on the pulse of how your customers are experiencing the product.

Another way to think about the difference between qualitative and quantitative analysis is to distinguish the attitudinal research versus behavioral research. So attitudinal research is about understanding users’ attitudes and perceptions and beliefs, whereas behavioral research is about what users do rather than what they say they do or would do.

So conducting quantitative analysis of your is one way of doing behavioral research. You’re going to go into your analytics data in the hope of coming away with some solid findings about how your users are behaving in the product. So what should you [00:35:00] be looking for? Think about feature usage.

Are there some features that are getting used more than others? Are there any that are being neglected? Right. Just because you shipped a feature doesn’t mean that it’s actually being used. So check that out. Look at session times, how quickly do people move on from your product or from the feature is, the feature not holding attention.

very long? Do they not finish making use of the whole journey of using that feature? Look out for friction points. Where do sessions most commonly end? Where is the time on page too short or too long? That might indicate where people are getting confused or frustrated and look at the conversion journey.

So consider the path through your, through both your initial sign up and onboarding across other flows throughout your product, like, product tours, for example, and look at the drop off at each stage. So you understand which stages there might be friction and where things can be improved. [00:36:00] And you should also consider the ways in which you are slicing and dicing that usage data.

One big overall look at your usage data may not be all that useful. It’s just too much. And it has all this noise. If you have very distinctive customer segments or user types. You’re going to need to look at each one separately. So this brings me on to segment and cohort analysis. This is where you are using these too.

Analyze your usage data by the type of users or when they’re using it. So segment analysis, you’re grouping users based on shared character rate characteristics and analyzing the usage data from them. We have different user types here at ProdPad. We have reviewers who are typically stakeholders invited by the product managers.

And we can run segment analysis to see how reviewer users behave in ProdPad versus the admin and editor users, [00:37:00] the product manager type users. Cohort analysis is one of my favorites. It can be a really powerful tool because you’re grouping users based on time. This. You can do things like grouping them into cohorts based on when they signed up.

So you can track behaviors on, based on when they signed up. So it can be really useful for stuff like your onboarding flow behavior before or after a major feature release, or checking whether things change with seasonal or, ongoing time based, changes as well.

So what’s really important here with sentiment or cohort analysis is that you’re really trying to make the analysis as meaningful as possible. You’re trying to look at a particular type of user or a particular point in time and glean insights from that. That’ll, function like direct feedback does, tell you what their problems in general and what your problems with your product might be as well.

And the other way which you can use quantitative analysis when [00:38:00] you’re looking for feedback to inform your product strategy is in conjunction with qualitative analysis. So let me dive into that. You can use your qualitative, quantitative analysis in conjunction, right? We’ve already talked about how quantitative can get you insight on your average Joe.

Those users haven’t been compelled to tell you how they feel. You can also use quantitative analysis to help you get qual analysis on these average Joe’s too. So let’s say you want to get feedback on a certain feature. You can go to your usage data to understand the breadth of usage for that feature.

Then you can identify users from the top. Middle and bottom percentile in terms of how much they’re using that feature, you can then use that insight to inform your qualitative analysis. That list of users becomes the list of people you reach out to for feedback on that specific feature. This becomes part of your research plan that you are making.[00:39:00] 

that you use, that you’re using to make sure you’re listening to a range of different user levels and experience levels with your product. So your quantitative analysis is telling you where to go for your qualitative analysis. And your quantitative analysis can also help you shape your research agenda.

So use your usage data to spot trends or identify patterns that you can then run discovery on and do. Qualitative research around it.

So you’ve done a whole bunch of analysis. ProdPad signals tool has done your thematic analysis for you. You’d run some quantitative analysis on your usage data. You have a whole bunch of insights. So what do you do with them? Like on a practical level, what are the steps now to turn all of these learnings from your feedback into decisions and feedback so feedback can influence your product strategy.

I guess three ways, broadly speaking. So it helps you evidence your existing [00:40:00] ideas or plans, validating, top down product decisions. It can help you with bottom up strategic thinking, giving you new problem areas to explore and ideate around, and it can inform your prioritization decisions.

As ideas get more and more related feedback, you need to adjust your prioritization accordingly. To the first point on a practical level, you can achieve this by linking Each piece of feedback that comes to you from any related idea that already exists in your backlog. And to the second point, this is where you take your input from signals, those themes that have been identified and you capture them, you can save them and then consider that list as your discovery to do list.

These are the new problem areas to explore. On a practical level take these saved areas that you think have potential and create a roadmap candidate. And this would then become your initiative that would sit to The right of your Now-Next-Later roadmap, ready to be brought into your [00:41:00] roadmap, probably into the later or the next when the time is right.

Once I have my new problem area captured as my roadmap candidate, I would gather ideas over the coming weeks, either mining my backlog for existing ideas that could be possible solutions to this problem or through new ideation with the team. And for. Each idea, I would look at the impact and effort, that impact and effort chart that we have.

I would look at that alongside the amount of feedback linked to it. So those 2 factors would help me make my prioritization decisions and determine when I need to promote this roadmap candidate. Into a new area of the road map, bring it forward to actually work on and this brings me to my final point here.

In ProdPad, it’s as simple as 1 click to see your entire backlog sorted by feedback, most to least. And you can also see at a glance at the initial initiative level, every card on your roadmap. That’s clearly labeled with the amount of feedback [00:42:00] every stage of your process. You can be able to see related feedback.

Clearly attached and flagged on all your ideas, initiatives, and roadmaps. And this is all part of what we call the golden thread, making sure that you can see all the way from your vision, right down through your OKRs, your roadmap, your ideas, right down to the feedback that has come from your customers.

And this golden thread is what keeps you focused on doing what’s best. Most important for your business, the visibility of that thread and ProdPad makes it super easy for your stakeholders, for your entire team to understand what you’re doing. And most importantly, why, how it’s all connected.

Megan Saker: Janet, just quickly on, on the second point there about the way in which your feedback analysis can spark new ideas, the sort of bottom up. Strategy. I was talking to someone on our product team the other day, and it was interesting. We’re talking about signals tool, and they were saying what actually what’s [00:43:00] what they find super useful about that tool is Maybe not necessarily the top signal with the most feedback, or even maybe the second one, but the one like a few down, because if you’re, you’re triaging your feedback, you’re talking to customers the top ones, the things that are bubbling up that are most commonly talked about, right, you should have your finger on that pulse.

You know them. It’s great. You see them. group together, you see them articulated and that’s great. But most of the time, like you’ll be tackling those things. So it’s a case of attaching that feedback to existing initiatives on your roadmap. But looking a few down, then you start to see things that are starting to bubble up.

And you get that sort of forward view. And so she was just saying, for her, that is what it really is. Given new insight and transform stuff, which feels like because of that automatic signals tool servicing those themes, you are looking [00:44:00] further ahead than ever before.

So I think that’s really interesting about the, bringing it from the bottom up, thinking of the new ideas, the new things and creating those roadmap candidates accordingly. 

Janna Bastow: Yeah, that’s a really good point is what what’s really important is giving that visibility and having product managers think further ahead and making more informed decisions about what it is that they are what they’re heading towards so that they End up building stuff that reflects what the customers are asking for, but isn’t just customer driven, isn’t biased by the latest, loudest customer, because in the past, the way that product teams have worked, I know this because I’ve done this oftentimes things get thrown off track because you end up with, a customer who’s.

Asked for something quite loudly through the sales team and waggled some dollars in your way, or something came up, three or four times recently. And so you get sidelined by recency bias. And so that ends up taking [00:45:00] precedence. What. Should be happening is removing those biases.

So you can say, well, here’s, what’s important to the business, right? We’re trying to be the X of Y and we’re trying to solve this problem and this problem. And, here’s what we’ve heard from our customers, not just in the last week, cause that’s what you can remember. And not just from this one customer, which, that’s obviously some implicit bias there.

Where, not just from one loud stakeholder, But actually our entire body of feedback going back, however many years you need to so that you can basically say, here’s what we’ve been hearing. And let’s tie that in with what we’re seeing in the app as well. Once you get that bigger picture, it’s, you’re not customer driven, you’re customer informed.

And that starts giving you a much clearer picture as to whether the stuff you’re working on is in fact the right stuff for your business or not. Right.

On that note. I want to open it up. Are there any questions that anybody here has? Throw them into the questions area or Megan, if you’ve got any that you want to throw away and let’s yeah. 

Megan Saker: So we’ve, [00:46:00] yeah, we’ve got one that’s come in, which is super interesting.

So the question is related to when you have a, Busy roadmap, a backlog full of ideas, like more ideas than you can get to. Do you keep triaging feedback? And I think I understand what they’re saying here. I think I think sometimes often you’ll think that, your feedback coming in is, Well, we’ve said, right, it’s the fuel for your product strategy, it’s where new ideas come from, it’s where new improvement ideas come or whatever what’s your advice if you’ve got a massive backlog already, your dev team are going full throttle, there’s plenty to be getting on with, should you still be looking at your feedback every day?

Janna Bastow: Yes. So you should be looking at your feedback. It should be part of what you’re doing, because the thing is even [00:47:00] though you’ve got a full throttle pile of stuff that’s in your backlog this feedback that’s coming in should be the type of thing that informs as to whether you’re heading in the right direction, right?

And you might discover that actually based on the feedback, the stuff that’s in your roadmap that was important because you put it on the roadmap six months ago, and you’re working on it now, turns out. Actually, you need to change direction, right? I know you put it in your plans, and that’s what the developers were going to work on in November.

But can you change direction and have them work on something else? Because, all the customers are now talking about this other thing. Now, it’s rare that you get so much customer feedback that you have to Hold the plug on something and change direction entirely. But an example is, for example, some new tech comes in, AI lands and all of a sudden you’re like, wow, like people’s expectations and our capabilities have changed as well.

Like you bet that we pulled the plug on our roadmap and changed direction when that capability came in and it’s worked wonders for us because we’ve been [00:48:00] able to take advantage of that. And that was all from listening to the market and listening to our customers and making sure that we had that flexibility.

So you do have to be listening to the insights from your customers. At the same time, it shouldn’t be an overwhelming job, right? If we’re doing our job, right. Then you should be able to do your job right. And it should be no more than a few clicks of a button and something pulling up to you, pulling up and saying, here’s what’s been in your feedback for the last week or so, right.

It shouldn’t be something that you’re sitting here with a thousand post notes stuck to the back of your head and you’re not sure which one is which, right. That’s product management of old. Product management today should allow you to understand everything your customers are saying and connect it really easily.

And that’s why we’re saying definitely stick to it, if you can daily, even if it is just go in there, click a few buttons, see what’s in there and draw those connections for you. Nice. 

Megan Saker: Great. Thanks. We’ve got a question here from Cameron. So [00:49:00] there he says we’re in the process of in, so they’re importing everything as ideas from their old JIRA based process.

They’re transferring it all, I assume, into ProdPad. feedback, is there a recommended way to decide what remains as an idea and what can be moved as feedback? I think this sort of relates to those guidelines, which we’ll send out to all you guys there. We’ve outlined exactly that. What should be what should be Considered feedback on what should be an idea, but maybe 

Janna Bastow: Yeah, so we’ve got to tackle this really quickly, but Cameron, we will send you those guidelines and we’ll post them out.

But basically the gist of it is that things that are coming directly from your customers should come in as feedback if they’re representing new product areas or solutions as opposed to Problems to solve. I put those in as ideas. So we’ve got a list of [00:50:00] if it looks like this, then put it as an idea.

If it looks like this, then put it in as feedback. So we’ve got a guide there that we’ll send you that sort of helps you delineate those. So I want to tackle that and send you those guidelines, but I want to tackle the other questions that we’ve got here as well. Cause Eric wrote in and he said, where is there a good place to hold status? He said that senior business stakeholders don’t want to hunt and peck for updates. Well, your senior stakeholders if we’re talking about. ProdPad world, your senior stakeholders will have access to it and we’ll get updates about what’s going on there.

If you’re talking about in general, that’s part of your role as a product manager is to keep people informed, but ProdPad is a tool that will help you keep people informed with that. If you want a demo of that, if you want information on that, hit Andy up. She’ll drop her details now, so everyone can follow up and she can talk you through how to make sure that your senior stakeholders are in the loop.

And I [00:51:00] want to get to the last question as well. Cause I see one more popped at the very last minute. Vinay says, thank you for this fantastic talk. Thank you, Vinay. He said, would you please share how ProdPad would help creating the golden thread? So what I mean by golden thread, this is a term that I see thrown around by companies who are trying to create a sense of cohesion and clarity with.

Okay. What’s important for the business all the way through down to what is happening, at the more tactical level. And so really what it’s about is making sure that when somebody says, this is what’s important for the business, right? Our goal is to be the X of Y. That is our vision and our mission statement.

And we’re doing that by hitting these objectives. And these are the things that we’re. The problems that we’re solving and the opportunities areas that we’re tackling is, and that’s our strategy. That’s our roadmap. And attached to that are things like, here’s more tactically the problems that we’re going to solve to do that.

And here’s why we’re doing it as our customers. Right? So everything is attached together [00:52:00] where you can see this feedback is happening or. This feedback is addressed by this particular experiment, which is being done because it solves for this particular problem. And this problem is being tackled because it helps us move this needle.

And this objective, and this objective is important because it speaks to this piece of strategy. And this piece of strategy is important because it’s tied to this vision for us. So the whole thing should be tied together conceptually, and also, digitally. In ProdPad, the whole thing is tied together.

You can click through from one to the other, and it tells you that story. So if anybody ever says, well, we’re working on this idea, why? Well, you can click all the way up and see how it’s connected to the big vision and the objectives in between. And you can see what Feedback has been driven by it.

And if you’re looking at the big vision, you can see, well, what are we actually going to do? That’s tied to it, and anybody who’s a senior stakeholder or junior tactical person doing the work, they’re all connected by this [00:53:00] golden thread. Very short answer, but I do recognize that we are just about out of time.

We went slightly over because questions came at the last minute, but thank you so much for those questions and good to hear from everybody. 

Megan Saker: Yeah, thank you, Janna for that insight. That was great. And yeah, anyone who wants to join our next webinar, please do. Here it is. We’ve got to say how to say no to the CEO.

So stake, stakeholder management tips. with Melissa Rappel. 

Janna Bastow: Yes. All right. So see you all back here. Same time, same place. We’ll be doing another one of these. Thank you all once again. Thank you, Megan, for teeing us up once again and Andy for being on hand to follow up with everybody who wanted to get the demos and all those guidelines and all those all those treats that we’re going to be sending out.

All right. And thank you everybody. Take care and see you again next time. Thanks. Bye now. Bye for now.