Product Management Webinar: Building Innovative Products

[00:00:00] Janna Bastow: This is the Product Experts series that we run here at ProdPad. We run them on a regular basis. It’s a series of webinars that we run. We have run a bunch of them in the past and they’ve all been recorded, and it’s a mixture of presentations from our experts and some fireside, some chats with myself, fireside conversations.

And it’s always with amazing experts that we bring in who bring their insights with a focus on their experiences, their content learning, sharing just chance to understand what these experts are bringing to the table and what they’ve learned out there. So today is gonna be recorded and shared, and you will have a chance to ask questions.

So feel free to take advantage of the chat and if you’ve got specific questions that q and a is gonna be a friend. Before we jump in and introduce our guest for the day, I wanna talk a little bit about the tool that we’ve built here at ProdPad. ProdPad is a tool that we built when we were product managers ourselves.

Myself and my co-founder Simon, we’re both founders of Mind The Product so you might know us from that, but we were also product managers Before we started all this, we just needed a tool to help us do our own jobs. We needed something to keep track of the ideas and feedback and experiments and everything else that was coming at us.

So we built something and what it did, it gave us control and organization and transparency, and it created this single source of truth for all the product decisions that we had. And it’s now being used by thousands of teams around the world. It’s a tool that you can try for free. We have a free trial.

We also have a sandbox account, which is preloaded with example data including lean roadmaps, now next later roadmaps OKRs feedback and experiments and ideas and stuff like that. So you can see how it all fits together. But also, really importantly is that our teams made up of product people. So we love to hear your feedback.

We love to talk to fellow product people who are trying it out. So jump in, let us know, what you love, what you don’t love. We’d love to hear how you get on with your first taste of it. And on that note, I wanna talk to you a little bit about some of the stuff that we’ve been building.

ProdPad is constantly being developed. The latest thing that we’ve been shifting out there is stuff around AI, as you might imagine, and we have turned ProdPad into a bit of your AI product coach and sidekick.

So just in the last. Quarter, maybe four or five months now. Since we got access to the GPT API, we’ve been able to allow you to generate ideas and user stories. Flesh out your idea details, generate key results based on your objectives. Come up with ideas for roadmap cards and ideas that go on your roadmap, but also give you, we didn’t wanna just generate stuff.

We wanted it to act as, like your sidekick and to help give you coaching on whether your vision was in fact aligned with your whether your ideas were aligned with your vision and whether your vision was actually any good, whether your roadmap was any good. And it actually gives you really clear instant feedback that you can act on and start using it to improve the direction that you’re taking your product.

We’ve got another one that’s in beta, which is actually a chat bot that you can use in your Slack to talk to it about. Product management principles and talk to it about what’s going on in your product account. Stuff like are my objectives any good? Or Can you summarize the last week’s worth of feedback for me?

And it’ll do that on the spot. So some really cool applications of AI that we’ve been playing with. So I would love to get your feedback on that. I know a lot of people have been using chat gpt, and other stuff like that for product management. Would love your feedback on it. But that’s enough about us.

I want to introduce our guest for the day. This is Matt Wallaert. Everybody say hi to Matt.

So for the last couple decades, Matt has used behavioral science to tackle a wide range of challenges. He has gone from being in charge of startups and big companies to becoming the head of behavioral science at Frog. And in 2022, he started BeSci.io. In his book Start At The End and his talks all around the globe, he shows how he can make products and services that really matter, all without needing an advanced degree.

And he’s got projects like Get Raise that have helped women who weren’t getting paid enough get over $3.6 billion in raises, which I think is absolutely awesome. He mixes humor in science and things like mediocre white men as another project which seeks to, so to drive positive change. He lives in San Diego.

He loves surfing and wearing cowboy boots. And with his educational background and diverse roles, he’s a leading figure in behavioral science. So with all of that, I am super excited to introduce you all to Matt Waller and we’re getting a chance to chat to him and hear from him about his background and where he’s coming from on this.

So everybody say hi to Matt. 

[00:04:58] Mall Wallaert: So as you mentioned, my name is Matt Wallaert. I’m an applied behavioral scientist. So I started my career in product because applied behavioral science wasn’t a thing at the time. Ooh, somebody knows me well and says not surfing in cowboy boots.

I hope No, I don’t wear shoes while surfing. I mostly don’t even stand up on surfboards anymore ’cause I have now done terrible things to my shoulder. And so I try to do things that are more in the water, but it happens. I started my career in product. I really started in academia.

I’m an applied social psychologist by training and then figured out that academia wasn’t gonna be applied enough for me. And so left, joined a startup as the head of product built and sold that to LendingTree, sold another startup. Went to Microsoft as their head of product, then chief Behavioral Officer at Clover Health.

Took them public. And then as mentioned was the head of behavioral science at Frog until I left last year. And these days I run Besci.io, which is just a collection of a bunch of applied behavioral scientists who do capability development within companies. So we don’t do project work, we just teach behavioral science.

So I want to talk for the next 15, 20 minutes about applied behavioral science and its connection to product and why I think product is a very interesting place right now because it’s. There’s this technical product management half, right? Sort of the t p m, like how do we get flog engineers to build things, how do we make stories, et cetera.

And then there’s what you might think of design, product design, right? How do we decide how we’re gonna change people’s behavior. So I think eventually product will devolve into those two pieces with more clear cuts of, because there are very different skill sets. I think I’ll primarily be talking about the sort of design product side of it.

Although I think it’s applicable to how you get engineers to build things too. So we gotta clarify terms before we talk. So when I say applied behavioral science, I don’t mean translational research. So I don’t mean Danny Kahneman or somebody wrote a paper and then we read that paper on something they did in a lab and then we say, yeah, that should be a good idea.

Then we do it in real life. That is not applied behavioral science, that’s translational research. I’m talking about applied science, which means we’re doing in situ experimentation in order to figure out what we wanna build. Underlying this is the idea that every business to me is really three models.

In a chain. There’s a business model set of behaviors that, a, that be that a business changes, and then a way to monetize those behaviors. A behavioral model, right? What, how do we make those behaviors actually happen or happen more frequently than they naturally would? And then an operational model, how do actually do that at scale?

So let’s take my canonical favorite example, Uber. What does Uber’s by the way, you’re gonna have to use the chat to answer these because of the way this is set up. So get your chatty fingers ready. What is Uber’s sort of business model? What behavior does Uber want you to do and how do they monetize it?

And I expect David Harrison to go first. ’cause he said, challenge accepted. David, you have to be the first person to answer every question. Go. What is it? What is Uber’s sort of business model? What behavior do they want and how do they monetize it?

People have abandoned the chat altogether. They’re frightened by me. I understand that Ms. Bastow should, shouldn’t have, you shouldn’t have invited me. 

[00:08:08] Janna Bastow: I’m putting my name as well. I think it just takes a moment for people to, 

[00:08:11] Mall Wallaert: Give people to ride to their destinations. Yeah. That’s the behavior they want.

And then how do they monetize it? They charge for it, right? It’s a direct payment yeah. I’m gonna challenge a few of these, right? Encourage people to travel spontaneously. This is really interesting because I think Uber is not in the business of encouraging people to go anywhere, right? They are meeting the need that is wanting to go somewhere, but they don’t engage in getting you to want to go somewhere.

They just meet that need. All of us, all of us. Any business has to meet some naturally existing need. They have to say, Hey, this is outside of our scope. This is what we’re harnessing, right? That’s true in the world. Let’s take another one. Let’s take Spotify, right? What does, what behavior does Spotify monetize listening?

How do they do it advertising and subscription, right? So every business model is some behavior plus some sort of way of monetizing that you could imagine, by the way that Uber had, you could have getting people to point a point B, but we’re gonna charge via subscription instead of through direct payment.

We’re gonna charge via. I don’t know, cash back from the government. The government’s gonna, this is vias model, right? The government is actually gonna pay us to move people from A to B. So you could imagine lots of different kinds of monetization engines and lots of different kinds of behaviors that you could, that sort of make up the companies in the world, right?

Think about Coke, right? Coke wants you to drink, right? They monetize the behavior that is drinking, and then they usually monetize through direct payment, right? Direct to consumer payments. Then there’s the behavioral model, right? How do you actually get people to do that? If you’re Uber, this is primarily through the app.

The majority of their behavioral space is the app itself, right? Although there are some other triggers. And then finally they have an operational model. You have to be able to do this in a bunch of company countries. You need legal and compliance and somebody to hire drivers and hr, and you have to process payments and have a technical infrastructure.

All of that lives in the operational model. So the, when we talk about applied behavioral science, we really talk about that middle bit. That’s the behavioral model. And what we’re talking about is, when I say behavioral science, behavior as an outcome and science as a process. So when we talk about a scientific process, we usually talk about four phases.

And I use side to remember them. Strategy, insights, design, evaluation. This is gonna be super familiar to most of your product folks, but what I wanna challenge you with is while you superficially are like, yeah, I do strategy, insights, design, and evaluation. That is a thing I am familiar with. Do you really, and do you really do science when you do it?

And do you really do it in the fullness of things? So let’s talk about it. When I say strategy, what I mean is writing a behavioral statement. This is where I take my orders from the business. Jan’s the new head of Uber. She says to me, Matt, I want you to get people to ride. Okay, what do you mean by that?

There are lots of technical little fiddly specifications that are gonna make me go in really different directions. Let’s do this for Uber just because. Who is Uber for, right? What is the target audience of Uber? Who’s Uber for this is the downside of chat. It just takes too much time. The public, right?

It’s for everybody. This is pretty unusual, by the way. I used to run Microsoft Ventures in New York, so I got pitched by thousand startups a year and they’d be like my app. It’s forever. No it’s not. Most products are not actually for everybody. Uber’s kind of a unique Anomaly there in that, I have a seven year old, I’m pretty sure if he could figure out how to order an Uber, they’d take him wherever he wanted to go and they’d be like, okay, cool.

You must have to have permission. Off you go. If a chicken can use a phone, they’d be like, get in the back, and they’d drive it across the road. They don’t care. They really are for everyone. And some of you actually have gone ahead and put in what I would think of as even the next step, which is limitations.

Because even while we think of them, for everybody, there are some people who cannot engage with Uber. What are limitations? Limitations are the binary preconditions that must exist in order for me to engage with a behavior. So if we look at something like. Impatience as an example. Impatience is what we call a pressure.

It’s something that can go up or down. It can vary between people, it can vary between times. I can probably manipulate it. Limitations are zero one things that must be true. So for example David said when you accessibility, you have to be able to physically, literally get in the back of the car, right?

Other people have said, Hey, you have to have a smartphone. If you don’t have a smartphone with connectivity and battery, you can’t actually do it, right? There has to be the availability of drivers. There’s a time and place component. Someone has to be willing to pick you up in the place that you are and willing to drive you to the place that you want to go at the time that you wanna get there, and at the time they wanna pick you up, right?

Those are all zero one things that must be true. And, they highlight one of the important features of a behavioral statement, which is there’s no right or wrong behavioral statement. It’s just a negotiation between me and my stakeholder. Janna, when you know you’re the new CEO of Uber, you come to me, you say, okay, I want you to get people to, to ride.

I can say, okay, right now we only accept payments, electronic payments. Are you okay with that? And you can say no, you gotta get the people who can only, we gotta get everybody. So you have to figure out a way that people who don’t have electronic payment can pay. And I say, okay.

And then I just put that in my thing. What it be, receive, it really is an agreement between me and my stakeholders on where am I gonna end up? Not, how am I gonna get there? Not what am I gonna build, because I don’t know any of those things yet. It’s just, it’s like a, it’s one of the analogies I use for behavioral statements often is, it’s like a.

Formatting your wish correctly. Every movie we’ve ever seen with a genie teaches us one thing that is, which is that people are really shitty at making wis’hes, right? Luke rubs Matt’s fuzzy Balded head. He says, I wanna be the fastest man alive. And Matt says, great, no problem. And he snaps his fingers and he kills all the other men on earth.

Congratulations, Luke. You’re now the fastest man alive. You’re the only man alive, but you’re the fastest man alive by definition. And then he has to use his second wish to undo the first wish. A lot. A lot of building product is like this, right? We think we’re aligned. I go and I say, I want you to get more engagement.

And then about six weeks in, we realize, hey, my meaning of engagement and your meaning of engagement are dramatically different things, right? And so this drift has occurred, and even if we’re just misaligned, what happens over time? We get farther and farther misaligned. So all the behavioral statement is trying to do.

By agreeing, we’re what will be what behavior, who will be doing what in the world. Observably, when we’re done is really just so that we can take all of those arguments that we’re gonna have later and move ’em up here at the beginning. I wanna fight at the beginning and then never again. It’s not conflict avoidance, it’s just conflict concentration, right?

So now we’ve got a target audience. When, people who, can physically, literally get in the back of a Uber and have a smartphone and can, pay electronically and all those things want to motivation. Motivation is a special kind of limitation. It’s the zero one thing that the person has to wanna do in order to engage with the service.

We talked about this earlier, right? They have to want to go from point A to point B, right? Nobody is thinking about Uber right now because nobody wants to go from point A to point B ’cause they’re chilling out watching this, watching this fireside chat. Our engaging with the service only exists when the motivation exists.

That zero one thing. Now again, this is different than a pressure, right? Pressures can vary between people. Being in a hurry might make me choose Uber over the tube, but that can vary between people can vary at times, et cetera. This is the zero one thing. I either have to wanna go somewhere, or I have or I don’t, and they will behavior.

They have to wanna do something, right? Or you want them to do something. In this case, it’s usually take an Uber. So when people who, all those things want to go somewhere, they will take an Uber. Now, there’s an important ethical check here that we have to do, which is that motivation and behavior need to match, right?

Motivation and behavior need to match. What do if I prompt you with the behavior, you should feed me back the motivation. And if I prompt you with the motivation, you’d feed me with the behavior. So Adam went to the doctor last week. I said to Adam, Hey man, you went to the doctor last week.

How’d you do that? He said, I took an Uber. Oh, I wanna go to the doctor. I prompt him with the motivation. How’d you do that? I took an Uber. I gave him the motivation. He gave him the behavior. Vice versa. Adam? Last week he took an Uber. Why’d you do that? Oh, I had to go to the doctor.

Okay, great. I gave him the behavior. He prompted back the motivation. That’s good. Now we’re ethical. If I said to Adam, Hey, last week you could took an Uber, why’d you do that? And he said, ah, I fancy the driver. I thought I’d get in and flirt with her. And drive around a little bit.

Oh. The reason that Adam thinks he’s doing the behavior and the reason I think he’s doing the behavior are different. That’s a big red, hit the stop button kind of moment where you have to make sure that those are still logistic because if they don’t reply to each other, if they’re not going back and forth right.

Then potentially what we’re trying to do is make people do something that they don’t have an intention to do, which is unethical. So you always wanna make sure those match. And then the last thing we need is as an as measured by any good product person. It has to be an observable behavior.

When we say take an Uber, what’s one way of measuring the behavior that is taking Uber? Stick it in chat. How are we gonna measure the behavior that is Uber taking

number of rides, right? Bookings payment. Okay. We can do total revenue, right? Again, remember behavioral statements. There’s no right or wrong, but imagine that each of these person was the c e o of Uber and you’re like trying to receive, behavioral directions from them. If you say hey Matt, I want you to maximize as measured by revenue, right?

Total money. Lemme tell you about my friend Tom. Tom lives in an Uber. He spends a thousand dollars a month with us. So I’m, if you tell me, get total revenue, I’m gonna go after Tom. Tom spends a thousand dollars with me. I’m gonna try and get that thousand in first dollar, right? Because my friend Jorge, he is never, ever used an Uber, right?

And so getting him to take his first one and gimme his first dollar, that’s gonna be really expensive. So if you say, Hey Matt, I want total revenue, what you’re gonna end up with is a niche app that gets a lot of money out of a few people usually, right? Whereas if you said, Hey, I want Jorge said, total number of rides, same thing applies, right?

The person who lives in the back of the Uber is gonna be easier to get that first. That thousand in first ride than trying to get a Jorge who’s never taken one before, right? But what if you had said total number of riders? So total number of people who have ridden at least twice in the last 30 days are active user metric, right?

Now I’m gonna build something totally different. ’cause now I have to go after Jorge, right? The person who’s riding 1,001 times is already past that metric. He’s already a one. I don’t need to go to a zero. I have to get Jorge who’s a zero into that one column. So what you ask, how you measure this behavior will change dramatically what people build.

Does this make sense? Once we have a good understanding of this behavioral statement, right? Once we understand at some level, hey, here’s what we want. And this is usually, as I said, I want conflict here. I want, when I’m do, when I’m taking this handoff from the business, I want there to be productive discussion.

I want them to argue with each other. I want to argue with them about Hey, do you really mean this? Do you really mean that? Is this really the right metric? Because in a sort of Mary poppin sort of way, begun is half done, right? If you can get this part right, the rest of the process will go much easier.

So remember I said side strategy, insights, design, evaluation. So now we need to move to insights. Now, this is a place where I think a lot of product teams struggle. There’s a ton of. User research that is usability research. That’s over an operational, right? I don’t even care about that. And then there’s a ton of user research where people spend six months and then they come back with these sort of like personas or vague insights like that.

I don’t really know how to translate into a product anyway. And it took six months and I’ve already built the product by then ’cause nobody’s gonna wait that long. So when I say insights, it’s the first thing to recognize is I’m a lying liar. Which, some of you knew when you invited me. Why did I invite him on this show?

He is a lying liar. What am I lying about? We don’t actually change behavior, right? Product managers don’t actually change behavior. Things we behave, that we build don’t actually change behavior. What they do is change pressures, and pressures change behavior. All behavior is the result of competition between promoting pressures, reasons to do something, inhibiting pressures, reasons not to do things that make a behavior less likely.

When I want a behavior to happen more, I can strengthen promoting pressures I can make, you wanna do it more, or I can weaken inhibiting pressures, right? I can make it easier to do this sort of notion of friction. And so when we are doing insights work, what we’re really trying to figure out is what is that existing balance of pressures right in the world today?

Because then I try to modify that the way I normally do that, right? We wanna make mixture of quantum qual, right? Because I want, I always think of insights as like a table of truth, right? So if I interview one person, I. That’s one leg at one corner of the table. And as we all know from physics class A one-legged, one-corner table goes and falls over and I can’t build shit on top of that ’cause it’s not stable at all.

So I need to A, add more legs and B, move ’em farther apart. So how do I do that? One, I interview more than one person, but if I interview a whole bunch of white dudes, right? That’s like just putting a bunch more legs at one end of the, one corner of the table. That doesn’t help me. I need diversity of who I’m looking at both in, I need diversity in terms of methodology, quant versus qual, right?

And using, cross-validation to get me there. Qual is very good at why quant is very good at what, right? So I need those working together to get me to this stable table of truth. But I need them to have shared definitions. This is why I hate personas, right? The qual people love to make personas and they’re like, let me tell you about Jorge.

He lives in New York City. He has two kids. They have this whole vignette of this person, right? And then the data science person’s going. How the, what am I supposed to do with that, right? I can’t make a column in which I assign people to the Jorge, persona because you’ve given me some instream vignette.

So there’s only five categories we actually care about. Only five categories we actually care about. Always, never, sometimes started, stopped. Only always, never, sometimes started, stopped. Tom, I’ve picked on you a bunch. What is Tom, what is your favorite sport to play? Physical activity. You like hiking, you play soccer.

Like what is your favorite physical activity to do? Tom? Put it in the chat.

Me, Tom, I’m assuming you, Tom. I think you’re the only Tom running great. The great thing about running is that everybody in the world fits into one of five behavioral categories with respect to running. They either are often running, never running, sometimes running. Are running more than they used to or are running less than they used to.

And these, we can make quantitative definitions around those, right? We can say, always runners run at least three times a month for 30 minutes. Never runners run, less than, sometimes runners run, some months they do, some months they don’t. Right? People, I can look over trailing average the last three months and see if it’s risen to get you in started.

If it’s gone down, then you’re in stop. I can get into these very, and that allows quad qual to work together. So now what I’m starting to do is have conversations about those things, right? So I could run a poll and everybody in here fits into one of those five groups and I could start to talk to people.

Let’s say let’s say that David says, oh, I’m in the stop group. And I say, oh, stop. You know what happened? And he says, oh, I hurt my knee, right? And so now it’s painful. And so I’m just taking a break. Oh, that’s a str. He had a change in inhibiting pressure. He didn’t say It’s not as fun as it used to be.

He said, ah, it hurts now. So it didn’t change his motivation to do it. It didn’t change his promoting pressures. He had strong change in pro in inhibiting pressure, but you could imagine a world where he said something different, right? David comes to me and he says, I used to do it with my friend Matt, and Matt died and now it’s just not really very fu you know, I was really doing it to spend time with Matt.

And so now I don’t really have the promoting pressure to do this. It’s not that there’s new inhibiting pressure. It’s not any harder than it used to be. My knees used to hurt. They still hurt, right? It’s that I just, the things that made it attractive to me are no longer as attractive. I think about this all the time because I have a seven year old, so what I do, my physical activity is highly bound by whatever the hell he’s interested in that week, right?

That has nothing to do with my personal, it’s not that my knee hurts or I can’t do this thing, or we don’t have a soccer ball anymore. It’s whatever the hell he was interested in that week changed my behavior, and that’s really what we’re doing in product. We’re creating unnatural variation. I have natural variation, right?

My son gets interested or disinterested in things and that changes my behavior. Products create unnatural variation, right? They lean in on either the promoting or inhibiting pressures to get me there. Now we have a systematic bias. One of the things that’s really important about. Science is that it’s a de-biasing process, right?

Even writing a behavioral statement forces us to acknowledge. I loved by the way that, that somebody put in accessibility. Not everybody can get into the back of Uber. Good job, David. Sometimes that one doesn’t come up. Why? Because most people can get in the back of an Uber, so they don’t think about that.

Most people on this call probably can get in the back of an Uber, have a smartphone, live somewhere that Uber will service, except whoever’s in Arkansas have an electronic form of payment, have a smartphone, all of those things, right? Like most of things, those things are true for most of us, and so we don’t think about them, right?

Limitations helps us call those out, right? It’s that de-biasing. So in the insights phase, we have this natural tendency, which is if I say to you, Hey, get people to do something more, you will automatically gravitate towards promoting pressure. We’ll make it more attractive. We’ll run a campaign, we’ll do all sorts of, like spicy things, animations, whatever.

And if I want people to do something less, I automatically think about inhibiting pressures, right? If I want people to stop smoking, people are like we’ll put big fuck off warning labels on them and we’ll, tell people it kills them and we’ll make them really expensive and moving them hard to get, it’ll make you have to go 20 feet from the door in order to smoke.

We’ll just make it really hard. But the reality is, that’s only half of the equation. If I want people to do, I want you to go on a thought experiment with me right now. Normally when I do this exercise, if we had more time, I’d have you generate pressures around m and ms. And one of the things you’ll notice about m and MSS is I can change your behavior without changing the promoting pressures at all.

M and MSS is very focused on promoting pressures. Mars is like new flavors ad campaigns, right? This is their standard way of getting us to eat m and ms. But in reality, why I can get you to eat m and ms simply by doing the following, putting a bowl of them in front of you. I didn’t change the promoting pressures at all.

They’re exactly as delicious and colorful and chloric and well advertised and all the other things. But if I just snap my fingers and put bowls of m and ms in front of you, I know my friends Stef Stephen or Stefan, I don’t know. This is a problem with Chad. You never know how to say people’s names. Senior Kincaid.

Senior Kincaid would be eating some m and ms right now, right? If I snap my fingers and put a bowl in my, he would be with me, right? But I didn’t change anything about their attractive properties, right? I just changed their inhibiting pressures. It’s actually the secret to Uber. Uber is not a better experience from a promoting pressure, right?

Perspective. I went from riding around in the back of a black car to riding around in the back of some guy’s to cell, right? Like it is not a better experience, but radical reduction of inhibiting pressure, right? I don’t have to sit there and try and pay, right? It’s very cheap. It’s like very convenient.

All of those are inhibiting pressure based things. This is where you can work with your marketing colleagues, right? If you think about Uber early marketing, They never did any marketing that wasn’t inhibited. It’s not entirely true. Every once in a while they do a promoting pressure thing. Oh, today it’s a Tesla or it’s, comes with a puppy or has popsicles or something, right?

They do some novel promoting pressure, but the majority of the time their marketing emails were three things. It’s cheaper than it used to be. There are more drivers on the road, so you’ll get one faster. Or we can go somewhere. We couldn’t go before. We couldn’t go to the airport. Now we can go to the airport.

We couldn’t cross this bridge. Now we can cross this bridge. They were really hyper-focused on inhibiting pressures and there’s an imaginary world that’s different than that. So they actually, there was another startup so Uber started in black cars, right? Picking up from the airport primarily as a service.

There was another startup at the same time called Silver Car, and their value proposition was instead of a kind of broken down black Lincoln, it’s gonna be a brand new silver Audi. You’re gonna be picked up in a brand new silver Audi, right? Very promoting pressure based. No, not, they’re not the Ubers, lemme tell you, they did not conquer the world with silver car.

Why? Because nobody cares, right? I don’t care how nice my car is. That’s not enough of a promoting pressure to change things. I would magically, I would care way more about those inhibiting pressures. So we can’t start to design for pressure change if we’re not accurately mapping those pressures.

Quantum qual. Five, five behavioral categories. Now we’re to the D, right? This is the part that product managers often think of as product. When I do product, I do design. And what I mean by that is I take these pressures and I start to create interventions around them. And this is probably the key step that gets missed, is even when you do have good research, most people ignore it, right?

They use it as what we call contextual research or foundational research. Meaning I look at these personas and then I’m like, oh, I understand cancer better than I did before so that I can build a better cancer app. We don’t want that. We wanna directly translate. The pressures that we found into the interventions that we build.

So when I suggest an intervention, when I say, Hey, here’s a potential thing that we could do, I’m not saying I think this will change behavior. I’m saying, I think this will change this specific pressure, right? So if we take eating m and ms and one of the promoting pressures is fun, hey, the thing I’m suggesting is I think this is gonna work because it increases fun, right?

I’m connecting the pressure to the intervention, right? David I’ll get you in a second, right? I’m connecting. Every intervention has to be clearly connected to pressures. So when my team used to pitch me on interventions, so they’d work through the week and then Friday would be the sort of review time.

So they’d come to this thing, they’d have all these interventions and every time, right? Like I would just, if, if you couldn’t tell me what pressures this came from, if couldn’t clearly link it back to our understanding and those pressures, by the way, They should have quantitative and qualitative evidence.

So I should be able, when you suggest, I think we should build fixture X. And I say, what pressure is that based on? You say it’s gonna increase the promoting pressure. That’s fun. And here’s the evidence that fun is a thing here, right? Here’s a user quote, here’s the data, right? That shows us that fun is an important pressure.

Hey, I’m gonna have a hell of a lot more confidence saying, yeah, that’s something we might wanna do, but we’re not done. A lot of times this is where things stop. People are like, ah, seems like a good idea. And the highest paid person in the room who’s often white and often male says, eh, that, and then we build that and that’s a stupid, terrible idea because you’re then you’re just have SSIDs, right?

You’ve done strategy, insight, design, but you now you’re just ssid. Nobody wants to be ssid. Sid’s a sloth, right? You want an E, you need some sort of evaluation, some sort of pilot Now, Almost inevitably, the pilots that I see are too big, too rich, too expensive. If you laid any code, you failed. They should.

All pilots should be code free, right? I often use, I tell my team, they have to be quick, easy, and cheap. Quick means four weeks or less. Easy means you can’t go outside of our team to run it. So you can’t ask the engineers to build anything. You can’t ask the mail room to send anything. We have to do ourselves and then cheap.

It has to be less than a thousand dollars, right? As an example, if we’re gonna test a new notification feature, I’m not gonna go ask the engineers to build new notifications. I’m gonna send people emails myself, right? Get a, I’ll create a fake email box so that I don’t screw up all of our workflows.

And then I will send out emails from this thing. I’ll send out, you want to test a new, you wanna test? Should we add analytics to the product? Make some analytic reports by hand and send them to people by email and see if you get engagement. I gotta give you, we’re not trying to, now, none of this is usability testing.

This isn’t, how should we build it? This is it even a vaguely good idea, right? Is there even a spark here? You need that level of test. And you could do this across anything. It doesn’t have to be digital projects. I remember AB InBev got together some of their like young, bright, talented people and they wanna do an innovation workshop.

And so we came and talked about this and someone was like we can’t really do this at AB InBev, right? Because how do I like pilot test a new beer? And I was like tell me what you think about it. He says I’ve been working on a project about a new beer for the Mexican American market.

I’m like, cool. Go get some bottles of beer. Take ’em home. Scrub the labels off, print out some new labels, like put new labels on and go sell them in front of your local bodega, right? Like you can test. Is this branding gonna be a Mexican American friendly, beer? We don’t have to make new beer for that, right?

We just have to, try out the brand. And there are a million ways to do that. Make a Burner brand. We do this all the time. One page website, throw up some Google ads, test a bunch of messages on the Google ads, so that we don’t pollute our own product. See if there’s any appetite. Hey, does that make people click on it?

Are people even interested in that at all? I promise I’m gonna get to these questions, so keep throwing questions in the chat. I’m gonna finish my soliloquy and then we’ll do questions. So this is the place where, you know, and this is a place where there’s insufficient training. So the thing I’ll say is Most of you have probably never had any formal training in experimentation.

Go take an experimental methods class to your local community college. Go pick up an experimental methods books. Scientists have got this shit figured out. Social psychologists do experiments all day every day. There are like whole disciplines around experimental design. You should figure that out, right?

That is a key skill. It is. One of the saddest things to me is every time I go on LinkedIn and I put it in the keyword experimentation, and I look at there’s a thousand jobs with the keyword experimentation and a hundred thousand jobs with the, with like data informed or like evidence backed or whatever.

I’m like, how the fuck do you think that you’re gonna be able to figure out how to be evidence backed if you don’t know how to plan and build an experiment? Like analytic capabilities are garbage compared to the ability to actually test something. So go and get some training along all of these.

I think, you wanna think of people as being very T-shaped, right? And so usually when I build my teams, people have either an expertise and strategy, insights, design or evaluation. But they can reach into, the things if you are product leaders, we used to do something on Fridays called Tea Time.

And so we would have somebody give a lunchtime talk on their legs, but at an arms level. So let’s say I was a data scientist, I would say, I’m gonna teach you how correlation works, right? So when I say 0.5 is 0.5 a strong correlation or a weak correlation? Most people can’t answer that question. You could put in the chat, I won’t say what it is.

You could put in the chat is 0.5 a strong correlation or a weak correlation? And so they just taught a like kind of a lunchtime lesson on what correlations mean, how we get them, what strong and weak looks like, how long, what different things are. Now that at least one person put in Robert gets points for being brave and loses points and gets extra points for being wrong because it’s always good to be wrong and fail forward.

Point five is actually relatively well correlated, right? That means 0.0 means two things are completely uncorrelated. As an example height and gender are about 0.7, right? So that I am a man and thus taller is about 0.7. Point seven is an insanely strong correlation. One means things are perfectly correlated, right?

Perfectly correlated means when a always b. Most of the things that you’re working on as a product person are like maybe in the 0.1 range, 0.2 range, right? When I change this color, maybe I’m getting a little tiny bit of a bump. So 0.5, somebody found 0.5, I’d be like, holy shit man, we should definitely build that thing immediately.

So it’s that kind of cross training that you want. And the reason we call it t time is, so we’d have a, an arms talk, right? That broaden everybody’s arms and then they’d break off. So all the data scientists would go talk about some data science thing that I didn’t care about, right? That was in their legs.

How do you do multiple hierarchical regression in our, I don’t care, right? But they did. And so they would have an hour where they would talk about deep things. You want that cross training. You wanna be broadening. I always tell my teams, your legs keep you grounded. Your arms help you fly. So you want that cross training, that your cross strategy, insights, design, evaluation.

Let’s see here. What, let’s flip to questions and then you and I can chat. No. Can you share some good. 

[00:35:49] Janna Bastow: We I can help with the questions here. So we had a question that that came from Catherine here. So she asked, do we have to test every new feature? Some don’t. Product managers sometimes know just what the customers want, if they know the customers well enough or if they themselves are customers.

[00:36:05] Mall Wallaert: So being they themselves, the customers is the biggest trap you’ll ever fall into, right? Everyone is always convinced all the time that they know shit. And knowing the customers well enough, my naive answer, so here’s my real answer, Catherine, you are testing it, whether you know it or not.

The question is just whether you’re testing it at scale. If you don’t run a pilot first and you just scale it, all you’re doing is saying I’m gonna run that test at massive scale. And maybe you don’t give a shit. Maybe it doesn’t matter. Maybe if it fails, oh but if you’re building features that actually matter, if they’re driving some revenue thing and you are wrong and you’re doing it at scale, you could lose a hell of a lot of money really fucking fast.

If you are wrong. Yes I would argue for testing every new feature ’cause you’re doing it anyway, so you might as well do it small first. 

[00:36:53] Janna Bastow: And I am living proof of that. ProdPad is living proof of that. So ProdPad was built by myself and my co-founder, Simon, when we were both product managers and we just, originally, we were building something to help us do our own jobs.

And we figured we’re product managers, so let’s build a tool that helps us do our jobs as product managers and then we’ll be able to sell it to other product managers. ’cause they do product management like us. Wrong. We built a tool that helped two product managers in total. And when we tried to put it out there for other product managers, we realized, oh, it doesn’t work for other product people.

We had to throw out months of code. Now, fortunately we hadn’t done the whole thing of raising a ton of money and having a ton of people that we had to fund pay for all this sort of stuff. We only lost time, but time is incredibly important and it was a ton of we literally threw out months of work because we didn’t bother talking to other product people.

As soon as we started talking to other product people, it was like, Mind blown. And that’s where stuff like the now, next, later roadmap came from was, oh let’s start figuring out what, how people are reacting to the first bits of what we’re showing to them. 

[00:37:54] Mall Wallaert: So yeah, I wanted to test everything David said.

Can you share some good resources? Can you start with behavioral statements? Yes. So let’s see here. I just threw in a link to the like very condensed guide. You can pick up my book Start At The End which is on Amazon and all sorts of other places and has a good guide to this. And then we are just about maybe like in two weeks.

Our head of education we’re actually testing it this week. There’ll be a self-paced educational course. I usually teach a live educational course inside of companies, but we’re doing a self-paced one that anybody can take. So that’ll come out in a couple weeks, so stay tuned. The book, the guide, and then education are probably the best pace I can give you.

And then just practice. The thing I’ll say is, All behavioral science skills benefit from just turning them over in your head, even if you don’t actually do them. So one of the things I often encourage people to do is, Hey, write a behavioral statement for a project that’s already done, right? You don’t even to worry about a new project.

Do something you already did. Can you like look at a failed project? Hey, would we have not failed if we’d been better about articulating these things upfront? One of my fav favorite things to do is to get a group of executives in the world. I really like to get the C team to do this. And I take a product feature that we launched and then I get them to, without talking to each other, write behavioral statements, who do you think this is for?

What do you think it’s gonna make them do? How are we measuring that? Do the whole thing. And then I put them all up and make them see how incredibly different than they are. And I’m like, somehow we managed to build this thing and scale it, but you don’t even agree what it’s supposed to be doing. So how can we judge the success of it?

We didn’t even agree what it was supposed to be doing. That’s why I really like starting with a failed project because I really like a bunch, get a bunch of c e o, C-level people to be like, ah, yeah, I was a failure, but you don’t even agree on what success looked like. So how the fuck can, you know if it’s a failure or not?

That’s not, 

[00:39:34] Janna Bastow: That’s entirely true. There’s a good few questions in the q and a, so I’m gonna prioritize those because it’s so much easier to to manage. But people have a question. Drop ’em into the q and a. We’ve got one here from Don Parker. 

[00:39:45] Mall Wallaert: So people to do the q and a.

There’s a q and a. There’s a separate q and a button at the bottom of Zoom. That’s not the chat. What she’s saying is use that rather than the chat in which they would get lost quickly. 

[00:39:54] Janna Bastow: Thank you for that clarification. Yeah. Don Parker asked this question. He said, how would you motivate individuals to change a 50 year old industry behavior that has known issues, but the individuals buried by this behavior don’t feel they have time to consider any type of change?

[00:40:09] Mall Wallaert: So you. What’s interesting there is motivate is a generally a promoting pressure word. And yet what you expressed is that they have an inhibiting pressure. It’s not that they don’t wanna change, it’s that they don’t think they have time, right? So you’re over here saying I’m gonna try and overwhelm these inhibiting pressures with stronger promoting pressure.

I just gotta convince them that they really need to do it. What if you made, they made them recognize it won’t take all that much time. So when I talk about this side process strategy, insights, design, evaluation, that’s an eight to 12 week sprint. Most people, when they think I describe it, they’re like, oh, that’s gonna take six months.

I’m like, no, it’s not. We’re gonna have three. We have a great Gantt chart. It’s one week for strategy. Three weeks for, three weeks for insights, two weeks for design, four weeks for you can get through it much quicker. And what is that doing that’s addressing their inhibiting pressure, Hey, this concern that it’s gonna take a really long time and that we don’t have time.

So I think being really clear about, the, one of the things I really love about science is it works at any level. So what I mean by that is, Most processes break or are meant for some zone. Oh, this works when you have five product managers, but not when you have 50. It works on a product that’s early stage, but doesn’t work later.

One of the great things about science and using science in this way is it works at any fidelity. I can do this same sort of thing at the individual level. I wanna work, I can write a behavioral statement for myself for working out. I want this thing to go differently, right? I can look at my own inhibiting and promoting pressures.

I can run experiments on myself at a unit level of one. I can also do it at a population level. I can do it for the whole fucking planet if I’m, if I’m the un, right? And I think you can think about internal behavior change in the same way you think about user behavior change. Hey, what is it that I want people to do?

So when you say change a 50 year old industry behavior that has known issues, can you define that? Can you measure it? How, who are we talking about changing? Write a b April statement for that Then. Somewhere someone is doing the right behavior. There are people who are always doing the right behavior and never do the right behavior.

Sometimes do the right behavior. Just start it just stopped. Go look at those people, right? Go inside of your own company and say some of the people are using this the way I intended them to. Some people aren’t. Let’s feel what’s going on. Take those insights, start to design against them. Run little tiny pilots.

Scale the ones that work, right? And you could do that for internal and behavior change. External behaviors change. Large populations, small populations, doesn’t matter. How might you approach, how might you approach change if the customer is unaware of the connection between the, oh, how might you approach change if the customer is unaware of the connection between the pressure and the behavior you’re driving?

They don’t have to be aware. Wess. Only we have to be aware. ’cause remember, we’re using quant and qual together, right? And so we can observe their actual behavior. What I’m not asking, everybody lies in the words of house, right? Like we are very poor at understanding our own.

Like motivations and things, right? I’m not suggesting that you just go run a survey and say, why do you do this thing and then observe people. So for example, I almost never ask people about emotions or cognitions. I only ask people about behavior. So it’s a small flip. So as an example, let’s say I’m a I’m interviewing Wess.

We’ll take the first product I ever built, right? So the product we exited, the lending choose in the personal finance space. I never said things like, how do you budget? Or, like, how do you feel about your budget? How do you feel about blank? People just make up a emotional answer. What I would say is, tell me about a time that money made you feel bad.

Tell me a time about a time that a money made you feel good or made you celebrate with somebody, or that you like, that sort of, I’m gonna prompt you with the emotional cognition and then you relay that story to me and then I’m gonna understand from listening to many of those stories I.

Where those promoting and inhibiting pressures are. And then I can check those using quantitative data, right? So I can say, oh, this person said, when I wasn’t able to go home for the holidays, it really, because I couldn’t afford it, it really made me feel quite sad. And so I can then say hey, do other people feel sad during the holiday period related to their finances?

I could do that quantitatively at scale, so I could take these things that I’m hearing and then I can validate them using quant data, right? And that helps me get at those things that are often somewhat less cog, less the con the connection is less conscious for people. There are many cognitive biases that influence their behavior and then he physicalize, they’re so pervasive that Matt thinks everyone should have a deeper understanding of him.

So it’s interesting. I look at my academic background. My undergraduate advisor was Barry Schwartz, right? The paradox of choice guy, right? Like I published many academic papers about many biases. I even talked about a bias, right? The fact that we tend to focus on inhibiting pressures when we think about getting people to do something less and promoting pressures when we tend to get to think about people doing something more.

There’s, so that’s the one I want you all to be aware of as product managers is like the tendency for that to happen. There isn’t any one bias that I really, and I often try to get people not to think, I want you to try and go in to the insights phase as blank slate as you can.

It’s really only the design phase where I say, Hey, there might be an opportunity to use this bias, how people perceive things in a motivated way. That’s where I tend to start to think about those things. But there isn’t a single bias that I’m like, Hey, everybody in the world should know about that.

I do think for, if you wanna, it’s not a bias, but if you wanna psychological construct I think that like cognitive dissonance probably is maybe the one thing that like rules, people really want their behaviors and their cognitions and emotions to line up, and when they don’t, they can’t change their behaviors, so they’ll change their emotions and cognitions like that one 

[00:45:37] Janna Bastow: is, how would you define that?

How, we’ve all heard of cognitive dissonance, I think, but how would you define that really? Cognitive, the, 

[00:45:44] Mall Wallaert: the thing that layman’s thing that I think you should, that try and remember is people really want what they do and what they think to line up and they can’t change what they do ’cause it already happened, so they’ll change their thoughts.

And if you get. Somebody to do something that is contra how they feel about something. They typically will change how they feel about something, right? Because they can’t change behavior, right? It has to be, they have to feel like they made the choice freely. And there’s a bunch of cons, constraints on that.

But I do think it, it drives a huge amount of behavior. Favorite brand of cowboy boots? I, so my pro tip for cowboy boots is they are very painful and literally painful to, to break in. And so the key is to get them used off eBay. ’cause lots of people have bought cowboy boots and gone, ah shit, I don’t wanna break these in.

And so they’ve broken ’em in for you, but now they don’t wear them. Lots of people bought very expensive cowboy boots. When they were drunk in Austin. And so get ’em used and get ’em slightly worn. I mostly wear TSS these days. I have a couple pair of juice Justin’s that I like. But I tend to wear one pair and then wear them literally to death and then move on to the next pair.

Like I have a pair of black and a pair of brown and then, once they die, I move on. Dom Don, I too love. I love I too love house. All right. Some features, maybe the, apprentice users are small, but maybe benefit a smaller group of people should we still build it? So Catherine, great question.

So when we’re looking at so this starts this, what is the sort of cost benefit ratio Starts all the way at the very beginning, right? Because we talked about a target audience. So I’m making an agreement upfront with whoever my stakeholder is about the size of my audience because we defined it in the very, very beginning, right?

Then in the insights phase, I generally ask my researchers to, to rank insights by three factors, right? One is how, how confident are we in this? Do we hear from lots of people? Did we see it in both qual and quant data? Like how confident are we in this insight? So if only one person told me one time, I would have low confidence, right?

If it was very well cross validated, cross quant, qual confidence would be really high. The second was impact. So maybe lots of people mentioned price, but it didn’t really seem like the thing that was changing their behavior, right? So price, it was high confidence, low impact, right? Versus, hey, I only heard this from Don, but it was the number one thing that changed his behavior.

That would be low confidence, high impact. And then the last one is the subjective one, which is likelihood to change. Meaning in the scope of this project, do we think we can do anything about it? Maybe everybody mentions price, but the thing that we’re working on, we just, the price lever’s kind of stuck because it’s set over here and we can’t change it in this product, in this project.

And then that carries through. So when I look at. When I evaluate and experiment, so what we’re come out with is this thing I’ll put in the chat. This is an outcome statement. We have some level of confidence that doing this thing will change direction, will change behavior in this way.

Here’s how much it’ll cost to scale, and here’s how big the change you’ll get. Because I wanna be clear, as behavioral scientist, I don’t make decisions. I make the menu that other people order from. So I say, here are 10 interventions that we tried, seven of them worked. Here’s the cost benefit ratio of those seven.

And then they can make a business decision about that. So let’s say you’re at eBay. I can go to the stakeholders and I can say, Hey, I have this thing. It’s only gonna affect a few people, but they’ll buy 10 times as much. Then I have this other thing, it’ll affect everybody, but it’s like a 1% change.

That’s a business decision. That’s not a Matt Wallaert decision. That’s a business decision about maybe as a business we really need 10 X over here in this one place, or maybe we need one X across a large number of people. Who knows? I don’t fucking know. That’s a business problem. So my job is to set up the menu and inform those decisions so that they can then order from the menu and I can go to operations and say, great, here’s the thing.

We’re gonna start, here’s the pilot version of this. Go forth and build. 

[00:49:23] Janna Bastow: So it sounds like this behavioral science is going to help lead to much more impactful product decisions, but a lot of product managers that we know are stuck in feature factories. They’re being asked to just build, they’re stuck in that design part and very much in the design stuff to just get built.

How do you recommend product people make the case for thinking more like you do, making the case for spending time in behavioral science? 

[00:49:51] Mall Wallaert: So I’m typing the answer to Katherine. How do you make the case for these things? I don’t, so you don’t have to. The great thing about behavioral science, you don’t have to permission, ask permission to do it.

You just start doing it and then you start coming back and you convince them through the value of the, of actually having done it right. I think too often people are like, oh, we want to do it this different way. And they think they need other people who bought in. No, you don’t. You can just start doing it.

You need permission. 

[00:50:13] Janna Bastow: Excellent. I love it. Do something small, prove that it worked, and then show the results. Right? 

[00:50:18] Mall Wallaert: So Catherine, this is this is fantastic. Catherine and I are in a debate right now. She’s I thought PMs decided what to build. No. PMs do not decide what to build. They might, if you are a T P M, you might decide how to build it, right?

And if you are a sort of design PM you might design you are creating the menu, right? You are giving them the available options, right? So you are making the decision in the sense that you’re saying, Hey, I’ve decided these are the 10 things we’re gonna try and here’s the evidence to do a thing.

But nobody’s really making the decision other than technically AI could make the decision, like we could do it algorithmically. If at the very beginning you said, I’m gonna do the thing that is most mathematically impactful, largest number of people, times largest number of change times revenue. I’m gonna do that one.

I could set up an algorithm at the beginning, and then I run a bunch of experiments, right? I run four or five pilots and I say, here’s the one that according to the pilot, will get you that outcome, right? The evidence makes the decision, and if the evidence is ambiguous, then whoever the hell has the accountability, which is usually some stakeholder, CMO CEO, whoever owns revenue or whatever it is, the behavior they’re trying to drive, that’s who makes that decision.

So many good questions. We didn’t even get to them all. 

[00:51:32] Janna Bastow: There were some great questions. Thank you for tackling all these questions today. This is a a great learning session here and thank you everybody in the audience for jumping in with your questions. So with that, I wanna say a huge thank you to Matt. You were absolutely brilliant. Really enjoyed the conversation today. Really enjoyed your answers to all the questions today.

And thank you everybody for all your questions and your attention today. This has been great fun. Thank you everybody, and I’ll see you again. 

[00:51:58] Mall Wallaert: Take care 

[00:51:58] Janna Bastow: Bye for now.