Your Product Roadmap is a Tool for Experimenting

Some people still think about the product roadmap as a planning tool. Old school roadmaps read like a list of features that you could or even will build.

I’ll be honest with you: this is not a good practice. (In fact, it’s #5 on my list of roadmap things to avoid.)

The roadmap is not a list of things that you will do. It’s a list of things that you could do.

More than anything, it’s an experimentation tool – a place to lay out the solutions that you can test, in order to address the most important problems facing your company.

Once you’ve identified the main problems according to the business’ OKRs, then you brainstorm potential solutions. There are usually many, varied paths toward your goal. Each problem could have 5 to 10 potential solutions attached to it. Narrowing them down and selecting the best one requires experimentation.

The product roadmap records what these experiments are, how they’re going, and what’s been learned.

All of your ideas are hypotheses

All of your ideas are hypotheses, and as a PM, your job is actually to invalidate most of them! That said, product ideas don’t pop out of the blue, and your hypotheses aren’t created in a vacuum.

Product ideas are derived from OKRs, top-level objectives that are defined by the business. The product team sets out to solve a problem around churn, retention, revenue, etc. For example, as a PM you can say, “Okay, the problem we’re tackling this month is — we need to get conversion rates up, and the issue is around our sign-in flow area. We need to move people into the purchase flow. How can we do that?”

From there, you might brainstorm three to ten different ideas or potential solutions. Some will succeed and some will fail – that’s a fact. But you don’t know which ones until you test them. And to test them effectively, you need hypotheses.

How to phrase a product hypothesis: If we [try this idea], it could solve [this problem] and it could uplift [this metric] by [N].

Cool, that’s the experiment to run! Then you see how the new feature performs and take steps from there. But while we’re here, a quick note about new features…

What counts as a potential solution? Think wider.

Product solutions aren’t necessarily new features. What about changing a feature – or even removing a feature? Either of these might improve usability or impact other metrics and objectives that you’re working toward.

Some solutions might not involve code at all! The trick could be changing the price or the packaging or the value proposition. Shift how you talk about a benefit on the homepage or rethink how you present certain information to the customer. Adjustments like these can transform the way people interact with your product – which, in turn, could solve some of the problems on your product roadmap.

The point is: experimentation doesn’t always require the development team. Instead, it might require marketing, sales, customer success, and support. PMs should take a step back and think more holistically about what it is they’re actually solving so that the rest of the team can be involved as well.

What counts as a product roadmap experiment? Think leaner.

Again, by “experiment” I don’t necessarily mean build full-on code. Experiments can also be fast litmus tests to check what’s viable or not before devoting any real resources to it. There are many ways to do customer decision discovery, do quick prototypes, and put in place little tests. It could even be a prototype on paper that you test with select customers. 

As you do these tests, you identify the experiment options that you think are going to be easy, the low-effort, high-impact ones, and float those to the top. Work through the list and see how close you come to solving the problem. 80% is close enough. Move on.

Most importantly, record everything.

The product roadmap is a record of learning

As a product team, one of the most important things you can do is log your decisions – for future you and future team members who join. This starts with each of your tests.

Logging experiments in the roadmap

You can use the product roadmap to display how you’re actually progressing with experiments, such as:

• This one is something that we’re still discovering. 
• This one shows promise, and we’re going to do some design prototyping. 
• This one failed.
• This one was successful.

But for any idea that you have, which is to say any experiment that you’ll run, you should also log your target outcomes along with the actual results. This is crucial for understanding the solution’s viability at the moment, and it also helps prevent repeating any work (or mistakes) down the line.

Let’s say there’s a delta between expected and actual outcomes, such as, “We thought Idea X would get us 100 new users each month, but it actually only brought in 20.” Well, obviously you won’t move forward with Idea X right now. And this result is recorded in case the idea comes up again.

How to preserve what you’ve learned

ProdPad is basically a log of all the decisions made and all the lessons learned over time. In addition to your working product roadmap, which is in the Now-Next-Later format, you also have a “completed roadmap.” Any initiatives that you’ve completed are shown, in reverse chronological order, with all the experiments that were attached and how well you did with them.

After all, a product manager’s impact isn’t just features that they’ve built. It’s also the problems they’ve tried to solve and the experiments they’ve tried to run, whether they worked or not. Designers have Behance, developers have Github. What do PMs have? That’s one reason we created Prodpad.

Set the stage for product roadmap experimentation in your team

Want to encourage this approach among your colleagues? Shifting to experimentation mode requires changing the language that you use. Here are a couple of ways to help your team embrace experimentation:

1. Continuously reframe conversations that become too feature-focused or committed to one solution. An excellent reframe is to ask: “What problem are we trying to solve?”

Ask these sorts of questions and give them time to think things through. Give them space to talk through how they might solve a problem. This requires vulnerability and some psychological safety among the team, so they can let go of needing to be right. This brings us to…

2. Talk about ideas as “hypotheses” or even “bets.” This invites people to get into that scientific, almost playful mindset.

Failed hypotheses and lost bets don’t really exist in product management, because you’re always learning something from the experiment. Besides, you aren’t trying to prove your idea right. You don’t have the answer, and no one does! You’re trying to find answers as a team.