8 Considerations When Building an AI Product

AI. Artificial Intelligence. Generative AI. GPT. Machine Learning. All terms that have snuck out of our tech offices, Zoom calls and workflows and burst onto the mainstream scene, splashing themselves all over the media. 

Of course, us product peeps know that AI isn’t new. AI products and features have been around for years now, but where they were once part of niche offerings, now they are finding their way into much broader, general, consumer and business software products. 

So, the chances are, as a product manager in software right now, you’re either working on a new AI product from scratch, or adding AI to an existing product. Either way, as an AI product manager, or someone involved in the product management process, you’ll need to consider a number of unique things in order to be successful.

Here is my list of the most important considerations you’ll need to keep front-of-mind as you manage any AI product or feature. This list has come from first-hand experience. 

Here at ProdPad we’ve been using AI for a good number of years now – we’ve been using natural language processing (NLP) since 2018 to power our similarity matching, helping product managers to cut their backlog refinement time right down by automatically surfacing duplicate ideas and linking customer feedback to the items in your product backlog. Right through to the present day where we’ve built the most advanced suite of AI tools of any product management platform, helping PMs eliminate the grunt work and benefit from intelligent coaching and guidance as they work.   

So I’ve been through the mill when it comes to building AI features. I’ve learnt a lot and had a load of fun doing it! I’ve also spent a lot of time talking to other product people, engineers and developers and learning from their experiences with AI. 

All of that has brought me here – to this list of the key considerations when building an AI product. Let’s dive in…

1. Avoid gimmicks

First and foremost, what you are building has to solve an actual problem for your users and customers. Don’t make the mistake of creating an AI product or feature that is just a gimmick. A gimmick might get clicks but that is all it will get. No subscribers, no sales. As with all product management, putting lipstick on a pig isn’t going to help you and will likely only cause problems for you longer term.

Do not run off and start building an AI feature because the boss is shouting ‘why don’t we have AI??’. Make sure you stick to your product management principles and first do proper user research to discover the pain points your customers have, THEN think about how AI could solve them.

AI can and does solve real problems. But, the reality is the types of problems AI is best at solving are often fairly boring problems. It’s rarely sexy stuff. However, those slightly boring, everyday problems are the problems people will pay you to solve for them. It’s making the boring stuff go away that will please people the most. See consideration 6 for more on this. 

Often the driver for the AI gimmick is marketing or funding considerations. However, as people are getting used to and understand more about how AI can help them, these marketing and funding gimmicks will backfire.

2. Understand the ‘magic valley’

Have you come across the concept of the uncanny valley before? It was introduced by robotics professor Masahiro Mori back in 1970. When plotting the human emotional reaction to robots, the uncanny valley is the dip in the graph where the emotional response becomes negative, correlating with the point at which the likeness to humans becomes too great. In short, we get a bit uneasy and a little repulsed (aka freaked out) when things are too human-like. 

And just as humans have this uncanny valley in robotics and human representations, there is a similar valley for the “magic” of AI. Knowing how your users and stakeholders will respond to different amounts of magic is important to the success of your AI powered product and features. You don’t want to weird anyone out! 

The magic valley is created from the combination of level of control (or more accurately the desired level of control) and the quality of the results. As each feature/product is different, you’ll need to experiment with different levels of control and results in order to get the level of magic right.

Too much control can be just as bad as too little. Most people won’t be sufficiently proficient on fine tuning/tweaking to produce good results on their own. Nor should they. Your users should be focused on using AI products and features to be more effective, not twiddling knobs to get good results. 

3. Set expectations properly

It can be tempting with AI products and features to sell the universe. Don’t. Like everything else in the world, AI is not perfect and as news headlines attest to, often gets it wrong. By selling the dream that AI will wash your dishes, clean the house and fold laundry while doing your job, you will disappoint and cause disillusionment with your product.

That isn’t to say you can’t sell the value, but you have to calibrate the sell to the actual value you are delivering and the problem you are solving. 

Let’s consider an example. There are an increasing number of content producing AI products that are basically selling themselves as producing Pulitzer prize level content that can be immediately published. They aren’t. These products might get you 60% or even 70% of the way there but you’ll still need to edit, revamp and otherwise improve. 

There are multiple reasons for needing to improve this content, not the least of which is dealing with hallucinations and lack of knowledge about a topic.  

What these products are doing is getting that rough first draft going. Overcoming the blank page syndrome, which is a real problem and there’s a lot of value in solving this issue.  But that isn’t necessarily sexy sales or marketing.

4. Understand the ethical and legal matters

Depending on what you are trying to do with AI, you’ll have differing levels of ethical and legal considerations. Generating user stories has a different level of ethical considerations than AI that is making a decision such as whether someone gets a mortgage or into a University.

As a product manager you’ll need to stay on top of the evolving statutory and regulatory environment. The legal aspects of the models will take years to play out in the courts in various jurisdictions. 

Be aware that you’ll probably face a very different regulatory environment in EU countries to that of the RoW even to the extent that some AI products and features maywill be unfeasible to provide in EU countries.

You should also consider the security of your customers’ data and the information they will need to feel happy with what you’re doing with it. This comes down to what AI service you are using and how you are working with it. If you’re relying on ChatGPT, for example, it’s worth familiarizing yourself with the case of Samsung. 

Samsung banned the use of ChatGPT and other AI products among its employees when it was discovered that some sensitive internal source code was uploaded to ChatGPT. Samsung feared that their sensitive code would be used to feed the ChatGPT model and could end up being shown to a ChatGPT user in the future.

Now, OpenAI actually changed the way they work back in March 2023 so that nothing going into ChatGPT or via their API endpoints now feds into the general model. But I’d say that isn’t widely known.

Certainly here at ProdPad, we found a number of our customers – especially those Enterprise customers from multinational household brands – asking for reassurances that their important strategic information was not at risk of being stored by OpenAI. Happily we had the info on hand and could reassure our customers that they are safe to use our features without the risk of having their secrets shared with the world through ChatGPT.

Therefore, especially if you’re managing a B2B product, I suggest you be ready with support materials or help center articles that explain how it all works and give customers the reassurances they may be seeking. You don’t want a lack of understanding to restrict the adoption of your AI product or features.  

5. Deterministic vs non-deterministic behavior

AI is non-deterministic, which basically means that for the same input you aren’t likely to get the same output each time you use it. It should be similar but not the same. This contrasts with other technologies which are deterministic. For the same input you get the same output each time it is used.

Being non-deterministic has a massive impact on the UX of using AI products and features. Not only in how you explain the results but also in the actual production of the results. 

A good example of this behaviour is that while you can ask GPT models to produce JSON using a specific schema, it won’t always produce the answer in the schema and you’ll need to be able handle that.

Considering that previously people could press a button and get the same result each time, the UI now also needs to prepare users that this is no longer always true with AI-powered features. Overtime people will become used to non-deterministic behaviour, but for now you’ll need to hand hold them so they can adapt to this change.

6. Biggest bang for your buck is boring

AI products and features that will be successful in the market are the ones that automate away drudgery. For example, the most used AI feature in ProdPad is our user story generator. Why? Because it automates away a lot of the drudgery of product management, taking an idea and functional definition and converting into user stories to pass to the delivery team.

The single biggest win for AI products and features is addressing boring and tedious tasks. Even if it is about just getting started.

7. Get ready to move. Fast

If you’re planning to add AI features to an existing product, chances are you’re hoping this will give you a competitive advantage. That or you’re trying to catch up with competitors who have gotten ahead of the curve and have AI offerings out in the market before you. Either way, speed is important here. 

I’ve already warned against the pitfall of the AI gimmick, so you don’t want to move so fast that you’re forgoing proper discovery and research, but you probably do need to move at a faster pace than you’ve been used to previously. 

That’s because everyone is now racing to take advantage of both the developments in AI technology and the consumer appetite for it. Soon features perceived as game-changing and unique will start to be regarded as a standard requirement. In the Kano Model view, certain AI capabilities will move from being Delighters that really impress potential customers, to Basic must-have requirements. 

Therefore, it’s worth rallying the troops before you embark on your AI quest, and getting everyone prepared for the pace you want to move at. Remember, this is all exciting and fun to work on! The team should be motivated to get stuck into this and spec, build, ship, promote and sell some truly useful features in quick succession.  

Here at ProdPad we managed to get a real whirlwind of enthusiasm going within the team and have been releasing an AI feature or major enhancement every single week. That’s how we’ve managed to have the most robust set of AI tools of any product management platform.   

8. Have Sales & Marketing ready to roll

This relates to the point above – moving fast – but it’s worth pulling out as its own consideration. That’s because it really can be the make or break of your AI success. 

You’ll want to sit down with the marketing team as early as possible. Do not leave it to the final hour to speak to marketing and tell them about something you’ve already built. If you want to do this right, you need to consider the marketing message and value proposition of the AI product or feature before you start speccing and building. 

You’ll want this in place for two reasons. Firstly it’ll mean there’s no delay when it comes time to go-to-market with your AI product, and secondly, it’ll ensure you have full alignment around the value proposition and what ends up getting built fits with the promise you want to make to the world through your messaging.  

Getting marketing heads involved in the early thinking will help you avoid the gimmick trap. This might sound surprising, after all, no one enjoys a click more than a marketer 😉, but as the protectors of the brand message, the marketing team will be really well placed to advise on whether your idea for an AI feature actually sits comfortably with the core value proposition of your overall product. If it doesn’t, there’s a high chance you’re doing AI for AI’s sake rather than using AI to further solve the problems your product vision sets out to solve. 

You’ll also want to loop Sales in sooner rather than later. This is especially important if AI hasn’t been part of your offering up to now. Your Sales team might need extra time to learn some of the basic technical details (so they can answer prospects’ questions) and think about the best way to demo the tools. 

That’s another thing we learnt during our AI experience here at ProdPad – you need to think ahead about the best way to demonstrate the features. Because AI features often rely on a degree of existing data, an empty free trial account isn’t always the best way to show the features off and let people play with them.

In our case, we opened up our sandbox environment and allowed people to play around with our AI tools with a bunch of pre-populated data. This way they can quickly see what the tools could do for them, and then, once they see the value, they can start a free trial and get started with their own data. 

Make sure you take the time early on, to plan how your Sales team can demo the features and show people the value.

If you keep those 8 things in mind as you work through your AI journey, I’m sure you’ll find success! One final thing I would always urge anyone working in the AI space to do, is try as many different AI tools as possible. You can get great ideas from seeing how different products are using AI to solve different problems. With that in mind, I invite you to start a free trial of ProdPad and take our AI tools for a spin. Let me know what you think!