Skip to main content

What are Analytics and Experimentation?

Analytics and metrics are used to discover whether changes you’ve delivered are of value to users. Experimentation is a continual process of testing out hypotheses for ways to make your product better. This scientific approach to product management is geared towards understanding how people use your products, and identifying ways to improve them.

Where do Analytics and Experimentation fit into the Product Management Process?

There are 3 main stages in the product management cycle where you should bring in analytics and experimentation.

  • When building up product specifications, you should set KPIs for how you’re going to measure whether the proposed change to your product has been a success.
  • When you’ve delivered a new change, you should measure its success against those KPIs. New changes should be introduced as an experiment to prove or disprove the theory of how they will transform the user experience.
  • You should continually carry out experimentation to understand how users interact with your products to identify any problems that you can improve upon

Analytics and Experimentation Best Practices

Good analytics and metrics for product management should be quantitative, comparable and actionable. When reviewing your KPIs, ask yourself:

  • Is your metric a number? Can you calculate and measure it?
  • Can you compare your metric across different experiments and over time?
  • Will you be able to make a decision and take a decision based on the results you collect?

Here are some examples of some specific metrics you could consider integrating into your analysis and experimentation.

  • Acquire: Did your users visit and arrive as intended?
  • Engage: Did your users interact with and enjoy the change?
  • Retain: Did your users come back?
  • Grow: Can you track that your users spread the word of your product?
  • Revenue: Did the change generate $$?
Up next – Customer Feedback