The usual definition of a business metric is a quantifiable measure that is used to assess performance of your company’s products or processes over time. Both process and product performance metrics are calculated on the basis of an event or a combination of events that are tracked and measured.
What are product performance metrics?
Now think about all the events taking place in an an app or website. They can vary from minor events such as a user scrolling down a page to more critical events that represent goals within the product like a user making a purchase or downloading an app. All these events are the key to understanding if our users are responding negatively or positively to our product, or if you prefer, if they value our value proposition and design.
It can be difficult to collect and measure all these events. Depending on the product ecosystem and its scale, there could be thousands, if not millions of events taking place every minute. It would be very expensive and laborious to collect, measure and make sense of all of them. To give ourselves focus and decrease the complexity of the task, we use metrics.
What is a good product performance metric?
As a rule of thumb, a good metric is actionable. It allows you to validate or reject your hypotheses, and therefore changes the way you behave. Actionable metrics are usually comparative and understandable. For example, the Monthly Variation of the Number of Products Sold is comparative, as it compares the main value over two months, telling you whether you made progress or not, and therefore orientating your marketing and product decisions.
On the other hand, it’s not a straightforward metric, as you have to look at the two monthly absolute values to understand what’s happening. A ratio or a rate would be more understandable than absolute values, as it would show with one single rate the variation of the number of products sold between two months and give a direct idea of the performance.
Metrics that don’t allow us to react as product builders are often for pure vanity, for bragging to your friends or for making you feel good about your decisions. A dangerous aspect of these vanity metrics is that they are good at hiding reality. So how do you distinguish good metrics from bad ones? Start by asking the right questions and then choose the right metrics according to what you want to measure.
Asking the right questions to define what you want to measure
Is it really measurable?
Let’s say you’ve just launched an online shop that sells fashion brands from young designers, you might be inclined to ask your visitors if your products are pretty. Coming to a consensus about what is pretty and what is not can be difficult because individuals have different perceptions of beauty. There’s an inherent subjectivity to this question. And since we’re looking for objective answers, we must consider this question to be of poor quality.
Is it linked to your business goals?
Using the same example, you hired a marketing agency to create and launch a Facebook ads campaign about your shop. You decide to measure how much traffic the campaign generated. As an online shop, your main goal is to sell products, not to have visitors. As a result, measuring the number of visitors generated by the ads doesn’t really tell if this strategy has an impact on your main objective. On the other hand, measuring the percentage conversion rate (number of customers divided by the absolute number of visitors) associated with the ads would be more helpful to evaluate the effectiveness of your campaign.
So you do that. You measure the conversion rate of your Facebook ads, but there’s something missing. You still can’t say whether it was a good or bad decision to hire the marketing agency. What do you do?
Is it comparable?
You decide to evaluate the relative performance of the different sources of traffic to see how much more or less effective your Facebook ads campaign was. In this case, it might be interesting to compare the conversion rate of the ads with other channels to know which channel performed better. This is a good metric, as it helps you take action and make better decisions. For example, by analyzing the conversion rate of the different sources of traffic, you might realize that the conversion rate of the ads was the worst among all channels and therefore not a good decision. You decide to stop the campaigns and cancel the contract with the agency.
Does it indicate whether or not your users value your product?
Remember, by value we mean either utility and money. If you want to measure utility you have to ask yourself if your users are achieving the goal of your product, how many times and how frequently.
1-) Or how many people saw my product and then used it? Using the example of the online fashion store, the journey to conversion would look like this:
- Saw the press release
- Visited the landing page
- Selected a product
- Verified the product’s photos, price and description
- Added the product to the cart
- Paid for the product
This is a good question, as it implies the percentage of your users accomplished all the phases until they achieved the goal of your product – from being interested in the value proposition to being activated or buying a product.
2-) After they were activated, how much and how often did they use your product? These two questions indicate the intensity and frequency of use and represent a function of usefulness and value.
- Use: the number of times a product is used during a session, or the number of times a user achieves a product’s goal.
- Frequency: the number of times (or sessions) a product is used over time.
In the chart above, we see that Dropbox is a high use, high frequency product. Its users engage with the app many times over a given period of time, and each time (or in each session) they save many files (the product’s goal).
Google Search is another example of a high use, high frequency product. In each session users might do several searches and they search for different things very often.
Zillow (an online real estate database), on the other hand, is a high use, low frequency product, as its users look at dozens of different houses in each session, but the frequency at which they do so is not high over an extended period of time.
The Weather Channel app might well exhibit an inverse behavior. Its users might come back to the website or app every day or even several times a day during an extended period of time, but their use might be shallow, as they tend to check the weather in only one location (the product’s goal) during each session, and the sessions are short.
Want to learn more about how to implement advanced product analytics at your company? Download our paper, “Advanced Product Analytics.”