- Optimizely Web Experimentation
- Optimizely Performance Edge
- Optimizely Feature Experimentation
- Optimizely Full Stack (Legacy)
Choosing the right metrics to track impact is a key part of good experiment design.
A conversion is an event that has a measurable impact on your business. Typically, a company's most important conversion events are revenue and top-level metrics that tie directly into revenue, such as purchase rate or average order value.
In experimentation, however, you seldom end up with actionable results if you rely only on these conversion events because they require huge shifts in behavior or large sample sizes to detect.
In this article, we demonstrate how to balance macro conversions like revenue with micro conversions that track visitor behaviors closely related to the changes you are testing. Knowing when to track micro-conversions instead of macro helps improve your win rate and show value when improving the user experience.
Macro-conversions
Macro-conversions are events that translate directly into revenue.
Each macro-conversion is tied to the industry's business model and occurs at the end of the funnel. A few examples of macro-conversions by industry include:
- Ecommerce – Purchases
- Travel – Bookings
- Media – Subscriptions, ad clicks
- B2B – Leads
Macro-conversions easily translate into revenue and are therefore very popular events to track in experiments and campaigns. But, the problem with macro conversions is:
-
They do not occur very often.
-
They usually occur at the end of the funnel.
Micro-conversions
Micro-conversions occur during the user journey as visitors travel from one step of the funnel to another. Each part of the visitor's experience may have different micro-conversions.
A few examples of micro-conversions for an ecommerce site include:
- Homepage – Searches submitted, category clicks, featured products clicks.
- Search results – Product clicks, filter and sort usage.
- Category page – Product clicks.
- Product page – Add-to-cart clicks.
- Shopping cart – Continue-to-cart clicks.
- Cart checkout – Fill-in payment details, purchase confirmation page views.
Each of these micro-conversions is a step to the purchase confirmation page view, which is the macro-conversion in this case. Micro-conversions are important milestones on the way to revenue and occur far more often.
Micro-conversions take more work to translate into revenue. If you increase the number of searches from the homepage, you get more visitors to the next step of the funnel. But it is hard to say how many more purchases you should expect from those events.
If you are using Optimizely Web Experimentation to test on a checkout page, you might need to configure your site for PCI compliance.
Optimize the metrics in your experiment design
The most efficient way to overcome the shortcomings of each type of event is to track both.
The problem with macro conversions is that they only measure home runs, but there are many ways to win in baseball.
- Hazjier Pourkhalkhali, Global Lead, Optimization Strategy at Optimizely
Imagine that you run an ecommerce website and are testing a new algorithm that selects products to feature on the homepage. Your main goal is to sell as many products as possible, so you decide to measure the number of purchases (a macro-conversion). If you sell more items with the new algorithm than the old one, it is a winner.
Unfortunately, the purchase rate for the average ecommerce business usually does not exceed 3-4%. Detecting a change to that small percentage requires a high volume of visitors in the experiment. If your company has low-to-medium traffic, this could take a couple of months.
Tracking micro-conversions is an effective strategy in this case. They usually occur more often than macro-conversions because they are closer to the tested change. So, you are more likely to detect a change in a reasonable period of time.
However important revenue is to a company, not all value that you will provide for your users will be directly captured as more revenue. Delighting your users by providing them with a top-notch experience is likely to increase important high-level goals such as user satisfaction, retention, or better engagement.
- Lev Tatarov, Strategy Consultant at Optimizely
Primary and secondary events are another way to think about your metrics strategy.
Example: Ecommerce metrics
Imagine you run an experiment on the product page highlighting a certain discount. You hope to make a purchase more attractive to visitors and increase revenue.
The macro-conversion for this case is purchases. If you think you can get a statistically significant answer on that metric—that is great! But in most cases, purchase completions are a few steps away from the product page. So, tracking a micro-conversion like clicks to the Add-to-Cart button is more effective.
Clicks to the Add-to-Cart button is the event closest to the change on the page and most likely to be affected. You have a much higher likelihood of getting a statistically significant result.
But how do you measure overall impact? Let us assume these baseline conversion rates for each step in the funnel:
- Product page -> Shopping cart: 20%
- Shopping cart -> Cart checkout: 75%
- Cart checkout -> Purchase completion: 60%
If you see a 10% increase in the add-to-cart rate, you can estimate the effect on purchases by multiplying the rates.
Estimated increase in purchases = .10 * .20 * .75 * .6 = 0.009
This calculation helps you estimate the overall value you would expect if you implemented the change on the site.
Best practices
It is a good idea use our A/B test sample size calculator to estimate the sample size for each variation in your test on micro and macro conversions in advance.
<%= heading %>
