Compound metrics

  • Updated
  • Optimizely Web Experimentation
  • Optimizely Personalization
  • Optimizely Performance Edge
  • Optimizely Feature Experimentation
  • Optimizely Full Stack (Legacy)

Compound metrics are calculated metrics that let you combine data about two different events in a single metric, giving you greater flexibility and letting you create more advanced metrics. Using compound metrics, you can create custom metrics based on the total count (total conversions), unique count (unique conversions), total revenue, or the total value of one event divided by the total count (total conversions) or unique count (unique conversions) of a second event. 

Compound metrics are in beta.

Calculation methodology

To calculate compound metrics, Optimizely does the following:

  1. Check the denominator event – Optimizely first checks if the user completes the denominator event.
  2. Check the numerator events – If the user completes the denominator event, Optimizely then checks for the numerator events within a 48-hour window. The numerator events must occur within 48 hours after the denominator event to be included in the compound metric calculation.

Examples

Ecommerce website example

For example, you have an ecommerce website and want to track purchases. You are running an A/B test on a product details page. Variation A has a large photo and large text, while variation B has a small photo and small text. You want to track the conversion rate of people who initiate a purchase by adding an item to a cart and then completing the purchase. In this scenario, you have two events to track.

  1. Clicking an add-to-cart button
  2. Completing the checkout

In this example, the compound metric is the ratio of completed checkouts (numerator event) divided by the number of add-to-cart clicks (denominator event). 

Example calculation

Using the previous example, this is how the compound metric, Purchases, is calculated.

  • 100 people were bucketed into variation A.
    • From them, 20 people clicked the add-to-cart button.
      • Of them, 17 completed their checkout within 48 hours.
  • 150 people were bucketed into variation B.
    • From them, 13 people clicked the add-to-cart button.
      • Of them, 12 people completed their checkout within 48 hours.

Metric 1 calculation (unique add-to-cart click count)

Variation A (large image, large text)

  • Unique conversions – 20
  • Conversion rate – 20/100 = 20%

Variation B (small image, small text)

  • Unique conversions – 13
  • Conversion rate – 13/150 = 8%

Metric 2 calculation (compound metric; the percentage of completed checkouts after adding to the cart)

Variation A (large image, large text)

  • Unique conversions for completed_checkout event – 17
  • Unique conversions for add_to_cart event – 20
  • Conversion rate – 17/20 = 85%

Variation B (small image, small text)

  • Unique conversions for completed_checkout event – 12
  • Unique conversions for add_to_cart event – 13
  • Conversion rate – 12/13 = 92%

Example metric calculation logic

Using the previous ecommerce example, here are four example users, their checkout experiences, and how Optimizely would count their metrics.

  • User 1 – Lands on the product details page. Clicks add_to_cart three times. Completes the checkout for these three items. After ten hours, returns to the product details page, clicks add_to_cart one time, and completes their checkout.
  • User 2 – Lands on the product details page. Clicks add_to_cart one time and leaves the page. After 14 hours, returns and clicks add_to_cart one time and completes their checkout.
  • User 3 – Lands on the product details page. Clicks add_to_cart two times. After three days (72 hours), returns to the product details page, clicks add_to_cart one time, and completes their checkout.
  • User 4 – Lands on the product details page. Clicks add_to_cart one time, leaves the page, and never returns or completes their checkout.

Compound metric calculation (total add_to_cart/total completed_checkout = counting events)

  • User 1 – 3 add_to_cart + 1 add_to_cart/1 completed_checkout + 1 completed_checkout = 4 add_to_cart/2 completed_checkout
  • User 2 –  1 add_to_cart + 1 add_to_cart = 2 add_to_cart/1 completed_checkout)
  • User 3 – 1 add_to_cart/1 completed_checkout. The first two completed_checkout events were over 48 hours from the purchase.
  • User 4 – Null. The completed_checkout event did not happen, and completed_checkout is the denominator.

Compound metric calculation (unique add_to_cart/unique completed_checkout = counting users)

  • User 1 – 1 add_to_cart/1 completed_checkout. Even though there were 2 and 4 add_to_cart events, what matters for Optimizely is that there is one user.
  • User 2 – 1 add_to_cart/1 completed_checkout
  • User 3 – 1 add_to_cart/1 completed_checkout
  • User 4 – Null. The completed_checkout event did not happen, and completed_checkout is the denominator.

Best practices

By following these methodologies and best practices, you can leverage compound metrics to gain deeper insights into user behavior while maintaining a comprehensive understanding of the underlying data dynamics.

Compound metrics can serve as your primary, secondary, or monitoring metric. You should create separate secondary metrics for the numerator and the denominator to ensure you understand the value of the compound metric. This is important because:

  • If only one event changes drastically, the compound metric can become skewed, potentially causing misleading conclusions.
  • If the ratio remains the same, but the total counts of both events drop dramatically, it could indicate an underlying issue not apparent from the compound metric alone.

From the previous ecommerce example, you should create the following:

  • A Completed Purchase compound metric.
  • A simple metric based on the completed_checkout event.
  • A simple metric based on the add_to_cart event.