Experimentation reporting dashboards overview

  • Updated
  • Optimizely Web Experimentation
  • Optimizely Feature Experimentation

View dashboards and reports for Optimizely Experimentation using Optimizely Reporting, which you can access through your Opti ID account.

Prerequisites

  • You must have an Opti ID account.
  • Your Opti ID user account must have a role for an Experimentation instance.
  • Your Opti ID user account must have a role for Reporting. This is assigned to users in your organization by default, so you should not have to configure this yourself.

See Get started with Optimizely Reporting for Experimentation for information on navigating and interacting with the reporting dashboards.

Program Overview

Click the Program Overview dashboard to view the related data and charts.

Experiment Status

You can view the experiment status for projects in your account, including how many are in Draft, Paused, Running, and Concluded status.

Completed Experiments over Time

You can view the outcome of the results for your completed experiments, including how many had Positive, Inconclusive, and Negative results and how many are Paused. These values are determined by the results outcome field in Optimizely Experimentation.

Experiment Velocity

  • Started Experiments – Experiments plotted by the date that you most recently started or published them. This shows the growth trajectory of your experimentation program.

  • Total Started Experiments – Groups the number of started experiments by experiment type, including the total number of variations for each experiment type. This helps you understand how you may be taking advantage of other experiment configurations.

  • Frequently Used Pages – Groups the number of started experiments by the pages used in the Web Experimentation setup.

  • Frequently Used Metrics – Groups the number of started experiments by the primary metric used in the experiment.

  • Variations per Experiment – Averages the total number of variations (including control) for the started experiments. This helps you understand whether you are following best practices of testing multiple variations in a single experiment to increase your chance of finding a winner (some source is needed about the benchmark report).

Trends

Click the Trends dashboard to view the related data and charts.

Trend of Completed Experiments

You can view a comparison of your completed experiments in line and bar graph format. The number of completed experiments is determined by the results outcome field in Optimizely Experimentation. 

Metric Impact Report

The Metric Impact Report dashboard is in beta.
You must configure at least one metric in your experiments to access the insights this dashboard provides.

Click the Metric Impact Report dashboard to view the related data and charts. See the interactive demo for help with using this dashboard.

Use the filters to select a specific metric and view how it was impacted by past paused or concluded experiments. You can also filter by the date an experiment started, an experiment name, whether the experiment is concluded or paused, and whether the results are statistically significant.

Overview

You can view an overview of the Positive experiments, Negative experiments, Total Experiments, and Win rate for your chosen metrics.

These values are inferred from the results of Optimizely metrics created and added to an experiment. These results are captured at the point of time in which the experiment changed status to Paused or Concluded (from Running).

Positive and Negative Impact

You can view a data table for Positive Impact (Gains), which displays experiments with improvement toward the winning direction, or a data table for Negative Impact (Loss Avoided), which displays experiments with improvement opposite the winning direction.

If the experiment contains more than two variations, including the original, this is determined by the following:

  1. Finding the variation that had the largest magnitude improvement in terms of absolute value.
  2. Evaluating if that improvement is in the same or opposite direction of the winning direction. If in the same direction, it had a positive impact, and if in the opposite direction, it had a negative impact.

You can click any numeric value in the dashboard tiles to drill down to a detailed table that shows each metric-variation combination for each experiment. See Expand, download, and view dashboard tiles for a list of options you have when viewing the detailed table for a dashboard tile.

Each row in the detailed table view for this specific dashboard represents an experiment-metric-variation combination, so there can be multiple rows for a single experiment.

Distribution of experiments by impact

View a histogram that counts and groups individual experiment impacts into a range of improvement percentages. This can be helpful for estimating the minimum detectable effect (MDE) size for future experiments.