Contextual Bandit Results page

The Contextual Bandit Results page lets you track your contextual bandits' performance, view different experiences and audiences to examine visitor behavior, and take action. Contextual bandits score variations based on the impact on the primary metric and user attributes to serve the most personalized variation to every user.

Contextual bandits use decision-level scoping for event attribution. This means Optimizely assigns events to the decision they happened in. In other words, Optimizely associates events with the decision event that occurred before the event. Optimizely uses decision-level scoping because while the contextual bandit is running, user context can change, and the contextual bandit would optimize for this user, using their new context. So, Optimizely uses decision-level scoping to capture these details. Optimizely uses decision-level scoping to capture these details. See the Decision-level scope section in the How Optimizely Experimentation counts conversions documentation for information.

Most metrics are calculated by Optimizely using a session scope within a Personalization Campaign. For contextual bandits, Optimizely automatically changes this into decision scope, as it is the best way to capture their performance. Other metrics (for example, revenue per conversion and value per conversion) are still calculated per conversion, even though the UI says decisions. You should not use these metrics for contextual bandits.

View results for contextual bandits

You can view your contextual bandit results from the Campaign Results page.

To access the Campaign Results page, complete one of the following:

• From the Optimizations dashboard, click Results.
• From the Manage Campaign dashboard, click Results.

Then, in the Experiences section, click on a contextual bandit to open its results page.

About the contextual bandits results page

Use the following example results page to understand each section of the Contextual Bandit Results page:

1. Segmentation drop-down lists – Select different options to see how different segments of users behave on different dates.
2. Download, Share, Edit – Download results as a PDF, copy a shareable URL, or edit the experiences.
3. Summary – Displays the total number of visitors and decisions for the primary metric. A higher number of decisions than visitors suggests the contextual bandit is helpful.
4. Distribution – Shows how the contextual bandit split traffic between users who saw randomized variations and those who received personalized ones.
5. Top User Attributes – Lists user attributes that the contextual bandit model considered the most important when assigning users to variations. Put another way, the "weight" of the attribute when deciding to put a user into a specific variation. These attributes influenced the model's decisions the most.
6. Download Results – Download the full list of user attributes provided to the contextual bandit model with their weights (relative importance).
7. Metrics – Displays how each variation performed for the selected metric, including Unique Conversions, Number of Decisions, and Conversion Rate.
8. View or Hide Graph – Expand or collapse the corresponding metric's graph.
9. Graph type selector – Use the drop-down list to switch between available graphs, including Decisions Over Time, Unique Conversions Over Time, and Unique Conversions per Decision Over Time.
10. Graph details – Hover over the graph to inspect how each variation performs.
11. Additional metrics – View performance for the remaining metrics in your contextual bandit.