Segment your results in Optimizely Feature Experimentation

  • Updated

Relevant products:

  • Optimizely Feature Experimentation

This topic describes how to:

  • Segment your Results page to see how different groups of visitors behave
  • Identify whether certain groups of visitors behave differently from your visitors overall
  • Use default segments or create custom segments using attributes

By default, Optimizely Feature Experimentation shows results for all visitors who enter your experiment. However, not all visitors behave like your average visitors. Optimizely Feature Experimentation lets you filter your results of unarchived experiments so you can see if certain groups of visitors behave differently from your visitors overall. This is called segmenting.

For example, imagine you run an experiment for a pop-up promotional offer. This generates a positive lift overall, but when you segment for visitors on mobile devices, it is a statistically significant loss. Maybe the pop-up is disruptive or difficult to close on a mobile device. When you implement the change or run a similar experiment in the future, you might exclude mobile visitors based on what you have learned from segmenting.

Segmenting results is one of the best ways to gain deep insight beyond the average visitor's behavior. It is a powerful way to step up your experimentation program.

Set up your experiment's segmentation

For more information, and specific SDK examples, refer to the Optimizely Feature Experimentation developer documentation.

Unlike Optimizely Web Experimentation projects, experiments in Optimizely Feature Experimentation projects do not include “out-of-the-box” attributes such as browser, device, or location. Optimizely Feature Experimentation's SDKs are platform-agnostic, so we make no assumptions about what attributes are available in your application or their format. This means that for Optimizely Feature Experimentation, all segmentation is based on custom attributes you create.

Here is how it works:

  1. Create the custom attributes you want to use for Results page segmentation.

  2. Create audiences based on your custom attributes.

  3. Pass the custom attributes into the activate(), track(), and variable getter functions for your experiment or app for legacy Full Stack experiments. For Optimizely Feature Experimentation experiments, pass the custom attributes into the user context for your experiment or app. 

After you define custom attributes and pass them in the SDK, they will be available as segmentation options from the drop-down menu at the top of the Results page.

Segment the Results page

In Optimizely Feature Experimentation, you can segment your entire Results page. Segmenting results helps you get more out of your data by generating valuable insights about your visitors.

In order for Legacy Optimizely Full Stack to use attributes for segmentation, the attribute must be defined in the datafile, and it must be included in both the activate() and track() calls. However, it does not have to be added as an audience to the test.

For Optimizely Feature Experimentation, pass the custom attributes into the user context for your experiment or app. 

  1. Navigate to your Results page.

  2. Click Segment and select an attribute from the dropdown.

Under the Segment dropdown, you will find default segments and custom segments in the same place.

segments.png

How visitors are counted in segments

When segmenting results, a visitor who belongs to more than one segment will be counted in every segment they belong to. However, if a visitor has more than one value for a single segment, the visitor is only counted for the last-seen value they had in the session. Learn more about how visitors and conversions are counted in segments.