Segment your results in Optimizely Feature Experimentation

  • Updated
  • Optimizely Feature Experimentation

By default, Optimizely Feature Experimentation shows results for all visitors who enter your experiment. However, not all visitors behave like your average visitors. Optimizely Feature Experimentation lets you filter your results of unarchived experiments so you can see if certain groups of visitors behave differently from your visitors overall. This is called segmenting.

For example, imagine you run an experiment for a pop-up promotional offer. This generates a positive lift overall, but when you segment for visitors on mobile devices, it is a statistically significant loss. The pop-up may be disruptive or difficult to close on a mobile device. When you implement the change or run a similar experiment in the future, you might exclude mobile visitors based on what you have learned from segmenting.

Segmenting results is one of the best ways to gain deep insight beyond the average visitor's behavior. It is a powerful way to step up your experimentation program.

Segments and filters should only be used for data exploration, not making decisions.

Set up your experiment's segmentation

Optimizely Feature Experimentation projects do not include “out-of-the-box” attributes such as browser, device, or location. Optimizely Feature Experimentation's SDKs are platform-agnostic, so Optimizely makes no assumptions about what attributes are available in your application or their format. This means that for Optimizely Feature Experimentation, segmentation is based on custom attributes you create:

  1. Create the custom attributes you want to use for results page segmentation.
  2. Create audiences based on your custom attributes.
  3. Pass the custom attributes into the user context for your experiment or app. See your SDK's user context documentation:

After you define custom attributes and pass them in the SDK, they are available as segmentation options from the drop-down list on the results page.

Segment the results page

In Optimizely Feature Experimentation, you can segment your entire results page. Segmenting results helps you get more out of your data by generating valuable insights about your visitors.

For Legacy Optimizely Full Stack to use attributes for segmentation, the attribute must be defined in the datafile and included in the activate() and track() calls. However, it does not have to be added as an audience to the test.

For Optimizely Feature Experimentation, pass the custom attributes into the user context for your experiment or app. 

  1. Go to your results page.

  2. Click Segment and select an attribute from the dropdown.

How visitors are counted in segments

For information on how Optimizely counts conversions when segmenting, see: