Segment your results

  • Updated
  • Optimizely Web Experimentation
  • Optimizely Personalization

By default, Optimizely Web Experimentation and Optimizely Personalization show results for all visitors who enter your experiment. However, not all visitors behave like your average visitors. Segmenting lets you filter your unarchived experiment results to see if certain groups behave differently from your visitors overall. 

You can filter results with default segments that Optimizely captures automatically:

  • Browser type
  • Campaign
  • Device type
  • Referrer
  • Source type

You can also add custom attributes to your experiments and campaigns and use these attributes for custom segmentation to get a more granular view of your business's most important visitors.

For example, you run an experiment that tests a pop-up promotional offer. This generates a positive lift overall, but segmenting visitors on mobile devices is a statistically significant loss. Maybe the pop-up is disruptive on a mobile device or difficult to close. When you implement the change or run a similar test in the future, you might consider excluding mobile visitors based on what you have learned.

Segmenting results helps you see beyond the average visitor's behavior and improve your experimentation program.

Optimizely Web Experimentation does not offer segmentation by audience.

Default segments

Optimizely Web Experimentation and Optimizely Personalization capture several default attributes that you can use to segment your results page:

You must have at least one visitor assigned to an attribute for it to display on your results page.
  • Browser type – 
    • Firefox
    • Google Chrome
    • Internet Explorer
    • Opera
    • Safari
    • Unknown
  • Campaign – Visitors who enter the experiment with the parameter utm_campaign capture the parameter value. Use this value to examine campaign keywords like holiday_sale.

    There is a 20-character limit for values. Longer values are truncated.

  • Device type – 
    • Desktop/laptop – Visitors rising from a desktop or laptop.
    • All phones – Visitors using any mobile phone. It is a combination of the iPhone and Other phone segments.
    • iPhone – Apple iPhone visitors
    • Other phones – Visitors using a mobile phone that is not an iPhone.
    • All tablets – Visitors using any tablet. It is a combination of the iPad and Other tablets segments.
    • iPad * – Visitors using iPads.
    • Other tablets – Visitors using tablets that are not iPads.
    • Other devices – Visitors using devices not identified as desktops, laptops, phones, or tablets.
    • Desktop (deprecated) – Visitors using a desktop, laptop, or other device. Use the desktop/laptop segment to focus on visitors using traditional desktop browsers.
  • Referrer – Initial referrer or the originating URL (the URL of a page outside the domain of the snippet that the visitor navigated from before landing on a page where the snippet is implemented).

    For example, if a visitor navigates to your domain by clicking on a link on https://www.facebook.com/search/top, events triggered during the session have a referrer value of https://www.facebook.com/search/top.

  • Source type
    • Campaign – Visitors that arrive on a URL containing a utm_source, gclid, or otm_source query parameter. If the URL contains one of these parameters, the visitor counts as Campaign traffic, even if they arrived through a search.
    • Direct – Visitors who do not have any external referrer in their URL.
    • Referral – Visitors from another URL that do not count as Campaign.
    • Search – Visitors who arrived through search.
*Apple changed the user agent for iPad in iOS 13 such that it will display as desktop/laptop traffic in Optimizely and other analytics tools. This may increase desktop/laptop traffic and decrease iPad traffic when segmenting your results.

The Source value is based on the document.referrer value in the browser, except Campaign. Visitor Source type value may change as they navigate your site, such as a checkout flow with a step where the visitor goes to a third-party site or different subdomain and then returns. Visitors would have an original Source value (campaign, referral, or direct) until they leave the site. When the visitor returns, they have a referral value because Optimizely detects that they went to another site.

You may see the same visitor's conversions in segments for different Source values on your Results page. For example, conversions that the visitor made before leaving the site show in Campaign or Direct. Conversions triggered after the visitor returns show in Referral without affecting the earlier conversions.

Custom segmentation

You can create custom attributes and use them for custom segmentation (with more granularity) on the results page. Custom attributes describe key characteristics of your different visitors. Use them to segment your results page in the exact same way as default segments.

For example, you might create a custom attribute that targets by plan type, which includes the values basic, plusand premium. You can use these to segment your results and identify when visitors with a basic plan behave differently from visitors with a premium plan.

Segment the results page

You can segment your Experiment Results page and Campaign Results page. Segmenting results help you get more out of your data by generating valuable insights about your visitors.

Segments and filters should only be used for data exploration, not making decisions.

Under the Segment drop-down list, you see default segments and custom segments for any attributes assigned to at least one visitor in the experiment.

segment-results-web.png

  1. Go to your results page.
  2. Click Segment and select one or more attribute values from the drop-down list.
  3. Choose Match All Values or Match Any Value if selecting multiple.
  4. Click Apply.

Match All Values and Match Any Value

When you select more than one attribute value from the segmentation menu, the results page can display visitors who match all values or visitors who match one or more values. The radio buttons control this behavior labeled Match All Values and Match Any Value.

For example, you implement two custom attributes to track a visitor’s location and plan type. If you want to see how visitors in the US with a Gold plan responded to your variations, your segmentation configuration looks like this:

Match All Values

  • Location US
  • Plan Type – Gold

If you select Match Any Value instead, the results page displays data for visitors who are in the US combined with data for visitors who have a Gold plan.

How visitors are counted in segments

For information on how Optimizely counts conversions when segmenting, see: