Build Experimentation-in-Personalization campaigns

  • Updated
This topic describes how to:
  • A/B test the personalized experiences you deliver to customers
  • Create two versions of an experience in a campaign, and compare how well they perform
  • Test and personalize together

In Optimizely Web Experimentation, the Experimentation in Personalization feature helps you test and personalize experiences at the same time.

Personalization lets you deliver tailored experiences to different audiences. But sometimes you are not sure what type of experience will make the biggest impact for a certain group of visitors. Use Experimentation in Personalization to test multiple variations for a single audience in your personalization campaign.

Visitors who meet the audience conditions for a certain experience are randomly bucketed into the different variations for that experience.

Imagine that you are personalizing the messaging on your homepage for two audiences: "Luxury Travelers" and "5x Shoppers." You are pretty sure what you want to show visitors who qualify as "5x Shoppers." But research shows that your "Luxury Travelers" visitors prize both luxury goods and a sense of adventure.

So, you create two different variations for that audience in your personalization campaign and test them.

"5x Shoppers" will see the "5x Shoppers" experience. "Luxury Travelers" will be randomly bucketed between the luxury goods and the adventure variations.

The Results page will show which variation performed better and how well each audience in the campaign performed overall.

Running an A/B test in a personalization campaign allows you to deliver value, while honing in on the optimal site experience for important groups of visitors at the same time.

This feature is available to accounts using both Optimizely Web Personalization and Optimizely Web Experimentation.

Add a variation to a personalized experience

Start by creating a new Personalization campaign. When you click Create Campaign, you enter the Manage Campaigndashboard.

pers-exp.png

Each audience that is attached to the campaign, such as "Luxury Travelers," will contain a single variation and a holdback. To experiment with multiple variations on a personalized experience for a single audience, add more variations.

  1. Add a new variation to an audience by selecting Add Variation.
    pers-exp-add-var.png

  2. Give your new variation a descriptive name. Then, click Save.
    pers-exp-add-var-2.png
    Great! You have added a new variation for that audience.

  3. To modify a variation, click Edit.

    pers-exp-edit-var.png

    Use the Editor to make changes to that variation.

Once all experiences and variations look and work to your preference, click Publish to start delivering them to visitors.

Visitors who meet the audience conditions will be randomly bucketed into the variations. You can then compare how visitors responded to the different variations and track results for that audience as a whole on the Results page.

Traffic allocation

Traffic is automatically split evenly between variations.

pers-exp-traffic-allocation.png

Visitors who qualify for the audience will be randomly bucketed into one of the variations or the holdback.

Change your traffic allocation with the following steps:

  1. Click the ... icon for the experience card and select Settings.
    pers-exp-settings.png

  2. Click into the traffic distribution to change it. Then, click Save Experience.
    pers-exp-traffic-allocation-2.png

Total traffic distribution must equal 100%. If it adds up to more or less, a warning message displays, prompting you to adjust your distribution.

Change the holdback

The holdback in a campaign helps you track how well the personalized experiences you deliver to each audience perform compared to a generic experience.

One holdback per campaign is set. So, no matter how many experiences and variations you add to a campaign, the holdback stays the same.

Here is how to change your holdback.

How many variations can be added?

There is currently no limit to the number of variations you can add for each audience in a Personalization campaign. You can also prioritize audiences so that if a single visitor qualifies for multiple audiences in Personalization, they will see only one experience by default.