Every Optimizely Personalization campaign has a holdback, the control of an A/B test. Optimizely's Stats Engine uses the holdback to help you track how well your personalized experiences perform compared to the generic experience.
By default, Optimizely Personalization shows a personalized experience to 95% of campaign visitors and holds back 5% who see the original.
Visitors are bucketed into the holdback by chance.
Holdbacks are persistent. When a visitor is bucketed into the holdback, they always see the holdback experience for that campaign.
Managing the balance between too big of a holdback and too small can be confusing. Five percent holdback is the industry standard because it is a good balance and lets you monitor how well your campaign is doing.
A larger holdback may not help you gather data about statistical significance faster. Instead, you risk underpowering your experiment. An experiment that is too underpowered may be inconclusive. Your traffic is finite, and having a large holdback prevents a large amount of traffic from seeing the personalized experience.
Set the holdback
When you create a campaign, set the holdback in the final step.
Optimizely Personalization sets one holdback per campaign. The holdback stays the same no matter how many experiences or variations you add to a campaign.
- Go to Optimizations and select the campaign.
- Click Settings.
- Edit the holdback.
- Click Save.
To reset the holdback, click Revert.
Article is closed for comments.