Enable email notifications in Optimizely Experimentation

  • Updated
  • Optimizely Feature Experimentation
  • Optimizely Web Experimentation
  • Optimizely Performance Edge
  • Optimizely Full Stack (Legacy)

Statistical significance notifications

To enable statistical significance email notifications, go to Profile Notifications and select the projects for which you want to receive an email notification when a running experiment in that project reaches statistical significance.

When any of the experiment's metrics reach statistical significance, you receive an email providing enough context to help determine the urgency of any corresponding action. For example:

Accounts-3.png

Optimizely Experimentation polls for statistical significance differently, depending on how long the experiment has been running:

  • 1-14 days – Polls every 6 hours.
  • 14-30 days – Polls every 24 hours.
  • 30-60 days – Polls every 36 hours.
  • 60-90 days – Polls every 48 hours.
  • 90-365 days – Polls every 96 hours.
  • 365+ days – Not monitored. 

You receive a notification for the metric's statistical significance value (measured between the control and each variation) that exceeds the project's statistical significance setting. Optimizely sends additional notifications for each new control and variation comparison occurrence that reaches statistical significance.

If you reset the experiment results and then a metric-variation reaches statistical significant (again), this registers as a first-time occurrence, and Optimizely sends an email notification.

This feature does not impact result calculations, nor does it delay the real-time processing of your results page. The default view of the results page that uses unfiltered visitor segments and date ranges determines statistical significance.

Impression notifications

  • Optimizely Full Stack (Legacy)

To subscribe to email notifications when your impressions used exceeded 75% of the impressions included in your plan, go to Profile Notifications. Under Impression Usage, select Impression usage notifications to enable email communications.

Accounts-4.png