Enable email notifications in Optimizely Experimentation

  • Updated

Statistical Significance notifications

Statistical Significance email notifications are currently in Beta so Optimizely can monitor the scalability of this feature. Contact your Customer Success Manager to become an early user.
Relevant products:
  • Optimizely Web Experimentation
  • Optimizely Performance Edge
  • Optimizely Feature Experimentation

To enable statistical significance email notifications, navigate to Profile > Notifications and select the projects for which you want to receive an email notification when a running experiment in that project reaches statistical significance.

profile-with-notifications-stats-sig.png

When an experiment's primary metric reaches statistical significance, you will receive an email that provides enough context to help you determine the urgency of any corresponding action. For example:

stat-sig-notification-email.png

Optimizely Experimentation monitors statistical significance multiple times daily. You will receive a notification for the primary metric's statistical significance value (measured between the control and each variation) that exceeds the project’s statistical significance setting. Additional notifications will be sent for each new occurrence of a control and variation comparison reaching statistical significance.

If experiment results are reset and then a metric-variation reaches statistical significant (again) this will register as a first-time occurrence, and a notification will be sent.

This feature does not impact result calculations, nor does it delay the real-time processing of your results page. The default view of the results page that uses unfiltered visitor segments and date ranges determines statistical significance.

 

Impression notifications

Relevant products:
  • Optimizely Full Stack Experimentation (experiments created prior to February 2021)

To subscribe to email notifications when your impressions utilized exceeds 75% of the impressions included in your plan, navigate to Profile > Notifications. Under Impression Usage, select Impression usage notifications to enable email communications. 

profile-with-notifications-impressions-usage.png