Statistical Significance notifications
- Optimizely Web Experimentation
- Optimizely Performance Edge
- Optimizely Feature Experimentation
To enable statistical significance email notifications, navigate to Profile > Notifications and select the projects for which you want to receive an email notification when a running experiment in that project reaches statistical significance.
When an experiment's primary metric reaches statistical significance, you will receive an email that provides enough context to help you determine the urgency of any corresponding action. For example:
Optimizely Experimentation monitors statistical significance multiple times daily. You will receive a notification for the primary metric's statistical significance value (measured between the control and each variation) that exceeds the project’s statistical significance setting. Additional notifications will be sent for each new occurrence of a control and variation comparison reaching statistical significance.
If experiment results are reset and then a metric-variation reaches statistical significant (again) this will register as a first-time occurrence, and a notification will be sent.
- Optimizely Full Stack Experimentation (experiments created prior to February 2021)
To subscribe to email notifications when your impressions utilized exceeds 75% of the impressions included in your plan, navigate to Profile > Notifications. Under Impression Usage, select Impression usage notifications to enable email communications.