Good experiment health

The bottom line 

When Experiment Health on your experimentation results page displays Good, it means that your Stats Engine experiment has the desired status.

The experiment's health status alerts you to untrustworthy experiment results on time. 

The green checkmark

Whenever you launch a Stats Engine A/B test, Optimizely Experimentation's automatic sample ratio mismatch (SRM) detection runs in the background. This algorithm finds unusual imbalances in visitor traffic to an experiment's variations and detects experiment problems early if they arise.

The Optimizely automatic SRM detection tool combines the power of automatic visitor imbalance detection with an indicator of experiment health. This indicator alerts you to any visitor traffic issues. It also tells you when everything works as expected and no traffic imbalance is present.

Good experiment health status

In your live experiment, the actual split of traffic assigned to a variation relatively aligns with the intended traffic split you assigned to a variation. It means that your running experiment experiences a non-statistically significant difference in visitor counts at the expected probability (such as a 50/50 split between variations). See Optimizely's automatic sample ratio mismatch (SRM) detection discovers any experiment deterioration early.

No underlying assignment bias is detected. It is not unusual for the traffic split to have this particular distribution. The algorithm finds no discrepancies between your intended outcome and the actual occurrences in the live experiment. See Imbalance detected: What to do if Optimizely's automatic SRM detection alerts you to an imbalance in your Stats Engine A/B test.

A sample ratio mismatch indicates a variety of data quality issues. 

Optimizely's automatic SRM detection continuously monitors your Stats Engine A/B tests and declares an imbalance carefully. The algorithm takes a dual-pronged, two-factor imbalance check by looking at the severity and consistency of the imbalance. This ongoing procedure during the life of your experiment maintains a Good status or changes to an imbalance detected and classified Critical. See Imbalance detected: What to do if Optimizely's automatic SRM detection alerts you to an imbalance in your Stats Engine A/B test.