Events from paused experiments, stopped variations, and archived experiments

  • Updated
  • Optimizely Web Experimentation
  • Optimizely Performance Edge

If you pause an experiment, you must consider the change in user experience for visitors who are bucketed into that experiment.

If you are using Optimizely Performance Edge, certain features described in this article behave differently than in Optimizely Web Experimentation. Optimizely Performance Edge is a lightweight experimentation product that delivers significantly faster performance than previous versions of Optimizely. It does this by relying on a streamlined microsnippet, which limits the range of available features.

You see this notation edge-info.jpg whenever the text describes a feature that works differently in Optimizely Performance Edge than in Optimizely Web Experimentation.

Paused experiments

If you pause a running experiment, the variation code is removed from the snippet. This means that the experiment and variation code can no longer run on the page because the code is removed from the snippet.

  • New visitors are not evaluated for the experiment. This means that no new visitors are bucketed into the experiment after the experiment is paused.

  • Visitors already bucketed into the experiment while the experiment was running see the native web page, with no experiment or variation code added.

Restart paused experiments

If you restart a paused experiment, the experiment and variation code are added back to the snippet. Visitors bucketed into the experiment while it was initially active see the variation they saw before. Returning visitors are not re-bucketed into a new variation because their bucketing information is stored locally in their browsers and does not change if the experiment is paused.

Optimizely Experiment Results page

When an experiment is paused, the results page stops refreshing until the experiment is restarted. Events scoped to the page that triggered when the experiment was paused are still collected. However, they do not display on the results page until the experiment is restarted.

While an experiment is paused, bucketing decisions are not collected for events scoped to the experiment. The results page may show a gap when the experiment was paused.

Network events

Visitors who were already bucketed into the experiment can continue to trigger events while the experiment is paused. If you were testing your experiment and were bucketed into it while it was running, then certain events (specifically, pageview, click events scoped to the page, and custom events) continue to trigger and are visible in the Network tab of your browser.

Events scoped to the experiment do not trigger when an experiment is paused. Only pageview events, click events scoped to a page, and custom events are sent to Optimizely when an experiment is paused. These events are not visible on the results page. 

Stopped variation Performance Edge note

If you stop a variation but do not stop the experiment, Optimizely removes both the variation code and the variation ID from the snippet and leaves the experiment and any other variation information intact when a variation is stopped.

  • New visitors are evaluated by Optimizely Web Experimentation for the experiment and bucketed randomly into the remaining variations. This means no new visitors are bucketed into the stopped variation.

  • In Optimizely Web Experimentation, visitors who were already bucketed into the stopped variation see the native web page with no experiment or variation code added.

  • Performance Edge note In Optimizely Performance Edge, visitors who were bucketed into the stopped variation are re-bucketed into another variation in the experiment.

When you stop a variation, you cannot restart it.

Optimizely Experimentation Results page

The results page continues to collect data for the experiment. However, you should expect conversions for the stopped variation to decline.

Updating visitor bucketing information

Visitors bucketed into the stopped variation continue attributing events to the results page until they return to a URL where the experiment is active. Then, bucketing information updates, and the variation ID is removed. At that point, events are no longer attributed to the variation ID on the results page.

Network events

If you are testing a stopped variation, you may see events triggered in the Network tab of your browser. Go to the page where the experiment activates and expand the network event. If the variationId is null, the bucketing information is updated.

Archived experiments

When an experiment is archived, the results page records a new stop date and time and ceases tracking events for the experiment. See manage experiments and campaigns in Optimizely Web Experimentation.