Data Retention Policy update and FAQs

  • Updated

Optimizely recently updated the Experimentation Data Retention Policy for event data to maintain the highest quality compliance and testing standards.

Starting October 2022, Optimizely will only retain active subscription experimentation event data for 18 months.

This means that active customers with experiments with event data older than April 2021 will start seeing result page data expire on a day-to-day rolling basis. 

Example

For example, an experiment was run on November 16, 2021. The goal was to increase conversions over Black Friday. The test was successful and paused on December 16, 2021. The 18-month time period would be on May 16, 2023.

If you visited the results page before the retention period expired, the data would reflect the whole month of data.

If you visited the results page one day after the retention period ended, the data would reflect all the data except the data from the first day, November 16, 2021. In other words, the data available would be from November 17, 2021, to December 16, 2021.

If you visited the results page two weeks after the retention period ended, the data would reflect the last two weeks of data captured, or data captured from November 30, 2021, to December 16, 2021.

Because this test was over the holiday season, the days that expire can impact the results as one day of a lot of data missing means the results are not telling the whole story.

 

FAQ

When does the 18-month retention period begin?

The 18-month retention period begins when the first event is processed for an experiment.

 

How does this update impact the Results page?

The Results page will still be accessible and continue to show data even if the experiment has data past the 18-month retention period.

The event data expires daily, meaning data shown on the page over 18 months will be statistically correct, but days will be missing from the data set. The missing days can impact the value of the data shown, which is why there will be a persistent message indicating there is event data older than 18 months.

 

Does it impact all experimentation platforms?

Yes, Optimizely Web Experimentation, Performance Edge and all versions of Optimizely Full Stack are all impacted the same.

 

Does it impact all types of experiments?

Yes, the updated retention policy applies to A/B Tests, Multi-armed Bandit Tests, Multivariate Tests and Personalization experiments.

 

How can I make sure I have the data I need?

The information displayed on the Results page can be exported via the CSV Export functionality at the top, right-hand corner of the page.

share_csv.png

 

Will I be notified when the retention period is ending?

There will be a reminder message on experiments within 30 days of the retention policy period ending.

 

Does resetting the results start the retention period time frame over?

No, resetting the results will not reset the retention time frame.

If an experiment began collecting events on November 16, 2021, the retention time frame would end on May 16, 2023.  If the results were reset at any time, the retention time frame would remain May 16, 2023, as the experiment started collecting events on November 16, 2021.

 

Does the Event Data Retention Policy update impact Enriched Event Export (E3) data files?

No. Enriched Event Export (E3) files are retained for 12 months and expire on a rolling basis by day. There was no change to the retention policy.

 

Can we extend the retention time frame?

We do not have plans to offer extended retention time frames, but if it is something that would be of interest, please let us know by signing up here. We will inform you of changes or news about extended retention time frame options.