The following is a cumulative list of enhancements and major bug fixes for the Optimizely Experimentation Platform. Because Optimizely Experimentation releases new features and fixes as soon as they are ready, this list will update regularly. Follow this article to receive notifications as soon as new content is added.
October 2023
Result page segmentation update
In 2022, Optimizely started to partner with Google to innovate together and bring stronger data capabilities to you.
As part of this partnership, we are making the next step and are migrating a large part of our data infrastructure to the Google Cloud Platform. This will come with some new features and changes to how Optimizely calculates experiment results for specific user segments. The transition should be seamless for you and no downtime is expected. Your experiment results will stay accessible and running experiments will not need to be paused or restarted.
Starting to roll out on October 9, 2023, Optimizely is transitioning to event-based segmentation on the results page. This new approach will reflect the specific attribute values of each event rather than the last-touch within a session, offering a more accurate insight into individual events and their associated attributes. Event-based segmentation aligns more closely with advanced analytical tools like GA4, enhancing the precision in attribute assignment during segment filtering.
Example 1: Event-based segmentation for registration
Imagine a visitor browses an ecommerce site and adds items to their cart as a guest ("not logged in"). They decide to create an account at the checkout ("logged in").
Optimizely will segment the events accordingly:
- "not logged in" – Events related to browsing and adding items to the cart.
- "logged in" – Registration and checkout events.
This clear separation lets you conduct a more accurate user journey analysis, ensuring each event is associated with its accurate logged-in state.
Example 2: Event-based Segmentation for Membership Tiers
For example, imagine you are an ecommerce platform user and wish to assess the shopping behavior of users across different membership levels: Basic, Premium, and Elite.
A user with a 'Basic' membership logs in and browses your curated selection of products. During their browsing session, they encounter a promotion, enticing them to upgrade to the 'Premium' membership for an exclusive discount. They decide to upgrade.
- The actions taken by the user while on their 'Basic' membership are accurately attributed to that tier.
- Once they upgrade to 'Premium', actions are mapped under the 'Premium' category.
- This precision in segmentation ensures that each user action is correctly associated with its respective membership level, allowing for a granular analysis of user behavior across different tiers. This approach ultimately equips you with insights that can guide tailored marketing and UX strategies for each membership category.
Example 3: Multiple attributes in events
A user sends five events; two have the attribute "not logged in", and three have "logged in".
When you segment your results for:
- "not logged in" – The two corresponding events are displayed.
- "logged in" – The three remaining events are shown.
Event-based segmentation lets you view detailed insight into events, ensuring you see each event in its correct context without attribute overlap or confusion.
Additional information
- Decision event attribute values are not taken into account anymore. The conversion event attribute values determine if the data will be displayed when segmenting.
- Optimizely Web Personalization campaigns will continue to take the attribute value of the decision event into account, as you may qualify for multiple experiences in one session.
- The event-based segmentation update is not a UI change of any kind. Instead, the update modifies how numbers are displayed on the results page.
Enriched Events Export - Session ID generation
Removed automatically generated session IDs from the Enriched Event Export. Optimizely will not automatically create session IDs in the event pipeline as they are rarely used for outside analysis. Sessions are now calculated as part of experiment results to improve data accuracy.
Bot filtering
Updated the bot and spider filtering to improve data quality and accuracy.
June 2023
Bug Fixes
Fixed a bug when switching the baseline on the results page would show the wrong baseline's data.
May 2023
Results page improvements
- Fixed a bug where the legend for the Conversion Rate Over Time graph was overlapping.
- Added Last update and Last event dates on the Experiment Results page.
Additional improvements
- Added 202 return code for the
get_experiment_resultsandget_experiment_timeseriesendpoints for the Optimizely REST API indicating the requested operation has been received, but not completed, and users should wait and try again later. -
Added back the ability to unarchive experiments for the following products:
- Improved page load performance of the results page by not blocking when encountering stale cache data.
Bug Fixes
- Fixed a bug where changing the baseline in the Results page would interfere with the Statistical Significance being displayed.
March 2023
Bug Fixes
- Fixed an issue where a single experiment in a Personalization Campaign would attribute the variation data to the wrong row in the Experiment Summary table.
February 2023
Architectural updates
Moved backend architecture to Google Cloud Platform, allowing Optimizely Experimentation to deliver more complex features quicker, measure success effectively, and enable data democratization.
Added new result page alerts that provide details about experiment result data freshness.
Bug Fixes
Fixed the date logic for the result page segmentation feature. The date picker now properly handles the start and end date chosen by the user.
Fixed ResultsAPI to handle traffic allocations set to less than 100% correctly.
<%= heading %>
