Skip to main content

Optimizely Feature Experimentation cumulative release notes

  • Updated

The following is a cumulative list of enhancements and major bug fixes in each Optimizely Feature Experimentation release.

Because Optimizely Feature Experimentation releases new features and fixes as soon as they are ready, this list will update regularly. Follow this article to receive notifications as soon as new content is added.

August 2023

Experiment Collaboration was released.

Experiment Collaboration is designed to help teams manage their experiment ideation, planning, and hypothesis-creation process more effectively. You can currently link your Optimizely Feature Experimentation experiments to your Hypothesis through your experiment URL. Learn more and create your first hypothesis by following the seven steps to set up Experiment Collaboration.

Feature Experimentation API improvements

May 2023

Breaking change SDK releases

The Advanced Audience Targeting integration with Optimizely Data Platform (ODP) for the following Feature Experimentation SDKs is now in beta:

New API added to the SDK's OptimizelyUserContext

  • fetchQualifiedSegments() this API will retrieve user segments from the ODP server. The fetched segments will be used for audience evaluation. The fetched data will be stored in the local cache to avoid repeated network delays.
  • When an OptimizelyUserContext is created, the SDK will automatically send an identify request to the ODP server to facilitate observing user activities.

New APIs added to the SDK's OptimizelyClient

  • sendODPEvent() – customers can build and send arbitrary ODP events that will bind user identifiers and data to user profiles in ODP.
  • createUserContext() – (For server-side SDKs only) – with anonymous user IDs, user-contexts can be created without a userId. The SDK will create and use a persistent VUID specific to a device when userId is not provided.

Additional releases

Bug Fixes

  • Fixed an error where Cross-project events do not always appear in all projects metrics.
  • Fixed various bugs with change history:
    • Fixed a bug that made change history unavailable for certain customers.
    • Fixed a bug where change history would return a 403 error.
    • Fixed a bug where change history would display Something Went Wrong and not load the change history.
    • Fixed a bug where existing rules would show as created instead of updated.
    • Fixed a bug where change history did not log the updated rule when the rule was added, updated, or removed from an exclusion group.
    • Fixed a bug where deleting a variation or rule also deleted the history of the variation or rule.
  • Fixed a bug on the Reports page where updated rule descriptions would not display.
  • Fixed a bug where new targeted delivery rules could not be saved.
  • Fixed a bug where pages would sometimes not populate when switching between differing project types.

Feature Experimentation API improvements

Additional enhancements

March 2023

Enhancements

  • Improved the layout of the Flags list for better usability.
  • Rules are set as paused on creation, which enables staging rules at scale.

Bug Fixes

  • Fixed an issue where the Flags UI dashboard was not loading for some customers.
  • Fixed an issue where a 500 error was being reported when trying to change the reporting date range in Program Management.

November 2022

New Feature Experimentation SDK 

Released an SDK for Flutter 

October 2022

Optimizely Feature Experimentation Parity Items

The following have been released to Optimizely Feature Experimentation, increasing the feature parity between the new Optimizely Feature Experimentation and legacy Optimizely Full Stack Experimentation products. 

August 2022

Optimizely Feature Experimentation Parity Items

The following have been released to the Optimizely Feature Experimentation, increasing the feature parity between Feature Experimentation and the legacy version of Full Stack. 

For more information on the Optimizely Feature Experimentation and Full Stack legacy feature parity work, refer to the developer documentation.

New Features

Created templates and starter kits to implement feature flagging and experimentation across major serverless edge compute platforms.

July 2022

New Features

Outlier Smoothing

To continue making our product better and easier to use for everyone, we have made a slight UI update to communicate better how Optimizely's Stats Engine handles outliers in your revenue metrics. There have not been any underlying changes in our Stats Engine or how Optimizely calculates results. This update is purely a phrasing enhancement to improve our user's understanding of what happens to their outlier values.  

What is an outlier?  

An outlier is an abnormally lower or higher value than other values in your results. 

What are Revenue Metrics?  

In Optimizely, a metric is a quantitative measurement of a visitor's action. Metrics are created out of events, which directly track actions like clicks, page views, form submissions, purchases, and scroll depth.  

The total revenue metric tracks the total revenue generated from user interactions with your event. Revenue usually is not your primary metric because, unlike clicks or page views, revenue does not measure a specific, discrete action taken by your visitors. However, tracking revenue is a fantastic way to tie your optimization efforts to the metrics that your company values most. 

What is changing? 

We are only updating the Optimizely UI and documentation to read "Outlier Smoothing" instead of "Outlier Filtering." We believe this update to the phrasing will help users understand what is happening with their outlier values.  

Why smooth outliers in revenue metrics?  

Outliers can severely skew the accuracy of any analysis conducted on a data set and can lead to potentially incorrect conclusions. Smoothing outliers results in more accurate findings. We recommend turning on this setting for your revenue metric to improve the integrity of your results. 

How does Optimizely treat outliers? 

For revenue metrics, Optimizely gives you the option to use outlier smoothing. Optimizely's outlier smoothing algorithm first identifies any values exceeding the daily exclusion threshold, extreme values three standard deviations higher than the observed mean. These extreme values are designated as outliers. Next, Optimizely replaces these outliers with the metric's harmonic mean value. This step in the process is known as outlier smoothing. Optimizely recalculates the daily exclusion threshold for each day using a moving average of your metric's arithmetic mean and standard deviation over the previous seven (7) days. This process repeats each day of the experiment. Please view our documentation on how Optimizely handles outliers for more detailed information.  

Conclusion 

If you have any questions or feedback, feel free to email us at support@optimizely.com! Keep on optimizing!

Forced Decision Methods

Here at Optimizely, we have been working hard to start 2022 with significant upgrades to our QA experience and security. We are proud to announce the release of new API methods for our Feature Experimentation SDKs and some critical security upgrades for Python. Thank you for reading this month’s release notes, and we are excited to see all the new places and faces 2022 will bring us!

Forced Decision Methods

We’ve recently rolled out a set of new APIs for our Feature Experimentation SDKs that will make overriding and managing user-level flags, experiments, and delivery rules even more straightforward.

These new methods extend our OptimizelyUserContext object, which previously allowed you to make flag decisions and flag events for a specific user. Now we’ve taken things up a notch!

The new methods allow you to do the following:

  • setForcedDecision—Forces a user into a specific variation
  • getForcedDecision—Returns the variation the user is forced into
  • removeForcedDecision—Removes a user from a particular variation
  • removeAllForcedDecisions—Removes a user from all forced variations

Using these “Forced Decision” methods makes it even easier to set up automated testing and QA by forcing certain User IDs into specific variations regardless of audience conditions and previously configured traffic allocations.

For more detailed information, please click on the SDK you are interested in to view the developer documentation on the Forced Decision methods:

Vulnerability and Python Support Upgrades

As part of Optimizely’s regular security updates and modernization of the Optimizely Feature Experimentation product, we will no longer officially support older versions of Python. These older versions do not provide secure libraries needed for the Optimizely Python SDK. For example, Python version 3.4 support has ended due to a known security vulnerability in the PyYAML library.

Optimizely supports the following versions:

  • Python 3.7
  • Python 3.8
  • Python 3.9
  • Python 3.10 and above
  • PyPy3

Developer Documentation UI Updates

Last month’s release notes announced that we are working on exciting UI changes to our developer and end-user documentation. If you visited our docs recently, you might have noticed some updates, including a completely new UI redesign. We’re happy to report that these changes are going well and will be fully released soon.

Conclusion

If you have any questions or feedback, feel free to email us at support@optimizely.com! Keep on optimizing!

Allowlisting

Quality assurance just got easier for Optimizely Feature Experimentation customers. Optimizely is excited to announce that Allowlisting is now available to all customers. Allowlisting, previously known as whitelisting, was available in the previous version of Full Stack, so we are excited to continue putting our developers first by fully releasing Allowlisting in our Flags experience.

Allowlisting enables you to force certain users into a specific variation of an experiment. This capability can be beneficial during the QA process of development. You can view updated developer documentation for additional helpful QA scenarios and steps to enable Allowlisting in your Flag Rules.

Dev Doc UI Updates

While viewing our docs, you may have noticed some exciting UI changes! We’re working behind the scenes on some exciting updates on our developer and knowledge base documentation. As our Optimizely brand continues to grow, we’re excited to combine all of our documentation under one instance. These updates will help you find the information you need quicker and easier than ever! There will be some more exciting updates coming soon, so stay tuned!

Conclusion

If you have any questions or feedback, contact us at support@optimizely.com!