Upgrade to Optimizely Feature Experimentation from Optimizely Full Stack

Optimizely Feature Experimentation has been the default experience for projects and Free Rollouts accounts since its launch in February 2021. Feature Experimentation offers a simplified workflow, model, and API for experimenting with and delivering experiences to your end users. It also provides a significantly more performant editing and management experience when working with feature flags at scale. See Key concepts and differences in Feature Experimentation. 

You can migrate your existing Optimizely Full Stack legacy projects. Still, before migrating a legacy Full Stack project to Feature Experimentation, Optimizely scans your project to check for edge cases that make it ineligible for migration. 

Eligible for migration (with considerations)

The following sections list functionality available in Optimizely Full Stack Experimentation today but are not yet complete in Optimizely Feature Experimentation. These items are not technical blockers and do not prevent you from migrating to Feature Experimentation. If the functionality listed below is essential to your project, you may consider delaying your migration to Feature Experimentation:

Full Stack (Legacy) project contains a Feature Test with an off variation

If your legacy Full Stack Experimentation project contains a Feature Test with an off variation, the variations remain off in your migrated project. Having off variations does not block you from upgrading, but you should assess what to do with your off variations before migrating. To check:

1. Select the Experiments tab.
2. Select the Feature Test you want to check.
3. View the experiment's variations.
4. Any variations will remain off in your migrated project if they are set to Off.

   

In Optimizely Feature Experimentation, you no longer toggle feature test variations on and off. Instead, you can use the one default off variation across all rules. Because of this, you cannot turn the Off variation back On after migrating. However, you could try the following possible solutions:

• Turn all variations On before upgrading to Feature Experimentation. 
• Delete any feature tests containing an Off variation.

Users with restricted roles

If your Full Stack (Legacy) projects contain users with a restricted editor or restricted publisher role, they are migrated to the normal editor or publisher roles after migration. See Feature Experimentation collaborator roles. 

To view your collaborators, go to Settings > Collaborators.

REST API usage

Optimizely Feature Experimentation has a new REST API that replaces a subset of the existing Optimizely API's functionality. If you currently manage the configuration of legacy Full Stack projects, features, or experiments using the REST API, you may need to update your code to use the Feature Experimentation API. Migrating before making required updates may result in broken functionality. See the following API reference documentation for information:

• Optimizely Feature Experimentation API Reference
• Optimizely Full Stack Experimentation (legacy) API Reference

Jira or Salesforce DNA integration 

The following integrations are not completed yet:

• Optimizely Feature Experimentation and Jira.
• Optimizely Feature Experimentation and Salesforce DNA.

If you upgrade to Optimizely Feature Experimentation now, you will lose the Jira and Salesforce integration functionality. See application and migration information.

See also how existing experiment results are migrated before moving to Optimizely Feature Experimentation.

Ineligible for migration (with possible solutions)

The following sections review the eligibility check and list steps to take if your project is ineligible to migrate.

Multiple feature tests with the same feature with identical variation keys

If your current Optimizely Full Stack Experimentation project contains the following scenarios, your project will NOT be able to migrate to Optimizely Feature Experimentation. There are several solutions for each situation, but you must decide how you would like to correct the issue for your project.

Your project cannot upgrade to Optimizely Feature Experimentation if your legacy Full Stack Experimentation project contains a single feature flag used in two (or more) separate feature tests with duplicate variable keys.

To check to see if your project is ineligible to migrate to Optimizely Feature Experimentation due to this scenario:

1. Select the Experiments tab. 
2. View your experiments and check if any experiments are using the same feature. You can tell which feature the experiment is using by viewing the features in the Feature column:
   
3. Select one of the experiments that share a feature.
4. Note the variation keys for the experiment.
5. Repeat steps 3-4 for the other experiments that share a common feature.
6. If any Variation Keys are identical, your project is ineligible to upgrade.

Example of ineligible setup

In the following example, the Yellow button experiment and the New homepage button share the same feature named First Feature. Additionally, both experiments use the Variation Keys variation_1 and variation_2. You cannot upgrade this example project to Optimizely Feature Experimentation. 

Yellow button experiment

New homepage button

In Optimizely Feature Experimentation, variations are shared across rules in a flag, making it easy to reuse variations across your rules. This is a notable difference from the legacy Optimizely Full Stack Experimentation, where variations are scoped to a single Experiment. 

During a migration, if a feature is used in multiple feature tests, all feature tests become A/B test rules within the flag. Further, all of the variations from the feature tests become variations within the flag. Optimizely cannot migrate the variations while maintaining backward compatibility if the legacy feature tests have colliding variation keys.

With the Yellow Button experiment and the New homepage button, the two feature tests share the same variation keys. 

To resolve this issue, you could do the following:

• Remove the feature flag from one of the features.
• Change the variation IDs so that they are unique across features.
• Archive one of the experiments using the duplicate variation keys.

"Everyone" rules are set between 0 and 100

If your Optimizely Full Stack Experimentation project contains multiple rules per feature and the Everyone rule has its traffic allocation percentage set at any value except 0 or 100, you cannot migrate.

To check to see if your project is ineligible to migrate to Optimizely Feature Experimentation due to this scenario:

1. Select the Features tab.
2. Select one of the features you want to check.
3. View the feature's audience rules.
   
   • If you have multiple rules, check if the last rule's traffic allocation is a percentage other than 0 or 100. If it is any number other than 0 or 100, your project is ineligible to upgrade.
   • For example, if your rule's traffic allocation is set to 50, your project is ineligible to upgrade.

   

4. Repeat steps 2-3 for the remaining features.

In Full Stack Experimentation, the default Everyone rule is evaluated last and has an adjustable traffic allocation.

Optimizely Feature Experimentation has a similar default rule that is evaluated last, but you only select a variation and do not set the traffic allocation. The remaining traffic is automatically delivered to the selected variation.  

Delivering Off variation in Feature Experimentation to remaining traffic

That difference in behavior means you cannot migrate feature flags in this state while maintaining backward compatibility. 

This happens because Optimizely went from N rules in Full Stack Experimentation to N+1 in Feature Experimentation. This is currently needed because Feature Experimentation does not let you control the traffic allocation on the last Everyone else rule, so Optimizely cannot map an Everyone targeted rollout rule to the Everyone else flags rule. Any targeted rollout features with a traffic value besides 0% or 100% for the Everyone rule will confront this issue.

To address this scenario, you can take the following action.

• Set the last rule's Rollout Traffic allocation to 0. After the migration, your feature flag is set to "off."
• Set the last rule's Rollout Traffic allocation to 100. After the migration, your feature flag is set to "on."

The experiment has more than ten whitelisted users

If your current Full Stack Experimentation project contains the following scenario, you cannot migrate your project to Optimizely Feature Experimentation. You must remove users from the allowlist to bring the user count to ten or fewer.

To check to see if your project is ineligible to migrate to Optimizely Feature Experimentation due to this scenario:

1. Select the Experiments tab.
2. Select one of the experiments you want to check.
3. Select Whitelist. View how many User IDs you have whitelisted.

   

4. Repeat steps 2-3 for the remaining experiments in the project.

In Optimizely Feature Experimentation, you are only permitted to allowlist ten users. 

To address this scenario, remove users from the allowlist to bring the user count to ten or fewer.