Ineligible to migrate to Optimizely Feature Experimentation due to having multiple feature tests with the same feature with identical variation keys

  • Updated

If your current Optimizely Full Stack Experimentation project contains the following scenarios, your project will NOT be able to migrate to Optimizely Feature Experimentation. There are several solutions for each situation, but you must decide how you would like to correct the issue for your project.

Scenario

Your project is not eligible to upgrade to Optimizely Feature Experimentation if your legacy Full Stack Experimentation project contains a single feature flag used in two (or more) separate Feature Tests with duplicate variable keys.

When creating a feature test, Optimizely creates two variables automatically, named variation_1 and variation_2.


If you have not manually updated the variation IDs, there is a high chance that they will be duplicated.

How to check

  1. Select the Experiments tab. 
  2. View your experiments and check if any experiments are using the same feature.
    • You can tell which feature the experiment is using by viewing the features in the Feature column:
      experiments-page.png
  3. Select one of the experiments that share a feature.
  4. Note the Variation Keys for the experiment.
  5. Repeat steps 3-4 for the other experiments that share a common feature.
  6. If any of the Variation Keys are identical, your project is ineligible to upgrade.

Example of ineligible setup

Here we have two experiments, the "Yellow button experiment" and the "New homepage button." 

Both experiments share the same Feature named "First Feature." Additionally, both experiments use the Variation Keys "variation_1" and "variation_2." This example project would NOT be able to upgrade to Optimizely Feature Experimentation. 

Yellow button experiment:

same-variable2.png

New homepage button:

same-variable1.png

Technical reasoning

In Optimizely Feature Experimentation, variations are shared across all Rules in a Flag, making it easy to reuse variations across your Rules. This is a notable difference from the legacy Optimizely Full Stack Experimentation, where variations are scoped to a single Experiment. 

During a migration, if a Feature is used in multiple Feature Tests, all Feature Tests become A/B Test Rules within the Flag. Further, all of the variations from the Feature Tests become variations within the Flag. Optimizely cannot migrate the variations while maintaining backward compatibility if the legacy Feature Tests have colliding variation keys.

Returning to our earlier example with two experiments, "Yellow Button experiment" and "New homepage button," we can see that when this project would not be able to be migrated to Optimizely Feature Experimentation as the two Feature Tests share the same variation keys. 

legacy_to_flags_figma-smaller.png

 

Possible solutions

  • Remove the feature flag from one of the features.
  • Change the variation IDs so that they are unique across features.
  • Archive one of the experiments using the duplicate variation keys.