New flag and rule lifecycle management FAQs

Flag and rule lifecycle management helps clarify the state of your experiments and targeted deliveries in Optimizely Feature Experimentation. New flag management statuses and various UI updates let you manage and track your experiments and targeted deliveries at a glance. See the walkthrough for a step-by-step guide on the new updates.

Flag management statuses

With the updated lifecycle management release, flags and rules display their current status on the flag and rule details pages. These statuses help you see your experiments' state at a glance. See the following sections for descriptions of each status.

Flags 

The updated flag statuses let you see your flag's condition for a specific environment. Instead of a toggle that was unclear if your flag was running, you can now check if the ruleset for an environment is in draft, running, or paused states. 

Each newly created ruleset (which includes the flag's environment and rules) starts in Draft status. Moving through your experimentation lifecycle, you update your ruleset's status accordingly.

Old flag management style (toggle)

• On – Flag is enabled and running in the environment.
• Off – Flag is disabled and not running.

New ruleset statuses

• Draft – Flag was recently created and never enabled.
• Running – Flag is enabled and running.
• Paused – Flag was previously turned on and is now paused with any rules not running or delivered to new users.

Rules

The updated rule statuses let you visualize your rule's progression. Instead of playing or pausing rules, you can see the connection between flags and rules and verify the correct combination. Additionally, the new Concluded status lets you deploy a variation without requiring a developer to update the code.

Each newly created rule (A/B test, targeted delivery, or multi-armed bandit optimization) starts in Draft status. After configuring and saving your rule, you can start delivering your rules to your audience by clicking Run.

Old rule management style

• Play – Press Play to start your rule.
• Pause – Press Pause to stop your rule.

New rule statuses

• Draft – Rule was recently created and never enabled.
• Ready to Run – Rule is enabled and ready to be delivered, but the associated ruleset is currently in a Draft or Paused state.
• Running – Rule is enabled and running.
• Paused – Rule was running and is now paused.
• Concluded – Rule has concluded, and no further action is necessary beyond reporting progress and delivering a winning variation. See the following paragraph for information.

The Concluded status changes the experiment status and displays it as Concluded in the experiment list. Additionally, concluding a rule removes it from the data file. Setting a rule to Concluded stops serving the variations unless you select Conclude and deploy. 

When concluding a rule, you can select the outcome of your rule to note if the rule had a positive, negative, or inconclusive effect on your hypothesis. Also, you can add a summary of your conclusion to recap your findings.

If you select Conclude and deploy, you can deploy the winning variation to new and returning visitors without increasing your Monthly Active Users (MAUs). You should have your developers make the winning updates to your code base, but you can continue to expose your audience to this winning variation until then. 

When deploying a variation, you can select:

• A variation from the rule to direct new and returning visitors to. 
• An audience to include in the variation.
• The total traffic allocation that sees the variation.

After deploying a concluded rule, the Optimizely Experiment Results page displays a message that the experiment was concluded and deployed:

Flags UI updates

Old UI

Previously, you toggled a flag On or Off to enable or disable all rules for the environment.

New UI

Now, click Run to deliver your entire flag's ruleset in that environment.

Clicking Run automatically updates the flag's status to Running. If your rule's status is Ready to Run or Running, they are deployed to your audience immediately. See Rule statuses for information.

To pause your flag, click Pause.

Clicking Pause automatically updates the flag's status to Paused and sets your rule's status to Ready to Run.  

Rules UI updates

Old UI

Previously, you clicked Play or Pause to start or stop your rule.

New UI

Now, after creating and saving your rule, the rule is in Draft status. To start delivering your rule to your audience, click Run.

Ruleset with a draft or paused status

If your ruleset is not set to running, you see a confirmation page explaining your ruleset is still paused:

Your rule's status is updated to Ready to Run.

Updating your ruleset's status to Running automatically updates your rule's status to Running.

Ruleset with running status

If your ruleset is already set to Running, after clicking Run on your rule, your rule is updated to Running.

After your ruleset and rule have Running status, your experiences are delivered to your defined audiences.

Pause rules

How you pause your rules in the new UI depends on the rule's status. 

Ready to Run status

If your rule's status is Ready to Run, you can pause your rule by clicking Pause.

Running status

If your rule's status is set to Running, click More (...) > Pause.

Conclude rule

The new UI lets you conclude your rule. How you conclude your rules in the new UI depends on the rule's status. 

Ready to Run status

If your rule's status is Ready to Run, you can conclude your rule by clicking More (...) > Conclude.

Running status

If your rule's status is set to Running, click Conclude.

Frequently asked questions

What changes are coming?

As part of this update, two major changes are coming:

1. New, clearer statuses for flags and rules to provide clarity on the state of your experiments and targeted deliveries. See Flag management statuses for information.
2. The ability to conclude and deploy winning variations. See the Conclude rule section.

What benefits am I getting from these changes?

This update includes various improvements, including:

• Visibility into the state of your experiments and targeted deliveries.
• Control over the lifecycle of your experiments and targeted deliveries.
• Consistency and accuracy in reporting on your experiments and targeted deliveries.
• Insights and learnings from your experiments and targeted deliveries.
• Confidence and ease in rolling out winning variations to your audiences.

Who is affected by these changes?

The new flag and lifecycle management affects anyone who uses Optimizely Feature Experimentation, including users on the Free Rollouts version. These changes apply to all existing and new flags, rulesets, and rules.

How is a ruleset different than a flag?

A ruleset lives inside of a flag. A ruleset is the collection of rules associated with a specific environment of a flag at a given time. See Key concepts for information.

Can I turn my rule back on after I conclude it?

No, but you can duplicate the rule and edit the new one.

Does concluding and deploying a variation cost me MAUs?

No.

How do you stop delivering traffic to the winning variation after concluding the rule and deploying? 

You can delete the rule that was concluded and deployed.