Manage rules

  • Updated
  • Optimizely Feature Experimentation

Rules contain the experiment and targeting logic within a flag. Rules describe what variation the flag should deliver to a given user. A ruleset is the collection of rules associated with a flag.

There is a limit of 100 rules per ruleset (100 rules across a particular flag and environment).

Rules can be in one of the following states:

  • Draft – Rule recently created and never enabled.
  • Ready to Run – Rule is enabled and ready to be delivered, but the associated ruleset is in Draft or Paused state.
  • Running – Rule is enabled and running.
  • Paused – Rule was running and is now paused.
  • Concluded – Rule has concluded, and no further action is necessary beyond reporting progress and delivering a winning variation. See Conclude rule.

See New flag and rule lifecycle management FAQs for information.

When you create a flag, Feature Experimentation automatically assigns you the Admin role for that flag using granular permissions. This ensures you can immediately manage rules associated with that flag.

Start a rule

Go to Flags and select your flag. Select the environment.

From the ruleset

  1. Click More options (...) on the rule.

  2. Select Run. 

From the rule details

  1. Click the rule.

  2. Click Run. 

Rule in a draft or paused ruleset

If your ruleset is not set to running, you see a confirmation page explaining that your ruleset is still paused: 

Click Ok.

Your rule's status is updated to Ready to Run, like in the following screenshot: 

To update your rule's status to running, set your ruleset's status to Running.

Rule in a running ruleset

If the ruleset is already running, clicking Run immediately updates the rule to Running. The rule does not enter Ready to Run status, and no confirmation page is shown. 

Stop a rule

Go to Flags > select your flag. Select the environment.

From the ruleset

  1. Click More options (...) on the rule.

  2. Select Pause. 

From the rule details

  1. Click the rule.
  2. Click More options (...).

  3. Select Pause. 

Conclude a rule

Setting a rule to Concluded ends the experiment and removes it from the datafile. Feature Experimentation stops serving variations unless you choose Conclude and deploy.

When you conclude a rule, you can select the outcome of your rule and add a summary of what you learned from your hypothesis. This helps track whether the experiment had a positive, negative, or inconclusive result.

After concluding your rule, you cannot modify that rule.

Go to Flags > select your flag. Select the environment.

From the ruleset

  1. Click More options (...) on the rule.

  2. Select Conclude. 

  3. Select an option for the Results Outcome.
  4. (Optional) Add a brief conclusion about what you learned about the experiment
  5. (Optional) Toggle Conclude and deploy on. See Conclude and deploy.

  6. Click Conclude.

From the rule details

  1. Click on the rule.
  2. Click Conclude.
  3. Follow steps three through six from the previous From the ruleset section.

Conclude and deploy

If you toggled Conclude and deploy, complete the following:

  1. Select which variation to direct traffic to from the Variation drop-down list.
  2. (Optional) Adjust the Traffic allocation percentage.

  3. Click Deploy. 

Duplicate rule across environments

To duplicate a rule to another environment, click Add Rule, and select which environment you want to copy the rules from.

Collaborators outside of your organization cannot create multiple experiments per flag.

Copy a rule to other environments

To copy a rule to another environment,

  1. Click More options (...) on the rule.

  2. Select Copy to ENVIRONMENT_NAME. For example, Copy to Production.

  3. Click Copy to confirm.

The rule will be in unsaved Draft status when copied to the Production environment. You can make the necessary changes before clicking Save.

Manage flag rules

You can add, remove, or update your flag's rules anytime. However, you should pause your flag before making changes.

You should not change your experiment while it is running.

You can configure the following for each rule:

  • Rule name – Identify the rule. The rule key cannot change after creation.
  • Rule description – Summarize the rule's purpose for collaborators.
  • Hypothesis – Document the assumption the rule tests. See Add hypothesis to a rule.
  • Test plan – Link or upload the plan that describes how the rule runs. See Add test plan to a rule.
  • Targeted audiences – Choose which visitors qualify, based on audience conditions evaluated in order.
  • Traffic ramp percentage – Set the percentage of the targeted audience that enters the rule.
  • Associated metrics – Choose the metrics that measure the rule's success. The primary metric determines statistical significance.
  • Traffic distribution mode – Split traffic across variations manually, or let Stats Accelerator adjust it to reach significance faster.
  • Variations – Select the flag variations the rule serves, each a set of flag variable values.
  • Exclusion groups – Add the rule to a group so each visitor enters only one experiment in that group.

After you create a rule, you cannot change its key.

Be careful when you are changing experiment parameters.

Do not change your rule unless you are making the corresponding changes in your code. No traffic is sent to the experiment or variation if you use a key not referenced in your code.

To edit a flag rule, complete the following:

  1. Click the rule name to open it.
  2. Change your rule parameters as needed.
  3. Click Save.

Your project’s datafile updates within a few seconds. Depending on how often your application fetches the datafile, the update may take time to reflect in production.

If you want your rules to update in real-time, use webhooks to receive datafile updates.

Changing traffic allocation or group assignment for a running experiment impacts user bucketing, causing users to be bucketed in or unbucketed from this experiment.

For information, see How bucketing works.

Add hypothesis to a rule

A hypothesis records the assumption a rule tests, so collaborators can understand the experiment intent. The hypothesis is optional.

  1. Go to Flags > select your flag. Select the environment.
  2. Click the rule.

  3. Click Add hypothesis.

  4. Enter the expected outcome and reasoning in the Hypothesis field. Click Save Hypothesis.

  5. Click Save.

To update or delete the hypothesis, click More options (...), then select Edit or Delete. Changes display in the Change History.

Add test plan to a rule

A test plan records how a rule runs, including duration, success criteria, and rollout strategy. Attach the plan as a link or upload it as a file, so the documentation stays with the rule instead of in a separate tool. The field applies to every rule type and is optional.

To add a test plan:

  1. Go to Flags > select your flag. Select the environment.
  2. Click the rule.

  3. Click Add test plan.

  4. Select one of the following options:
    • URL and reference – Enter the plan link in the URL field and a label in the Reference field.
    • Upload from device – Select a file from your device.

  5. Click Save Test Plan.

  6. Click Save the rule.

The reference label or file name displays on the rule details page. To update or delete the plan, click More options (...), then select Edit or Delete. Changes display in the Change History.

The test plan fields accept the following values:

  • URL – Link to the test plan. The link text supports up to 2,000 characters.
  • Reference – Plain-text label for the plan. The limit is 100 characters.
  • Upload from device – Accepts .pdf, .doc, .docx, and .md files up to 10 MB.