Run Feature Rollouts in Feature Experimentation

  • Updated
  • Optimizely Feature Experimentation

Feature Rollouts is currently in beta. Contact your Customer Success Manager or sign up for the beta on Optimizely.com.

Feature Rollouts are a rule type in Optimizely Feature Experimentation that combine the simplicity of Targeted Deliveries with the measurement power of A/B tests. Use Feature Rollouts to release a single variation to production progressively while you track impressions, conversions, and business metrics.

Feature Rollouts give you full impact analytics, metric tracking, and flag evaluation insights for feature releases that are not designed as experiments. A typical use case: an A/B test concludes with a winning variation, and you want to release that winner gradually while you monitor its effect on key business metrics. Rather than create a single-variation A/B test as a workaround, you create a Feature Rollout. The rule type provides measurement, allowlists, evaluation-order control, and impression tracking.

When to use Feature Rollouts

Use Feature Rollouts when you:

  • Release a winning variation from a concluded experiment to production traffic.
  • Roll out a feature progressively (10%, 25%, 50%, then 100%) and measure impact at each stage.
  • Release a feature to a defined audience, such as beta users or a region, and track performance.
  • Pre-test a release with an internal allowlist before broader traffic exposure.

Configuration overview

To configure a Feature Rollout, complete the following:

  1. (Prerequisite) Create a flag in your Feature Experimentation project.
  2. (Prerequisite) Handle user IDs.

You should configure a user profile service to ensure consistent user bucketing if you are using a server-side SDK.

  1. Create and configure a Feature Rollout rule.
  2. If you have not done so yet, implement the Optimizely Feature Experimentation SDK's Decide method in your application's codebase through a flag.
  3. Test your Feature Rollout rule in a development environment. See Test and troubleshoot.
  4. Discard any test user events and enable your Feature Rollout rule in a production environment.

Compare Feature Rollouts, Targeted Deliveries, and A/B tests

The following table compares the three rule types across capabilities that matter when you choose between them:

Capability Targeted Delivery Feature Rollout A/B test
Number of variations One One Two or more
Impression tracking No Yes Yes
Metrics and analytics No Yes Yes
Allowlist for pre-testing No Yes Yes
Configurable evaluation order No, always last Yes Yes
Statistical analysis Not applicable Single-variation results with confidence intervals Multi-variation comparison with significance testing
Consumes monthly active users (MAU) and impressions No Yes Yes
Best for Simple toggles where measurement is not required Progressive release of a chosen variation with measurement Comparison of multiple variations to determine a winner

Create a Feature Rollout

Create a Feature Rollout from a winning experiment variation or build one from scratch on a flag. Both paths produce the same rule type with the same configuration options.

  1. Go to Flags, select your flag, and select your environment.

  2. Click Add Rule and select Feature Rollout.

  3. Enter a Name for the rule.
  4. The Key field auto-populates from the Name. You can optionally update it.
  5. (Optional) Click Add description to add a description of the feature rollout.
  6. (Optional) Search for and add audiences. To create an audience, see Target audiences. Audiences evaluate in the order in which you drag and drop them. You can choose whether to match each user on any or all of the audience conditions.
  7. Set the Traffic Allocation to assign a percentage of your audience to bucket into the experiment.
  8. Add Metrics based on tracked user events. See Manage events in Feature Experimentation for information on how to create events using the UI or Create events for how to use the Feature Experimentation REST API.
  9. Select the variation to deploy.
  10. Set the traffic allocation. For example, 10%, the recommended starting point.

  11. (Optional) Click Allowlist: Force up to 50 users into any variation(s) and enter the User ID. See Allowlisting.

  12. Click Save.

Click Run on the rule to start the Feature Rollout. If the ruleset (flag) is not running, click Run on it.

Permissions and roles

  • Create or modify a Feature Rollout – requires the manage_production or manage_flags role.
  • View rollouts and analytics – requires the view_flags role for read-only access.

Optimizely records all Feature Rollout create, read, update, and delete operations in change history for audit purposes.

Frequently asked questions

What is the difference between Feature Rollouts and Targeted Deliveries?

Feature Rollouts provide full analytics and metric tracking. They consume monthly active users (MAU) and impressions and require an Enterprise or Growth plan. Targeted Deliveries are free, do not consume MAU or impressions, and provide no analytics. Use Feature Rollouts when you need to measure impact. Use Targeted Deliveries for simple toggles where measurement is not required.

What is the difference between Feature Rollouts and A/B tests?

Feature Rollouts support one variation and target progressive deployment of a chosen variation. A/B tests support multiple variations to determine a winner. Use a Feature Rollout when you already know what you are deploying. Use an A/B test when you need to compare options.

Do Feature Rollouts consume MAU and impressions?

Yes. Feature Rollouts fire impression events the same way A/B tests do, so they count toward your MAU and impression limits. Targeted Deliveries remain free for cases where you do not need analytics.

Which SDK versions support Feature Rollouts?
  • JavaScript – v6.4.0 or later.
  • React – v3.5.0 or later.
  • Python – v5.5.0 or later.
  • Java – v4.4.0 or later.
  • C# – v4.3.0 or later.
  • Go – v2.4.0 or later.
  • Ruby – v5.3.0 or later.
  • Swift – v5.3.0 or later.
  • Android – v5.2.0 or later.
  • Flutter – v3.5.0 or later.
Do Feature Rollouts slow down flag evaluations?

No. Feature Rollouts use the same evaluation logic as A/B tests and Targeted Deliveries with negligible performance impact, less than one millisecond per evaluation. Datafile size grows by approximately 200 bytes per rollout.

Can I use Feature Rollouts in multiple environments?

Yes. Feature Rollouts are environment-specific. Configure separate rollouts in development, staging, and production. For example, run at 100% in development and at 10% in production.

Does enabling Feature Rollouts affect existing flags or rules?

No. Targeted Deliveries continue to work as before. A/B tests work without change. Existing flags continue to evaluate correctly. Feature Rollouts are a separate rule type and do not change the behavior of any rule type you already use.