Integrate Optimizely Feature Experimentation into the product development lifecycle

Product teams often risk investing time, effort, and money into building features that customers do not use. A new feature may not deliver business value or meet the needs of customers. Experimentation gives several advantages for building and iterating on features:

• Use real-world data to discover how your customers engage with features.

• Deliver functional prototypes of features early to learn about their potential value.

• Roll out features gradually to mitigate risk.

• Use conversion rate optimization and growth experiment techniques.

Use Optimizely Feature Experimentation's Feature Management capabilities to drive adoption, engagement, and conversion for features at any stage of the product development process.

Painted door tests commonly validate demand and gather data before committing resources to specific projects. Companies with closer integration between experimentation and product development like creating minimum viable product experiments and engaging in feature rollouts. They use experimentation to build data and fine-tune controls in the product development process.

Discover what your customers want

To integrate experimentation into your product development lifecycle, learn what your users want. Many teams think they know the answer, but those answers are not always based on data.

To see what feature offerings attract your users, use a painted door test to create the suggestion of a feature rather than building out the feature. This shows you how many users may use it before you invest resources in its development. Painted door tests are also useful for assessing discoverability.

Painted door tests are usually used in Optimizely Web Experimentation and not Optimizely Feature Experimentation. However, the test validates investment in a higher-cost, higher-fidelity Optimizely Feature Experimentation feature.

For example, if Optimizely believed some customers might be interested in an automated personalization feature, the company could run a painted door test comparing different approaches for naming and describing this feature.

In the process, Optimizely learns several important things:

• Is there enough interest in the feature to justify its development?

• Which customers have use cases that this feature can address?

• Which approach to naming and description drives engagement better?

Painted door tests let you gather data from users and validate the features you decide to build with low investment.

Design features and test them

You may want to use quick and efficient experiments to evaluate a feature idea before allocating resources to develop a comprehensive version. A minimum viable product (MVP) test is when you build a basic, barely functioning version of a feature you are considering. It is similar to a feature rollout in Optimizely Feature Experimentation, but the feature is not expected to be finished.

For example, a media company might consider a feature that lets readers share the site’s content with their contacts. With an MVP test, they can build it out for whichever platform would be easiest, whether mobile, web-only, or email. The idea is to engage users with the feature to validate the idea that people will use it, but the test also delivers insights on how and how often they are most likely to use it.

Build features

Once you collect data on your users' needs and preferences and have a design in mind, the next step is to build your feature in Optimizely Feature Experimentation. This means setting up feature flags that let you roll out a new feature when you are ready or switch it off if something goes wrong and a set of variables that define how the feature looks and acts.

For information on feature flags, see the developer documentation.

Run experiments

If you have not developed a design for your feature because your data did not find a single solution for your users' problems or you want to run multiple designs against each other, you can use a feature test. They are similar to A/B tests in Optimizely Web Experimentation but include a few components specific to Optimizely Feature Experimentation.

Use them to experiment with different versions of your feature to see which performs best or to measure its performance against the current experience. Feature tests provide a better understanding of the feature's potential impact on your key metrics without you deploying any code. You can also use feature tests to iterate on failed features until they meet your organization’s standards for launch.

Launch and roll out new features

Once your feature is built and you select the variation you want to use, you can roll it out to your customers.

If you are concerned that your feature may degrade the visitor experience instead of improving it, you can launch it to a small percentage of visitors and then observe the results. Feature rollouts in Optimizely Feature Experimentation use feature targeting to control which users access new features and when.

Use targeted rollouts to provide beta access to new features or to experiment with features internally before a public rollout. A slower, controlled rollout mitigates risk by limiting the impact of any newly surfaced bugs that may have slipped through an internal QA process.

You can also use a rollout to launch winning variations you identify through experimentation. Show a winning feature variation to your visitors in a controlled fashion. You can set feature visibility based on a specified environment and user characteristics like subscription tier, authentication state, region, or language and manage the process on your terms.

Iterate and improve

If you roll out your feature to all your customers and adoption is high, you can continue to create improvements. For example, after your feature launches, you may notice something unexpected about how your customers use it. You may want to iterate on your new feature to improve the metric you use to gauge success and roll those changes out quickly—without waiting for the next code deployment.

Use feature configurations to make changes and create variations through Optimizely without deploying code. This often involves running feature tests to determine the optimal combination of variable values, setting those values as your default feature configuration, and launching with a rollout.

See the developer documentation for more in-depth information.