Implement wins on your production site

  • Updated
This topic describes how to:
  • Implement the changes from a successful variation permanently on your site 
  • Involve a developer in your optimization process
This article is part of The Optimization Methodology series.

At the end of your testing cycle, you will decide which changes to implement permanently on your site. When you decided on a winning variation, you likely sent all traffic to that experience by pausing your running experiment, duplicating it, and allocating all traffic to the winning experience. Next, you will work with a developer to integrate these changes permanently into your site's codebase.

Sometimes, the moment of the handoff to a developer can be a delicate one. If the optimization team has not prepared the developer for implementation, the project may hit a slow point or even be de-prioritized. For example, a new feature may test well—but take a month of your developer's time to build.

By looping your developers in early, you can smooth the process of implementing permanent changes from testing and build a stronger optimization culture. Read more about how and when to integrate developers into your testing cycle.

Materials to prepare
  • Code from Optimizely variations
  • Documented changes from your roadmap
  • Identify all new functionality of the variation
  • Identify regression test cases where changes may break functionality on your site
  • Identify upcoming site changes
People and resources
  • Developers
  • QA team
  • Stakeholders
Actions you will perform
  • Verify that new code functions correctly
  • Verify changes for regression testing
  • Update production environment
  • Update staging environment
  • Notify QA team, Marketing, Support, and Engineering
Deliverables
  • Updated versions of pages on the developer environment
  • The updated version of pages on the production environment
Watch out for
  • Not sharing about new site changes that might affect other experiments with the entire team

This article is a part of the Optimization Methodology series.

What the developer should know

To implement the changes you decide to make, your developer will translate Optimizely Web Experimentation's jQuery variation code into the framework your site is built with. In the beginning, you might be able to add Optimizely Web Experimentation's variation code directly to your site, but over time this process is not sustainable when you implement changes from multiple experiments. A developer will translate the variation code and integrate it into a part of your site’s framework.

This process requires familiarity with both your site’s codebase and Optimizely Web Experimentation. It may also require the developer to translate some of Optimizely Web Experimentation's client-side script into a server-side script. Help smooth your developer’s path by providing context around Optimizely and the experiment you ran.

A few things your developer should know about Optimizely Web Experimentation:

Also, share your results document with the developer. A detailed plan provides context for the changes that will be implemented.

Your results document also provides a glimpse into the power of Optimizely Web Experimentation to create a quick proof-of-concept before fully dedicating developer resources. Only site changes that have been tested and validated with data are passed to the development team for implementation.

Integrate your communications about the site build with your developers’ communication systems—often JIRA or Trello. Create an unified record so potential communication gaps—such as the overlapping deployment of live features and experiments—do not cause conflicts that break the site.

The workflow might look something like this:

  1. Create a test based on your experiment plan, with your developer’s input.

  2. Analyze the results of a test and make decisions based on your results.

  3. If a variation is successful, duplicate the experiment and set traffic allocation to 100% to show all visitors.

  4. Share results with stakeholders, including the developer.

  5. The developer enters the Optimizely Web Experimentation experiment and reviews the variation code with the project manager.

  6. The developer abstracts the code, builds a permanent feature in the production environment, and then verifies that the new feature works correctly.

  7. The developer notifies the optimization program manager when the feature goes live.

  8. The optimization program manager turns the experiment off and archives it.

  9. Implement the same feature in your development environment if you have one.

Integrate your implementation process with your developer's systems to track projects and execute the handoff smoothly.

Get a developer involved early

As your optimization program matures, start incorporating your testing program with your developer’s sprint cadence. Involve the developer in the testing cycle too.

As you do so, you may notice a decrease in the time it takes to translate successful changes to your live site. But the effort it takes to set up a new test may slightly increase. These slight shifts are good signs: your developer is becoming more deeply involved in testing. You are probably running more complex experiments, too.

And by involving your developer early, you smooth the way for the last step—optimizing your live site. Optimizely Web Experimentation makes creating variations with static HTML easy, but a developer can help you design more efficient dynamic experiments. Changes you make with the Visual Editor generate code executed in the browser after your site loads. Inefficient changes can balloon into hundreds or even thousands of lines of variation code.

For example, you may want to make a button more prominent, so you use the Move and Resize feature and edit the color a few times to get it right. With every change, a new line of code is generated. Hundreds of lines of code can cause performance issues such as flashing.

You can mitigate this issue by following a few best practices, but a developer can optimize the variation code for the best performance. They can hand code part or all of the variation and reference the codebase with JavaScript to create a dynamic experiment—such as replacing a module on the page or changing the experience as a visitor clicks around.

A developer can also think ahead about abstracting variation code into the code base. Then, when you decide to deploy changes to the live site, they can build new or altered features more efficiently.