The Optimization Methodology

Optimizely Experimentation lets you experiment with and personalize your website and applications. There are five major stages in the Optimization Methodology:

1. Ideation
2. Planning
3. Development
4. Analysis
5. Implementation

Your team works through each stage in every experiment you run. This iterative cycle helps you learn and improve your site experience to drive business goals.

The articles below help you plan more effectively through each stage. When you analyze results in production (stage 5), you can monitor how improvement in one test affects other valuable revenue sources on your site because you set up monitoring goals in design (stage 3). You can also turn data in your results into the next great idea to test (stage 2).

This series provides:

• Strategic recommendations for building a powerful, sustainable program.
• Actionable templates to help you start prioritizing, designing, and sharing your experiment and campaigns.
• Resources to help teams with varying experience levels build a strong, data-driven culture.

Stage 1 – Implement Optimizely Experimentation and establish your experimentation program

First, establish an optimization framework that prepares your team for long-term success. Complete this phase once and revise only based on significant program or company-level changes. Consult the resources often to orient your efforts.

• Build a goal tree to improve metrics that matter with your optimization program.
• Guide your program with an experimentation charter.
• Build an effective optimization team.
• Boost your program with an Optimizely Experimentation Solutions Partner.
• Follow the basic and advanced implementation checklist for Optimizely Web Experimentation.

Stage 2 – Decide what to test

When you set the direction for your program, research and brainstorm ideas for testing and personalization.

• Use a business intelligence report to ask the right questions.
• Generate ideas for experimentation based on direct and indirect data.
• Generate ideas based on basic and advanced analytics reporting. 
• Follow the Best practices: From research to hypothesis creation.
• Design an effective hypothesis.

Stage 3 – Organize your experiments

Prioritize, plan, and design individual tests.

• Create a basic prioritization framework.
• Create an experimentation roadmap.
• Use minimum detectable effect when designing an experiment.
• Use minimum detectable effect (MDE) to prioritize a test.
• Create a basic or advanced experiment plan.
• Set primary, secondary, and monitoring goals.
• Create Common metrics by revenue model.

Stage 4 – Build and run your experiment

Create your experiment in Optimizely Web Experimentation, Optimizely Performance Edge, or Optimizely Feature Experimentation. Before you make it live, QA to ensure it is set up how you want, then launch it.

• Follow the Six steps to creating an experiment in Optimizely Web Experimentation or Performance Edge or Run A/B tests in Optimizely Feature Experimentation.
• Create an advanced experiment plan and QA checklist.
• Launch and monitor your experiment.

Stage 5 – Analyze and take action on your results

Analyze your results and take action based on what you find. Use the insights you generated from winning, losing, and inconclusive tests to design the next round of tests and campaigns.

• Interpret your Optimizely Experimentation results.
• Take action based on your results of an experiment.
• Share your results with stakeholders.
• Implement wins on your production site.

Every optimization team is different. As your program grows, the iterative cycle from ideation to production helps you adapt your process to meet the changing needs of your testing organization.