Hypothesis creation walkthrough

Prerequisites

• An Optimizely Web Experimentation project with an A/B experiment set up. 
• Experiment Collaboration enabled in the project. See Connect Optimizely Web Experimentation with Experiment Collaboration.

Introduction

Follow this walkthrough guide to create a hypothesis and use a workflow in Experiment Collaboration to create, deploy, and analyze an A/B test in Optimizely Web Experimentation.

Imagine you are responsible for optimizing a website for an ecommerce business. You want to increase conversions, so you brainstorm some experiment ideas to do so. You discuss with your team and develop the following hypothesis:

• Problem – Users find the featured products on the homepage irrelevant. Only 12% of users click on them. 9 out of 11 users say they never found interesting products on the homepage. 
• Solution – Add a recommendations carousel that features products from recent categories the user has visited. If the user visits the category, they have expressed interest in those types of products.
• Result – Increase the percentage of users who click on the featured products from the carousel and add them to the cart.

Create a hypothesis in Experiment Collaboration

Work is accomplished through a hypothesis in Experiment Collaboration. Each hypothesis moves through a workflow from creation to completion. Workflows are flexible ways for your team to assign and track the steps to complete your experiment. 

Click + > Hypothesis from the global header. 

This opens the Create New Hypothesis page. 

1. Enter Featured products algorithm for the Title. 
2. (Optional) Select a campaign from the drop-down list.
3. Select today for the Start Date for when work begins on this experiment. Select a Due Date for when the experiment is completed and results shared.
4. Leave Workflow and Fields blank. You can add them later.
5. Click Create.

Configure your hypothesis

After creating your hypothesis, it is time to configure it. Update each section accordingly:

Brief

The hypothesis brief is a place to upload files and document your hypothesis. This facilitates collaborative brainstorming by ensuring everyone on your team has an essential and shared understanding of the experiment, including key objectives and target audiences, before beginning their work. 

1. Click Select Brief. Select Brief Template.
2. Select Experiment Test Design.
3. Click Next.
4. (Optional) Update the Title.
5. Enter the following for the Hypothesis:
   • Problem – Users find the featured products on the homepage irrelevant. Only 12% of users click on them. 9 out of 11 users say they never found interesting products on the homepage. 
   • Solution – Add a recommendations carousel that features products from recent categories the user has visited. If the user visits the category, they have expressed interest in those types of products.
   • Result – Increase the percentage of users who click on the featured products from the carousal and add them to the cart.
6. (Optional) Enter additional information about the hypothesis in the Problems this solves and Business Goal fields.
7. Click Create Brief.

Variations

Variations let you propose and brainstorm different variation designs and gather insights and feedback. You can create multiple variations of your content or application, letting you test different ideas and compare their performance against each other and the baseline or unchanged variation. You can iterate and optimize your strategies based on real-time data and user feedback. 

1. Select the Variations tab.
2. Click Add Variation.

3. Select Text Editor (Classic) from the drop-down list.

   

4. Click Add Title. Enter Homepage featured products. 

5. Enter the following into the text editor:

   • Baseline (original variation) – The homepage displays all featured products.
   • Variation – The homepage displays featured products from categories the user has recently visited.

6. Click Done.

   

Experiment

The Experiment tab lets you link an experiment in Optimizely Web Experimentation to a hypothesis in Experiment Collaboration. Linking your experiment automatically transfers information about the experiment in Web Experimentation to Experiment Collaboration. 

1. Click Experiments.
2. Click Link Experiment. 
3. Select the Optimizely Web Experimentation project created for this walkthrough.
4. Click Next.
5. Select the Experiment to link.
6. Click Link.

Fields

Fields represent the metadata of Experiment Collaboration. You can use them to organize your work and structure for classification. Reuse fields to filter through hypotheses and activities.

1. Click Fields. 
2. Click Add Field. 
3. Click Create new. 
4. Select Rich Text in the Field Type drop-down list.
5. Enter Business goal in the Field Name field. 
6. Click Create.
7. The newly created field is automatically selected in the Add field multi-select list. 
8. Click Add.

Your new field is added to the hypothesis.

1. Click on the Business goal field, and enter Increase the percentage of users who click on the featured products list, which will increase the number of purchases. 
2. Press enter or click outside of the Business goal field. Your changes are automatically saved.

History

The hypothesis history serves as a comprehensive activity log, meticulously documenting all modifications and actions within the hypothesis. This feature ensures seamless end-to-end visibility, offering insight into the evolution of the hypothesis over time. By displaying a chronological record of who made specific changes and providing context regarding the rationale behind each modification, the History tab facilitates transparency and accountability, enabling users to trace the progression of ideas and decisions within the experimentation process.

Create your experiment workflow

After configuring your hypothesis, it is time to create your experiment and collaborate with your team using a workflow. Workflows let you systematically document and streamline the experimentation process. As you move through the different steps of your experiment, such as designing, building, testing, deploying, and gathering results, update the workflow. Your team can see what step each hypothesis is at and their due dates.

1. Click 1. Enter a step name - E.g. Research.

2. Enter Hypothesis approval. Press Enter.

   

3. After adding a step title, you can Add step due date or Add assignee. 
4. Click 2. Enter a step name - E.g. Rough Draft. 
5. Enter Ready to launch experiment. Press Enter.
   1. Click more options and select Add sub-step. 
   2. Enter Configure A/B experiment. Press Enter.
   3. Click Add Sub-Step.
   4. Enter Start A/B experiment. Press Enter.
6. Click Add Step. Enter Stop and analyze A/B experiment. Press Enter.

Your Workflow should look similar to the following image:

As you work through your experiment life cycle, you can assign people and due dates to steps. You can update the step's status, such as In Progress or Completed. You can also Skip steps.

Conclusion

You have created a hypothesis and a hypothesis workflow using Experiment Collaboration. You can expand this walkthrough guide based on your experimentation process needs. Using pre-built fields and workflows, you can also streamline the hypothesis creation process.