Best practices: From research to hypothesis creation

  • Updated
This topic describes how to:
  • Lead a productive brainstorming session
  • Generate a list of hypotheses for experiments and personalization
This article is part of The Optimization Methodology series.

Generating a powerful, data-informed hypothesis is one of the most important steps in experience optimization. Sometimes, teams eager to make an impact jump straight into hypothesis creation, or even experiment design. But the best, most effective experiments start with questions.

Curiosity and a knack for asking meaningful questions help an experimentation program focus on impactful changes. Great questions capture points of uncertainty about your site or probe long-held assumptions. Which value proposition is the most compelling for a particular segment? What's the best way to ask visitors to fill out a form? Turn these questions into strong hypotheses that you'll evaluate through testing.

The more tests your team runs, the more ideas they should get for new tests. Data should be generative, not conclusive.

- Sara Critchfield, Founding Editor of Upworthy and author of "How to push your team to take risks and experiment," Harvard Business Review

See requirements
Materials to prepare
People and resources
    • Program manager (facilitator)
    • Designer
    • Marketers
    • Analysts
    • Merchandiser
    • Developer

Actions you'll perform 
    • Meet with a broad team
    • Translate data, observations, and insights into questions
    • Turn questions into hypotheses
    • Set a goal for the number of new ideas
    • Document ideas
Deliverables What to watch out for
    • Producing only a small number of ideas
    • Producing ideas that are very similar (not creating a variety of tactics)
    • Producing ideas that are either too similar or too difficult, on average: an unbalanced list.
    • Producing a list that focuses on one or two pages of the site, instead of the full funnel
  • This article is a part of the Optimization Methodology series.


Prepare with research

Feed your ideation process with a business intelligence report and a product roadmap. The results of previous experiments, analytics about your site, industry research, and your product roadmap provide a detailed view of customers’ experiences and expectations.

Use your research to probe assumptions about your site experience and generate questions you’ll answer through testing.

Bring key insights to the meeting, not raw data. Sometimes, brainstorming sessions get sidetracked by undigested data; you spend time reviewing the metrics, and debating how to interpret or apply them to the site. Suddenly, creative momentum for ideation evaporates.

Analyze your data first to avoid this result. Find pain points, anomalies, and interesting moments. For example, bring the following data to your session: 

  • Top trafficked pages

  • Conversion CTAs on your top pages

  • Top 3-5 user flows

  • Drop-offs at each stage of those user flows

  • Bounce rate on each page of the site, to understand where your biggest opportunities are

Use the insights above and your business intelligence report to focus brainstorming on the important opportunities on your site.

Create an idea submission form

An idea submission form is a short form or survey that employees around your companies can use to submit their ideas for testing and personalization. Many organizations use a submission form to gather different perspectives, socialize data-driven optimization across the company, and create a pool of ideas for the team to choose from.

Ready to create your own? Check out this article for tools to create an idea submission form.


Here are a few guidelines for productive brainstorming.


Bring diverse perspectives into the room when you brainstorm test ideas. Stakeholders from different parts of the company can help identify new problems to test through and provide fresh takes on familiar parts of the site.

The optimization program manager should assemble:

  • Stakeholders who are directly involved in testing

  • Analysts who work with your site’s data

  • Developers involved in creating new features

  • Designers who create new assets for the site

  • People at the company who see your visitors at a different stage of the customer journey

Let’s talk about that last one. Visitors come to your site at a certain point on the customer journey. What parts of the company do they see or interact with before and after? A holistic view of the customer life cycle can feed new questions into your brainstorming process.

For example, if some visitors are driven to your site through an email campaign or Google ad, your SEO or content marketing manager may have insights about effective value propositions to carry through on the site. Or, customers may purchase on your site and then work with an account manager. The account manager, whose key priorities are retention and loyalty, can help brainstorm ideas to optimize for repeat purchases.


In-person meetings promote cross-functional context sharing and productive debate: a rich environment for great ideas.


Or, meet on a frequent and regular cadence. For some mature optimization programs, the ideation session also presents an opportunity for a brief retrospective on experiment results. Monthly meetings will help you feed what you’ve learned into the ideation process and prioritize areas of the site to test next.


Bring a prioritized list of the pages on your site that make the biggest impact on your business goals. Frame your brainstorming session to focus on each of these pages in order of priority.


Focus each session on generating approximately 5-10 questions that you seek to evaluate through testing.

Customers excited about testing often jump straight into hypothesis generation or even test creation. Experiments created this way may help to alleviate certain frustrations on the site, but they often fail to get at the root cause: assumptions about visitors or other discontinuities that surface in the site design. Instead, redirect that enthusiasm to ask productive questions and probe untested assumptions about your site. Consider your business model, your visitors, and your product roadmap to find opportunities to optimize.

At the end of your ideation session, you should have:

  • A focused set of questions or assumptions that you seek to evaluate through testing or personalization. Organize this list by location, or where the test or campaign would be focused (for example, the home page, product details pages, or the checkout funnel).

  • A description of the visitors you’re focused on. Are you evaluating first-time visitors to the site? Customers, who make large purchases? Visitors, who use the search function? Identify specific audiences you’ll target when you test or personalize the site.


Focus on generating questions, not answers. When brainstorming, participants shouldn’t feel pressured to design tests or present changes. Explore the “why.”

When you revise your idea into a complete hypothesis and create an experiment plan, you’ll think more carefully about what to change in the site experience to optimize for certain visitor behaviors.

Create a hypothesis

After your brainstorming session, turn your questions into measurable hypotheses.

A strong hypothesis is the heart of data-driven optimization. Hypothesis statements help you turn a wealth of data and insights into focused proposals that you will take action on.