- Lead a productive brainstorming session
- Generate a list of hypotheses for experiments and personalization
Generating a powerful, data-informed hypothesis is one of the most important steps in experience optimization. Sometimes, teams eager to make an impact jump straight into hypothesis creation, or even experiment design. But the best, most effective experiments start with questions.
Curiosity and a knack for asking meaningful questions help an experimentation program focus on impactful changes. Great questions capture points of uncertainty about your site or probe long-held assumptions. Which value proposition is the most compelling for a particular segment? What's the best way to ask visitors to fill out a form? Turn these questions into strong hypotheses that you'll evaluate through testing.
The more tests your team runs, the more ideas they should get for new tests. Data should be generative, not conclusive.
- Sara Critchfield, Founding Editor of Upworthy and author of "How to push your team to take risks and experiment," Harvard Business Review
- Idea submission form (always-on for external contributors)
- Business intelligence report
- Goal tree
People and resources
- Program manager (facilitator)
Actions you'll perform
- Meet with a broad team
- Translate data, observations, and insights into questions
- Turn questions into hypotheses
- Set a goal for the number of new ideas
- Document ideas
- List of hypothesis statements, including:
- Experiment name
- Primary and secondary goals
- Variation description
- List of hypothesis statements, including:
- Producing only a small number of ideas
- Producing ideas that are very similar (not creating a variety of tactics)
- Producing ideas that are either too similar or too difficult, on average: an unbalanced list.
- Producing a list that focuses on one or two pages of the site, instead of the full funnel
- This article is a part of the Optimization Methodology series.
Prepare with research
Feed your ideation process with a business intelligence report and a product roadmap. The results of previous experiments, analytics about your site, industry research, and your product roadmap provide a detailed view of customers’ experiences and expectations.
Use your research to probe assumptions about your site experience and generate questions you’ll answer through testing.
Bring key insights to the meeting, not raw data. Sometimes, brainstorming sessions get sidetracked by undigested data; you spend time reviewing the metrics, and debating how to interpret or apply them to the site. Suddenly, creative momentum for ideation evaporates.
Analyze your data first to avoid this result. Find pain points, anomalies, and interesting moments. For example, bring the following data to your session:
Use the insights above and your business intelligence report to focus brainstorming on the important opportunities on your site.
Create an idea submission form
An idea submission form is a short form or survey that employees around your companies can use to submit their ideas for testing and personalization. Many organizations use a submission form to gather different perspectives, socialize data-driven optimization across the company, and create a pool of ideas for the team to choose from.
Ready to create your own? Check out this article for tools to create an idea submission form.
Here are a few guidelines for productive brainstorming.
DRAW ON A BROAD TEAM
Bring diverse perspectives into the room when you brainstorm test ideas. Stakeholders from different parts of the company can help identify new problems to test through and provide fresh takes on familiar parts of the site.
The optimization program manager should assemble:
Let’s talk about that last one. Visitors come to your site at a certain point on the customer journey. What parts of the company do they see or interact with before and after? A holistic view of the customer life cycle can feed new questions into your brainstorming process.
For example, if some visitors are driven to your site through an email campaign or Google ad, your SEO or content marketing manager may have insights about effective value propositions to carry through on the site. Or, customers may purchase on your site and then work with an account manager. The account manager, whose key priorities are retention and loyalty, can help brainstorm ideas to optimize for repeat purchases.
MEET IN PERSON
In-person meetings promote cross-functional context sharing and productive debate: a rich environment for great ideas.
Or, meet on a frequent and regular cadence. For some mature optimization programs, the ideation session also presents an opportunity for a brief retrospective on experiment results. Monthly meetings will help you feed what you’ve learned into the ideation process and prioritize areas of the site to test next.
Bring a prioritized list of the pages on your site that make the biggest impact on your business goals. Frame your brainstorming session to focus on each of these pages in order of priority.
GENERATE 5-10 QUESTIONS
Focus each session on generating approximately 5-10 questions that you seek to evaluate through testing.
Customers excited about testing often jump straight into hypothesis generation or even test creation. Experiments created this way may help to alleviate certain frustrations on the site, but they often fail to get at the root cause: assumptions about visitors or other discontinuities that surface in the site design. Instead, redirect that enthusiasm to ask productive questions and probe untested assumptions about your site. Consider your business model, your visitors, and your product roadmap to find opportunities to optimize.
At the end of your ideation session, you should have:
Focus on generating questions, not answers. When brainstorming, participants shouldn’t feel pressured to design tests or present changes. Explore the “why.”
When you revise your idea into a complete hypothesis and create an experiment plan, you’ll think more carefully about what to change in the site experience to optimize for certain visitor behaviors.
Create a hypothesis
After your brainstorming session, turn your questions into measurable hypotheses.
A strong hypothesis is the heart of data-driven optimization. Hypothesis statements help you turn a wealth of data and insights into focused proposals that you will take action on.