Best practices: From research to hypothesis creation

Bring key insights to the meeting, not raw data. Sometimes, brainstorming sessions get sidetracked by undigested data; you spend time reviewing the metrics, and debating how to interpret or apply them to the site. Suddenly, creative momentum for ideation evaporates.

Analyze your data first to avoid this result. Find pain points, anomalies, and interesting moments. For example, bring the following data to your session: 

• Top trafficked pages

• Conversion CTAs on your top pages

• Top 3-5 user flows

• Drop-offs at each stage of those user flows

• Bounce rate on each page of the site, to understand where your biggest opportunities are

Use the insights above and your business intelligence report to focus brainstorming on the important opportunities on your site.

Create an idea submission form

An idea submission form is a short form or survey that employees around your companies can use to submit their ideas for testing and personalization. Many organizations use a submission form to gather different perspectives, socialize data-driven optimization across the company, and create a pool of ideas for the team to choose from.

Ready to create your own? Check out this article for tools to create an idea submission form.

DRAW ON A BROAD TEAM

Bring diverse perspectives into the room when you brainstorm test ideas. Stakeholders from different parts of the company can help identify new problems to test through and provide fresh takes on familiar parts of the site.

The optimization program manager should assemble:

• Stakeholders who are directly involved in testing

• Analysts who work with your site’s data

• Developers involved in creating new features

• Designers who create new assets for the site

• People at the company who see your visitors at a different stage of the customer journey

Let’s talk about that last one. Visitors come to your site at a certain point on the customer journey. What parts of the company do they see or interact with before and after? A holistic view of the customer life cycle can feed new questions into your brainstorming process.

For example, if some visitors are driven to your site through an email campaign or Google ad, your SEO or content marketing manager may have insights about effective value propositions to carry through on the site. Or, customers may purchase on your site and then work with an account manager. The account manager, whose key priorities are retention and loyalty, can help brainstorm ideas to optimize for repeat purchases.

MEET IN PERSON

In-person meetings promote cross-functional context sharing and productive debate: a rich environment for great ideas.

MEET MONTHLY

Or, meet on a frequent and regular cadence. For some mature optimization programs, the ideation session also presents an opportunity for a brief retrospective on experiment results. Monthly meetings will help you feed what you’ve learned into the ideation process and prioritize areas of the site to test next.

PRIORITIZE OPPORTUNITIES

Bring a prioritized list of the pages on your site that make the biggest impact on your business goals. Frame your brainstorming session to focus on each of these pages in order of priority.

GENERATE 5-10 QUESTIONS

Focus each session on generating approximately 5-10 questions that you seek to evaluate through testing.

Customers excited about testing often jump straight into hypothesis generation or even test creation. Experiments created this way may help to alleviate certain frustrations on the site, but they often fail to get at the root cause: assumptions about visitors or other discontinuities that surface in the site design. Instead, redirect that enthusiasm to ask productive questions and probe untested assumptions about your site. Consider your business model, your visitors, and your product roadmap to find opportunities to optimize.

At the end of your ideation session, you should have:

• A focused set of questions or assumptions that you seek to evaluate through testing or personalization. Organize this list by location, or where the test or campaign would be focused (for example, the home page, product details pages, or the checkout funnel).

• A description of the visitors you’re focused on. Are you evaluating first-time visitors to the site? Customers, who make large purchases? Visitors, who use the search function? Identify specific audiences you’ll target when you test or personalize the site.

DEFER JUDGMENT

Focus on generating questions, not answers. When brainstorming, participants shouldn’t feel pressured to design tests or present changes. Explore the “why.”

When you revise your idea into a complete hypothesis and create an experiment plan, you’ll think more carefully about what to change in the site experience to optimize for certain visitor behaviors.