Generate ideas for experimentation using indirect data

  • Updated
This article is part of The Optimization Methodology series.
This topic describes how to:
  • Build a business intelligence report to drive your personalization and A/B testing strategy
  • Use indirect data to generate strong hypotheses
  • Consider how to use multiple data sources together

Indirect data is industry expertise about your market space and site experience. Organizations often do not conceptualize indirect data as a form of business intelligence, but this data can provide valuable insights to drive the ideation process. By contrast, direct data is directly focused on your site experience; it tells you about a specific problem or opportunity to optimize your site experience.

Look to indirect data—blogs, academic studies, and competitor sites—for inspiration on how to tackle that opportunity. The industry perspective helps you contextualize all the site-specific intelligence you gather.

For example, imagine that your web analytics (direct data) reveal a high exit rate from your sales funnel. Based on the best practices recommended by an industry blog (indirect data), you might test whether extra links on the page distract your customers from completing the purchase. Indirect data, such as optimization best practices in this example, help make your direct data actionable.

Here are the most common sources of indirect (industry) data, discussed further below:

Add these data sources to your business intelligence report. Do not feel that you must consult all these sources equally; do use this list to consider how different streams of data can deepen your insights about how visitors behave.

Competitor sites

Used in moderation, a competitive analysis can serve as inspiration for ways to optimize your site. Perform a qualitative assessment of your competitors’ sites to better understand the market space and generate hypotheses for experiments and campaigns.

Use case

How this adds insight

Evaluate what your competitors do better than you and where their weaknesses lie.

What CTAs and messaging does your competitor use? Does your competitor show a preference for a certain type of value proposition or conversion mechanism?

Compare the most important, revenue-driving pages on your site to those on your competitor’s site. What differences do you notice? Are there any opportunities to test different experiences or personalize for certain audience segments?

Use third-party tools like EyeQuant or UserTesting to analyze your competitor’s site. Compare these reports to similar analyses of your own site to decide how to optimize the experiences you deliver.

Or, if your web analytics show a decline in mobile conversion rates, compare your own mobile experience with a competitor’s. Explore opportunities to deliver a better experience based on the device a visitor is using.

Best-in-class experiences

Look for sources of inspiration beyond your own market space. Experiences delivered by companies in other verticals can help you find inspiration for how to optimize your site experience.

Use case

How this adds insight

Best-in-class customer experiences help you rethink your own site design.

What are best-in-class companies like Amazon and Target doing to facilitate a superior shopping experience? How does their approach differ from your own? How does Netflix present a series of choices for visitors? How do native apps such as Facebook or iTunes address problems like discoverability? How does the New York Subway system use iconography and color to guide millions of riders per day?

Use your own analytics to ground your search for best-in-class experiences for ideation. Themes such as “best way to introduce your brand,” “the easiest way to sign up for something,” or “effective filtering for a complex database” help you focus your efforts.

Blogs and webinars

Blogs and webinars contain a wealth of information. Source hypotheses and best practices from experts who specialize in experience optimization, marketing, A/B testing, and your industry vertical.

Use case

How this adds insight

Consider the many case studies, industry discussions, and plug-and-play test ideas on Optimizely’s blog when sourcing ideas for your optimization strategy. Articles are often broken out by specific verticals and business goals. You can also find a series of Optimizely Experimentation webinars here.

Industry leaders such as Avinash Kaushik write extensively about best (and worst) practices in web analytics. Consult these resources when designing experiments, performing usability testing, or conducting a competitive review. Use themes to focus your research by searching for terms like “examples of ecommerce filter UX” or “ecommerce best practices.”

User groups

Many optimization professionals share knowledge at conferences such as MozCon or on online communities like the Optiverse. Aggregate these insights in your intelligence report so they can be socialized amongst your team and referenced regularly.

Use case

How this adds insight

Document top findings and interesting cases when you attend talks at events like MozCon or Opticon.

Look for case studies that your program can emulate or trends that help your team brainstorm strategies.

At industry events, you have the opportunity to think through current industry trends. Find the strongest trends and identify how competitors and best-in-class organizations are approaching them.

Compare this snapshot of your industry’s future with the shorter-term opportunities that you identify in your analytics reviews.

Academic literature

Academic studies and long-form writing about your industry and experience optimization can help build deeper expertise and provide a broader context for ideation.

These resources provide an excellent foundation for teams making a serious investment in experimentation:

Indirect data can be highly subjective. Strategies that work for one company do not necessarily translate to another. But it can provide a much-needed higher level view of the customer experience on your site.

Use indirect data to contextualize insights from your direct data so you can create strong hypotheses and a robust optimization roadmap.