Testing ideas for media and publishing sites

  • Updated
This topic describes how to:
  • Come up with ideas for your media or publishing site A/B tests
  • Design tests based on these ideas

This article provides A/B test ideas for elements commonly found on media and publishing sites. You may want to run these experiments using Optimizely Web Experimentation on your site. For each example, we provide a common default example and an alternate concept, with ideas of what to test and how to measure.

How to use this guide

Consider this article a starting point for your optimization efforts. After all, experience optimization is a journey, not a destination. The tests you run help you discover what works (or does not) for your visitors, so it is important to pair the ideas in this guide with research on your site or app's true opportunities and obstacles.

Do:

  • Use this guide as a starting point to identify opportunities for your site or app
  • Use the ideas from this guide that help you answer key questions about your users or move key metrics
  • Pair the ideas in this guide with data from your analytics platforms and user feedback, so you run tests that explore identified issues

Do not:

  • Use this guide without also looking at your qualitative and quantitative data
  • Try to test every idea in this guide
  • Use these test ideas verbatim without thinking about how to adapt them to your experience

Key metrics for media and publishing sites

Learn more about setting primary and secondary goals.

Home page testing opportunities

Your home page is the entry point to your site and it sets the tone for your brand's value and site's offering. It has a high traffic volume and will more quickly collect data. We will discuss several types of tests to run on your home page's navigation, as well as content encouraging return visits.

Allowing users to browse by category without forcing them to click into that category can increase engagement rates and get users to consider content they may not normally go to. Hover states are a great way to attract users' attention to something they might not have been explicitly looking at. With this test, you may find an increase in return visitors as users consider a broader range of content on your site.

Common default:

When hovering a cursor over a common category, there is no hover state showing subcategories, top stories, or any other form of related content.

What to test:

  • Add subcategories to hover state
  • Add top stories to hover state
  • Add imagery to hover state

What to measure:

  • Increase in page views
  • Decrease in home page bounce rate

Alternate option:

When hovering a cursor over a common category, the top subcategory links appear on the left, and a top story with an image appears as well as a list of top headlines.

Consider: Many companies may shy away from a test like this, believing it is too complicated. But it does not have to be. For the purposes of a test, make it simple by:

  1. Choosing your five most clicked sections
  2. Selecting the top subcategories for each section (if you do not have official “sub categories” choose a common article tag)
  3. Display those subcategories on cursor hover

If the initial test with only subcategories or manually hardcoded article links drives more engagement, you now have some data to bolster a case for building this out in a more complete way.

Reduced categories options

Too many categories may overwhelm users. It may not make them bounce but it may cause them to engage with a different section of the page instead of drilling into a category of content that has many articles to their liking. Reducing categories can focus the user experience. It also may give you more space and allow you to increase the size of your navigation font.

Common default:

Menus often display 10 or more navigational choices. In this example, imagine a user sees Politics and Justice. Momentarily, they do no know which to click. A flashy piece of Entertainment content catches their eye. The user reads the entertainment article but gets bored halfway through and bounces. Had this user only seen Politics in the top navigation, they may have engaged within that section and ultimately read three or four articles instead of one.

What to test:

  • Reduce the choices to 8 or fewer
  • Remove media categories (like video) from the navigation (but expose it elsewhere on the page if it is an important feature)

What to measure:

  • Increase in time on site
  • Increase in pageviews

Alternate option:


More limited navigational choices may, paradoxically, get users to engage with them more.

Consider: Consider a custom navigation based on cookies. If a user comes to your site and always goes to a particular category or subcategory, expose that more prominently or give the user the ability to bookmark that specific page.

Top and left rail navigation

An emerging trend for many media and publishing sites is to use a top navigation bar in addition to a left navigation. The left navigation sometimes contains subcategories of the selected top navigation section or it contains top headlines.

Common default:

Top navigation.

Left rail navigation.

What to test:

  • Display either a top or left navigation only (in contrast to the current default)
  • Display a left navigation with top headlines
  • Display a left navigation with subcategories
  • Add a subcategory top navigation bar for trending topics

What to measure:

  • Increase in page views
  • Decrease in bounce rate

Alternate Options:

This site uses the top navigation for categories and the left rail for subcategories.


This site uses the top navigation for categories and the left rail for headlines. It also has a sub-navigation for trending topics at the top of the page.

Consider: If you are exposing subcategories or popular tags that were previously not displayed, you may discover a newly popular area of content.

Content encouraging return visits

Consider the content on the site that consistently gets users engaged. Interactive content or short repeatable content modules give users a reason to visit the site on a routine basis.

Common default:

Most media and publishing sites display a variety of general news article content with the hope that one of the stories will catch a reader's interest.

What to test:

  • Move interactive content higher on the page to engage users
  • Create or display brief, lighthearted (but interesting) and repeatable content high on the page

What to measure:

  • Increase in return visits
  • Increase in pageviews

Alternate Concept:


This site promotes interactive content (like quizzes or polls).

Consider: If you have distinct and engaging content that may give users a reason to return to the site on a frequent basis, consider showing newsletter signup promotions in that area.

Value propositions

Users who click Subscribe are qualified and interested but they may not yet be convinced. Many sites miss this opportunity to better demonstrate not only what the user will get if he subscribes but also why. Dedicate a space to this.

Common default:

This page does not clearly call out the value propositions for subscribing. It essentially assumes that users who land on this page are already convinced.

What to test:

  • Value propositions vs. no value propositions
  • The number of value propositions
  • The content of the value propositions (purely descriptive of the offering or genuine value props that actually promote the content)

What to measure: Increase in subscription conversion

Alternate option:


This page clearly calls out several different value propositions entirely separately from the subscription fields. It makes the assumption that there is additional convincing that needs to be done.

Consider: Test the impact of targeted content based on the users most recently visited content category and see if users who receive targeted content convert at higher rates.

 

Article page testing opportunities

Your articles likely form the bulk of your site content, and they are great opportunities to get users engaged with more content. We will discuss several types of tests to run on your article pages, including related sidebar content and text above the fold.

An article page needs to balance between a good reading experience for the primary content while providing an easy way to access other content. If related articles are placed too high in the sidebar, they are likely to be missed entirely or simply distract the user from their original content. Placing related articles lower may make users notice it at the right time as they read and scroll down the page.

Common default:

Related article content directly follows the primary article's headline.

What to test:

  • Related articles at the top of the sidebar
  • Related articles in the middle of the sidebar
  • Related articles low in the sidebar
  • Other modules (e.g., Tech Gear in the example above) higher/lower on the page

What to measure:

  • Increase in pageviews
  • Increase in time on site

Alternate Concept:

Any of these could be used as the default search result.

Most article pages show related articles in the sidebar. Typically, however, they only show a headline for that article. Displaying additional article details on hover may garner more clicks.

Common default:

If the user hovers over the article on the left, no additional information is provided or displayed.

What to test:

  • Hover state with simple “alt text”
  • Hover state with a formatted window showing the first article sentence
  • Hover state with a formatted window showing text and an image

What to measure:

  • Increase in pageviews
  • Increase in time on site

Alternate Concept:

As the user hovers over the article on the left, a preview of the article is displayed with text, an image, and a Read more link.

Consider: You do not have to test this universally. Test it on a few specific article pages by hardcoding those pages and overriding your recommendations engine. Only when you have tested and understand whether it is likely to have an impact should you consider making these changes automated across many pages.

Text above the fold

Some users may be less inclined to engage with content if it is shown in large blocks of text. Through imagery, quote blocks, or other methods, test the impact of breaking content up much like a magazine does.

Common default:

A small image leaves a large block of text above the fold.

What to test:

  • Increase the size of the image
  • Show a quote block within the article

What to measure:

  • Increase in pageviews
  • Increase in engagement with other content due to users scrolling further down the page

Alternate option:

A larger image means a smaller block of text above the fold.

Consider: What works on one site may not work on another and what works on one article may not work on another. Consider not only your audience but also the content itself and look for trends based on article type.

Video page testing opportunities

Your videos can be a great way to engage users, which also makes them great for testing. We will discuss several types of tests to run on your article pages, including video thumbnails vs. category links and content breadcrumbs.

Video thumbnails vs. category links

Video landing pages need to have a balance of suggested content with enticing images and—depending on the site—create an easy way to browse for videos on a particular topic. Cooking sites may attract users who are looking for very particular content as well as users who are interested in browsing whereas a news site is likely to have users more focused on browsing.

Common default:


This site has opted for several medium-sized video thumbnails but makes it difficult to find specific content.

What to test:

  • More video thumbnails vs. fewer but larger
  • More links to specific video content

What to measure: Increase in video views

Alternate option:

This site has opted for a large slideshow of featured videos with category images and links to subcategory videos.

Consider: It may be beneficial to create two different “video landing” pages. One may focus on a video browsing experience while one may focus on an experience that allows users an easy way to navigate to specific content.

Breadcrumbs (and category tags) can be a great way for users to find a wealth of content they may not have even known existed. A site that has hundreds of videos on a particular topic can only display a fraction of these with thumbnails. But breadcrumbs create an opportunity for users to see the full scope by clicking to the content’s category.

Common default:

Many sites rely on related videos (or related content in general) to drive increases in engagement.

What to test: Increase the size of the navigation breadcrumb text

What to measure:

  • Increase in video views
  • Increase in return visits, as users better realize the scope of the content on the site

Alternate Concept:

A more prominent breadcrumb navigation can also drive higher engagement. For many users it will be an easier way to navigate to content.

Consider: Your top-level, global navigation that appears on every page is important but it is always the same. The breadcrumb is the contextualized version of this for each page. Different users navigate in different ways and it is important to give a variety of ways of finding your content.

Disclaimers and other information

The specific examples were chosen at random from Optimizely Web Experimentation clients and non-clients alike. The examples are purely a representation of what could be tested and are not necessarily representative of actual tests that have been run. Keep in mind that these alternate concepts may not win—they are intended to help you run experiments to gather data that will be unique to your site. No test data has been shared without the explicit consent of the clients.