Testing ideas for ecommerce and retail sites

  • Updated
This topic describes how to:
  • Come up with ideas for your ecommerce or retail site A/B tests
  • Design tests based on these ideas

This article provides A/B test, also known as split testing ideas for elements commonly found on ecommerce sites. You may want to run these experiments using Optimizely Web Experimentation on your ecommerce site. For each example, we have provided a common default example and an alternate concept, with ideas of what to test and how to measure.

 
Tip:

Learn more about the opportunities and specific challenges for retail businesses.

How to use this guide

Consider this article a starting point for your optimization efforts. After all, experience optimization is a journey, not a destination. The tests you run will help you discover what works (or does not) for your visitors, so it is important to pair the ideas in this guide with research on your site or app's true opportunities and obstacles.

Do:

  • Use this guide as a starting point to identify opportunities for your site or app
  • Use the ideas from this guide that help you answer key questions about your users or move key metrics
  • Pair the ideas presented in this guide with data from your analytics platforms and user feedback so that you run tests that explore identified issues

Do not:

  • Use this guide without also looking at your qualitative and quantitative data
  • Try to test every idea in this guide
  • Use these test ideas verbatim without thinking about how to adapt them to your experience

Key metrics for ecommerce sites

When you set up experiments, you will want to have a good idea of what to measure so you can define success. Here are some of the key metrics you will likely use and the key pages on which you will likely track them.

We also provide suggestions for metrics in the description of each test. You can also learn about about primary versus secondary goals.

 

Homepage testing opportunities

Your homepage is the entry point to your site, and it sets the tone for your brand's value and site's offering. It has a high traffic volume and will more quickly collect data.

When testing on your homepage, consider that it is far from your purchase event, so your traffic is not yet "qualified." Consider metrics aside from purchases to measure success (for example, product page views or add to cart events).

The following elements are commonly found on ecommerce homepages, and we will discuss testing opportunities for each:

  • Search boxes
  • Scrolling banners and hero images
  • Categories and features/offers
  • Taglines and value propositions (brand differentiation)
  • Targeted content

This is an easy A/B test to set up within Optimizely Web Experimentation and target to any page on your site, and it is a missed opportunity for many businesses. Most boxes simply read "Search" or have no text at all.

Common default:

What to test:

  • Show text suggestive of the range of products your site offers or the ways search can be used
  • Test showing a search CTA (Call to Action) and the value of an icon CTA vs. a word CTA

What to measure:

  • Increase in product page views and conversion by driving users to products faster
  • Increase in average order value by highlighting products not previously considered

Alternate Concepts:


Product-specific


Inspirational


Functionality-focused


Functionality-focused


Functionality-focused

Consider: If your search results page is not strong but is schedule for a revamp, this is a great test to see if you can drive more users in that direction.

Many scrolling banners and hero images (large banner images placed prominently on your homepage) span the width of the page, move too quickly, make it hard to navigate between them, and have inconsistent placement of the CTA (Call to Action) between images.

Common default:

This site has a large scrolling banner that moves quickly, with CTAs appearing in different places.

What to test:

  • Eliminate auto-scroll through the hero
  • Increase the prominence of hero navigational controls with color, location or hover states
  • Reduce the hero to 2/3 width and show other top content above the fold
  • Grid layout of images instead of single image
  • Use of JavaScript scrolling banners instead of Flash
  • Product images vs. lifestyle images

What to measure:

  • Reduction in homepage bounce rate
  • Increase in awareness and click-through on important hero messaging

Alternate Concepts:


This example uses easy navigation on the left and a consistent CTA location on the bottom-right.

This example uses no rotation in its banner, a centered hero image, text promos on the left, and a grid layout with small images on the right.

Consider: There may not be an increase in hero click-through but be aware of changes to other key metrics down the funnel. If your hero content will be changing during the run of the test, consider how this may impact certain metrics. (For example, prominent sale messaging during the second week of the experiment may drive a conversion rate increase that week but a decrease in average order value.) Finally, consider targeting your hero content or layout based on previous on-site behaviors (categories visited, product page visited, item currently in cart, etc.).

Category and navigation repetition

Many homepages use the space below the hero image to showcase special offers and featured products, much of which is likely to be ignored in favor of a user clicking the top-level navigation.

Common default:

This site's offers and featured products are below the fold of the hero image.

What to test:

  • Showcase category content
  • Varying amount of text to describe categories (simple title vs. title and description)
  • Clear and highly noticeable hover state to increase a feeling of interactivity

What to measure:

  • Reduction in homepage bounce rate
  • Increase in AOV as additional products/categories are considered

Alternate concept:


This example uses top and side banners instead of a hero image, and special offers below the categories.

Consider: What are your top user paths from the homepage? If it s interaction with certain navigational elements, test showcasing those outside of the navigation.

Brand differentiation

A/B testing is the simplest and cheapest way to test slogans and taglines, and this is also one of the easiest experiments to set up and target to any page on your site in Optimizely Web Experimentation. Many sites do not make clear what they do or why they are different. Simply because a user has landed on your site does not mean they are convinced they should buy from you.

Common default:


This tagline may be hard to read and is not focused on price differentiation.


This tagline could communicate how the service works.


This site has six layers of navigation above the hero images. Its value props (free shipping, international shipping, no hassle returns) can easily be missed.

What to test:

  • Test an alternate tagline or marketing slogan
  • Make tagline more prominent by your logo
  • Explicitly state your value props

What to measure: Increase in conversion

Alternate Concepts:

The scope of offerings and price value are made clear.


For sites with lots of organic, non-branded items, a tagline with clear explanation, directly above a link to learn more, works well.

Consider: You may not get statistically significant results with this test—particularly if you only target it to the homepage which is far removed from your ultimate conversion event —but measure other metrics which can tell you if you are moving in the right direction. For example, an increase in product page views or time on site when you have a price point differentiation tagline may lead to a variety of other tests focused on price messaging.

Targeted content

With Optimizely, you can dedicate a section of the homepage to serve targeted content based on pre-defined criteria. This may be based on buckets of search terms, geo-location or previous on-site behavior. This moves your site in the direction of personalization and can take your user experience to the next level, but many sites miss an opportunity to display personalized content based on user targeting information.

Common default:

The user has previously visited Maternity, but that content does not display by default. Maternity appears only on the left-hand side of the screen because this company is running a sale.

What to test:

  • Location of targeted content (e.g., above/below the fold, main hero vs. sidebar, etc.)
  • Specificity of targeted content (e.g., target with product image vs category image)

What to measure:

  • Increase in conversion
  • Increase in RPV (revenue per visitor)

Alternate concept:


In this example, the user has previously visited Maternity, so the Maternity hero image appears by default.

Consider: Start with a simple targeted test that will get a lot of traffic. This may be a common organic search term or reading a cookie that was set based on a user visiting a section of your site.

Search results page testing opportunities

The following elements are commonly found on search results pages and we will discuss testing opportunities for each:

  • Sort order
  • Filter options

Sort order

This is an easy thing to test without making changes to page layout. Many sites default their search results to Featured Items. This does not communicate to the user anything meaningful about the way the results are shown.

Common default:

This sort menu defaults to Featured Items.

What to test:

  • Default to a user generated search order like top rated, most popular, or best sellers
  • Default based on price (low to high or high to low)
  • Default based on recency, like New Arrivals

What to measure:

  • Increase in conversion
  • Increase in product page views
  • Decrease in bounce rate
  • Increase in searches

Alternate concept:

Any of these could be used as the default search result.

Expanded vs. collapsed filter options

Like sort order, this is also an easy test that requires a bit of simple code or a redirect, but will not change the page layout. Many sites default to showing all their filters on the left side of their page expanded. This makes it difficult to see the filtering options available or to notice the most important filtering options.

Common default:

Only one main filtering option is shown above the fold with many subcategories shown.

What to test:

  • Defaulting to all the filters collapsed
  • Defaulting to some (but not all) of the filters collapsed; only shows the most commonly used ones expanded
  • Changing the order of the filtering options to make the most important ones show at the top

What to measure:

  • Increase in conversion
  • Increase in product page views
  • Decrease in bounce rate
  • Increase in searches
  • Increase in interaction with filters (bringing broader awareness of product offerings)

Alternate concept:

All the main filtering options are shown above the fold and none of the subcategories are displayed.

Category and subcategory page testing opportunities

Your category and subcategory pages can turn "unqualified traffic" into "qualified traffic." In other words, they turn browsers into buyers. On these pages, you can set a cookie for users' broad interests for future targeting and personalization.

When testing these pages, consider that they are somewhat far from your purchase event and you likely have many category pages, so consider testing changes manually on one page before running the experiment on all pages.

The following elements are commonly found on category and subcategory pages, and we will discuss testing opportunities for each:

  • Image size
  • Descriptions
  • Compare
  • FAQs and buying guides

Image size

Subcategory pages that show a grid of images often are too small to make out any detail for the individual products, requiring users to click through to the product page to determine if they are interested.

Common default:


This subcategory page has five images across.

What to test: Using larger images so the product details can be more easily made out

What to measure: Fewer product page views per session but more subcategory page views and a higher conversion rate

Alternate concept:

This subcategory page has larger images and shows only three per row.

Consider: This may be a technically difficult test if done on every subcategory page of the site. But a simple way to do a test like this is choose one subcategory, or even simply the first page of a subcategory, and manually code that page with larger images. If you get a higher conversion rate on those products, this may be a sign that it is worth trying on other subcategories in a more automated fashion.

Descriptions

There is a balance between having too much clutter on the page and providing a bit of extra detail that might entice viewers to take a closer look.

Common default:

The subcategory page has no descriptive text for any of the products.

What to test: Add a bit of detail, either product description, or specification details (sizes and measurements, material types, etc.)

What to measure: Fewer product page views per session but more subcategory page views and a higher conversion rate

Alternate concept:

This subcategory page has the first part of the description from the product page and a Read more link.

Consider: This may be a technically difficult test if done on every subcategory page of the site. But a simple way to do a test like this is choose one subcategory, or even simply the first page of a subcategory, and manually code that page with the first part of the descriptive text. If you get a higher conversion rate on those products, or a higher product page view rate on those products, this may be a sign that it is worth trying on other subcategories in a more automated fashion.

Compare

If your site offers the option to compare products, this can be a significant tool to help users make decisions on products with many details. Testing your compare feature is relatively easy to set up and apply to a wide variety of pages.

Common default:

The compare box is far from the image and the last item before the price.

What to test:

  • Move the Compare button closer to the product image
  • Change the text to read Compare Products
  • Add hover state text that explains how it works (e.g., “Select up to three products to compare features”)
  • Make the Compare text larger

What to measure:

  • Increase in conversion rate
  • Increase in Compare feature use

Alternate concept:

The compare box is immediately below the image.

Consider: If you are applying this to a variety of category or subcategory pages, make sure those pages are all using the Compare feature and have the same basic template.

FAQs and buying guides

If your site sells products that require a significant amount of research prior to making the purchase, test giving more prominence to FAQs, buying guides and customer service features. Depending on your site, this should be very easy to set up in the Optimizely visual editor using the rearrange drag and drop functionality. Many sites bury their FAQs and buying guides.

Common default:

The FAQs, buying guide, and customer service phone number are in the far corners of the page and not likely to be noticed. FAQs and Buying Guides link are below the fold.

What to test:

  • Move the FAQs, buying guides or other support options higher on the page above the results filter
  • Move the support options as a row between the product rows to make them obviously noticed after the user has scanned the first products

What to measure:

  • Increase in conversion rate
  • Increase in time on site
  • Increase in return visitors as users think of your site as an industry expert and a place to start their research

Alternate concept:

The FAQs, buying guide, and customer service phone number are placed high on the page. Users will still be likely to find the Narrow Results option because they are used to finding this in the left column.

Consider: Even if your site does not have extensive buying guides, test the prominence of your help features and any FAQs you may have. If you are considering creating content like buying guides, this will give you the data to support your case for prioritizing those efforts.

Product page testing opportunities

Your product page is a sweet spot of high traffic volume and qualified traffic. Keep in mind, however, that not all changes will work across all products, so be aware of the varying page templates that you use.

The following elements are commonly found on ecommerce product pages, and we will discuss testing opportunities for each:

  • Imagery
  • Ratings, reviews, and testimonials prominence
  • Related products
  • Free shipping

Imagery

Every product page has product images, but not every page has lifestyle imagery which can help communicate additional information (such as size and fit, common uses, etc.) and provide inspiration.

Common default:

One product image is shown for two different product colors. Product is not shown in a lifestyle context.

What to test:

  • Add lifestyle images in the primary image area
  • Add lifestyle images in the description area of the page

What to measure:

  • Increase in conversion
  • Increase in time on site

Alternate concept:


Multiple product images are shown including two lifestyle images in the product description area.

Consider: Once you learn if users are motivated by lifestyle imagery, try testing this concept in other places on your site. Test on many pages or only a few pages.

Ratings and reviews

The prominence of ratings, reviews or even user testimonials (on ecommerce but non-retail sites) can strongly influence users behavior for certain types of products. Make sure this content gets noticed. This test can often be done by moving existing elements around on the page using the visual editor.

Common default:

The ratings and reviews content may be difficult to notice since it is surrounded by a lot of other content.

What to test:

  • Move ratings and reviews content closer to the price
  • Add hover states to the ratings so it feels more interactive and may be noticed on incidental hovers
  • Use larger ratings and reviews stars
  • Make testimonials stand out by adding large quotes or other stylistic changes
  • Make testimonials stand out by adding a user’s image

What to measure:

  • Increase in conversion
  • Increase in average order value (AOV)
  • Increase in revenue per visitor (RPV)

Alternate concept:

The ratings and reviews content is immediately below the price in an area that almost all users' eyes are likely to look. The content is also given its own tab at the top of the page.

Consider: Try an “extreme” version with very large review stars. Create segments for products with no reviews, low reviews, medium reviews, and high reviews (based on star ratings). The goal of this is to see if interaction with reviews goes up and by segment how the more prominent version affected conversion.

Finding the right balance of showing related products and not distracting users with too many options (creating “FUD” - Fear, Uncertainty, Doubt) is unique to your product offerings and your site. This makes it a great test.

Common default:

Users are shown competing products immediately by the description of the product page they are currently on. This test can easily be done using Optimizely’s visual editor.

What to test:

  • Remove showing related products
  • Show fewer/more related products
  • Test alternate algorithms (higher/lower price point, same/different brand only, highly rated etc., users who viewed vs. users who bought)

What to measure:

  • Increase in conversion

Alternate concept:

Competing products are removed and only People who bought also bought... is shown low on the page.

Consider: What works for one product category may not work for others. Create segments to determine what types of products users are converting on most or run this as multiple test at separate times on different categories.

Free shipping

Free shipping is regularly shown to significantly affect user behavior. Before you optimize your offers based on price, make sure users are seeing the offer in the first place. This test can teach you how promotional offers can impact user behavior (do users spend more or convert more to get special offers) and it is also easy to setup.

Common default:

Free Shipping is displayed above the navigation and below the product details as part of the generalized value props.

What to test:

  • Increase free shipping prominence by displaying it in the product details
  • Duplicate the same look and feel for the generalized message as for the this product qualifies message
  • Test various price points
  • Test adding a shipping logo instead of simply text

What to measure:

  • Increase in conversion
  • Increase in average order value (AOV)
  • Increase in revenue per visitor (RPV)

Alternate concept:

The same “Free Shipping” style shown above the navigation is shown immediately below the price in the product details area.

Consider: Consider creating conversion goals for multiple price points. For example, does your free shipping offer work best for users who are buying over a particular dollar amount? Also, if your site does not currently show (on the product page) when a product qualifies for free shipping, you can add this functionality in with Optimizely using a bit of custom JavaScript and measure its impact!

Shopping cart testing opportunities

Your shopping cart is the "tipping point" of your site. The right offer (free shipping, discounts, etc.) can make the difference for customers, so this is a great place to test. The shopping cart is the last (good) opportunity to upsell customers, so experimenting here can help determine what "tips" your customer one way or the other.

The following elements are commonly found on shopping cart pages, and we will discuss testing opportunities for each:

  • Checkout button
  • Promo and coupon code
  • Recommended products
  • Continue shopping

Checkout button

Users who arrive on the shopping cart page are doing only one or two things: reviewing their cart to confirm they have the right items in the right quantities (and making changes if they do not), and moving on to the checkout page. In short, the shopping cart page is about usability. The most usable pages do not make the user think. Having an obvious checkout button that draws the user’s attention is key.

This is an easy test to set up in Optimizely. Image buttons can easily be replaced, and HTML-based buttons can be modified with basic CSS. Testing your shopping cart is particularly effective because it is close to your final conversion event, so small lifts from testing can translate to big ROI.

Common default:

The checkout CTA is the same color as the top navigation, making it harder to see.

What to test:

  • Use a button color that stands out (it is OK if it is a bit “off brand” as long as it stands out)
  • Provide more empty space around the button
  • Use a larger button and text
  • Show checkout buttons above and below the cart to make them easy to find
  • Add a hover state to the button to make it stand out

What to measure: Increase in conversion

Alternate concept:

The checkout CTA is a contrasting color to the rest of the page. There are a variety of things that could be optimized on this button and page but if nothing else, the button immediately stands out.

Consider: Not every small change on this page will make a difference for your ultimate conversion event. But if you move more people into the checkout flow, you are probably on the right track.

Promo and coupon code

When there is an empty coupon or promo field—particularly on pages where users need to take some kind of action—users focus their attention on it and will often search the web for a coupon code which takes them away from your site, resulting in a lower conversion rate.

Hiding this field behind a link preserves the essential functionality for users who genuinely do need to enter a code, but does not distract users who don’t have it.

You can set up this test without any back-end manipulation if the field is simply hidden behind a link. Check out this article on price testing in your checkout funnel to learn how.

Common default:

The coupon code field is exposed and has a prominent CTA alongside it.

What to test:

  • Hide the field behind a link and expose the field when the link is clicked
  • Remove the CTA associated with the promo code field and capture the coupon details on the next page

What to measure:

  • Increase in conversion
  • Decrease in page bounce rate

Alternate concept:

The coupon code field is hidden behind a link. When the link is clicked, the field and the CTA are exposed.

Consider: If possible, try testing the page within the checkout flow that is optimal for displaying the coupon code. The example here shows it on the shopping cart page but it may be better to have it on the payment page when a user is more deeply committed to buying from your site.

Related or recommended products are often shown on the shopping cart page. Depending on the products in the cart and what your business sells, this may make sense. But it may also distract the user from completing their purchase or even confuse them if they’re trying to confirm what items they are about to purchase.

Common Default

Recommended products within the same product category are shown in the right sidebar.

What to test:

  • Remove related products altogether
  • Show products only within a certain price range (usually lower works better)
  • Show products only from different categories than what is in the users cart
  • Clearly label the recommended products as Related items or Users also bought

What to measure:

  • Increase in conversion
  • Increase in average order value (AOV)

Alternate concept:

Only one recommended product is shown and it is not a related product.

Consider: There is tons of opportunity for personalization and targeting based on the results of these types of tests. If you add nuanced segments based on average product price (in cart), total cart value, product categories etc., you have the ability to gain in-depth insights into the patterns and behaviors of your users. For example, high spenders, buying electronics over $300, are likely to add on products when presented with electronic accessories but users spending less than $50 are not.”

Continue shopping

Users are often given multiple options on the shopping cart page that drive them away from checking out. There is a balance between giving users options and suggesting new products and driving users towards the primary goal. Test your site to see what works best—this is an easy test to set up.

Common default:

The top navigation is shown and immediately to the left of the checkout button there is a continue shopping button.

What to test:

  • Remove the top navigation and replace it with a continue shopping or return to shopping link
  • Keep the top navigation but remove the continue shopping CTA next to the checkout button

What to measure:

  • Increase conversion
  • Increase revenue per visitor (RPV)

Consider: What other distractions do you have on the shopping cart page that may be driving users away from the primary goal?

Checkout flow testing opportunities

In the checkout funnel, users are highly qualified. They have already made a buying decision, so even a small conversion lift will directly correlate with revenue. Focus on usability and few distractions so that the user can reach the end of the funnel easily.

When testing on your checkout flow, keep in mind that low traffic volume means that tests may need to run for longer.

The following elements are commonly found on ecommerce checkout flows, and we will discuss testing opportunities for each:

  • Sign in or guest checkout
  • Hiding unnecessary fields
  • Hiding unnecessary functions
  • Key messages

Sign in or guest checkout

Once users indicate that they are ready to checkout, the last thing you want to do is make them believe they must sign up for an account. Even if your preference is to have them sign up, a conversion without signing up is better than no conversion at all. Tests on your sign in/guest checkout function are fairly clear-cut, as users either advance down the funnel at a higher rate or they do not.

Common default:

Exposed sections show all their fields. Guest checkout is hidden.

What to test:

  • Exposing only fields for one section
  • Do not expose fields or a CTA for any section until the user has made a selection, then expose fields and a single CTA
  • Listing easy-to-read value props
  • Use a title at the top that makes it clear the user has a choice (e.g., Begin Checkout: Select an Option Below)
  • Display choices left to right or top to bottom

What to measure: Increase in conversion

Alternate concept:

Only one section shows fields, and clear value props are listed for each section.

Hiding unnecessary fields

Which form would you rather complete, a long one or a short one? Forms mean work for the user, and when a user sees a bunch of fields it can be overwhelming. This happens commonly on shipping and billing pages, where users often enter the same information twice. Instead of forcing the user to make a selection indicating their information is the same, force them to make the selection only if their information is different. This reduces the likelihood of the user feeling overwhelmed and abandoning.

Common default:

The "same as" box is not selected by default and all the shipping fields are exposed.

What to test: Select the "shipping same as billing" checkbox by default and hide the fields

What to measure: Increase in conversion

Alternate concept:

By default, the "same as" box is selected and the fields are hidden from view.

Consider: With a bit of custom JavaScript code entered in Optimizely, the fields can simply be hidden. As the user enters information in the Billing section, the information will also be entered in the Shipping section. Then, if the user deselects the “same as” checkbox, the information is removed and the user can enter fresh information.

Hiding unnecessary functions

Much like an empty form field, when users see a drop-down list, they often think they need (or feel compelled) to interact with it. If you have a shipping cost estimator or similar functionality, those can drive users away from converting. By preserving functionality on the page, no backend changes are required to run this test.

Common default:

Users can estimate shipping costs here, but this is not actually where they select the shipping method.

What to test:

  • Hide the shipping cost estimator behind a link
  • Move the shipping cost estimator functionality deeper into the funnel

What to measure: Increase in conversion

Alternate concept:

All the functionality is the same as the common default above, but it is hidden behind a link.

Key messages

What are the most important messages for users to read in your checkout funnel? Do you know if they are reading them? Are they obvious and easy to see? Are they located in the right place? Test making the messages in your funnel more prominent. You can set this test up in minutes.

Common default:

The message is gray and hard to read.

What to test:

  • Use larger text for messages
  • Place a box around the message or add a graphic (perhaps showing “!” to indicate importance)
  • Move it earlier or later in the funnel

What to measure: Increase in conversion

Alternate concept:

The text is black, large, and easy to read.

Consider: A messaging test like the example shown not only helps you fix a potential usability problem, but it helps you better understand the fears, uncertainties, and doubts users may be having on your site. In this example, it is a travel site that allows you to book a vacation home directly from the owner. If this alternate concept were to win, it might signal that showing additional “trust” messaging elsewhere on the site or within the conversion funnel might lead to an even higher conversion rate.

Disclaimers and other information

This article provides Optimizely customers with a resource for learning about common testing ideas on retail and ecommerce websites. The specific examples were chosen at random from Optimizely customers and non-customers alike. The examples are purely a representation of what could be tested and are not necessarily representative of actual tests that have been run. No explicit test data has been shared without the explicit consent of the customers.