Build a Recommendations experiment

  • Updated
Optimizely Web Recommendations is not currently included in the packaging for new customers. It is not an out-of-the-box solution and requires great effort to configure. If you are looking for personalization solutions to use with Optimizely Web Experimentation, our Customer Success Managers can help review our portfolio of tools to find the right fit for you.
This topic describes how to:
  • Create a Recommendations experiment
  • Add or edit content, product or category recommendations on your site

Optimizely Adaptive Recommendations is a recommendation engine that makes it easy to introduce visitors to products and content, based on browsing behavior. Once they are published, you can be hands-off with your recommendations, but it is a good idea to always be experimenting with them.

With Recommendations, you can test out different recommendation algorithms and measure the impact in real time with Stats Engine.

Recommendations also works hand-in-hand with other Optimizely Web Experimentation products for more powerful experimentation and targeting.

This article walks you through the process of using Recommendations in your experiments:

  • Creating an experiment targeting the pages on your site where you will deliver recommendations;

  • Optionally, adding an audience to specify who should see those recommendations; and

  • Verifying your recommenders are working properly.

Click to learn more about Recommendations or speak to your account manager.

Create an experiment in Optimizely Web Experimentation

To create a new Recommendations experiment:

  1. Navigate to the Experiments dashboard.

  2. Click Create New... and select A/B Test.

  3. Name your experiment.

  4. In the Target by drop-down list, select Saved Pages.

  5. To add an existing page where your recommendations will appear, you can search for pages in the Browse for Pages field or you can select a page from the list of recently-created pages. If you want your recommendations experiment to run on a new page instead, click Create New Page to add one.

  6. Click Create Experiment.

Add an audience, metrics, and traffic distribution

test_experiment__Variations_-_Optimizely.png

  1. If you want to show your recommendations only to a certain group of visitors, you must add an audience. Click Audiences to begin the process. This article explains how to create a new audience in Optimizely or you can add an existing audience.

  2. Choose metrics to measure the success of your recommendations. Click Metrics to begin the process. This article explains how to create a new metric in Optimizely or you can add an existing metric.

  3. Use traffic distribution to decide what percentage of visitors see your recommendations. By default, Optimizely shows 50% of visitors the variation with recommendations and 50% the original variation. Click Traffic Allocation to adjust the distribution to show recommendations to a higher or lower percentage of visitors. This article explains how traffic allocation works in Optimizely.

Next, you will choose an algorithm and decide where to show your recommendations.

Configure recommendations

Use the Editor to choose a recommender and place the recommendations on your page.

  1. On the Experiments dashboard, click the experiment to open the Manage Experiment dashboard.

  2. Select a variation.

  3. Click Create to make changes to the variation.

  4. Find and select the Recommended Products extension. This should have been built during the Recommendations onboarding process.

  1. Modify the recommendations so they look the way you prefer.

  • Click the Header Text to modify the header for your recommendations. For example: "You might also like."

  • Click Maximum Products to select the number of recommendations to show at one time.

  1. Next, click the Algorithm drop-down list to choose an algorithm for your recommendations.

  1. Decide where you want your recommendations to display.

  • To change the position of your recommendations module, click Insert After Selector and add the selector that the module will appear below.

  1. Finally, choose the timing for loading your recommendations on the page: synchronous or asynchronous.

  • Consider using asynchronous changes to allow the rest of the page to load while your recommendations load. If you want the rest of the page to wait for the recommendations to load, choose synchronous timing.

  1. Click Save.

  2. QA your Recommendations experiment with the Preview tool. When it looks and works the way you prefer, publish your experiment.

Your recommendations are now live to the world! To measure the impact on your key metrics, check out the Results page.

Verifying Recommendations Setup

Before proceeding too deep into building recommenders, we recommend you verify that you are collecting the data you were anticipating. There are two ways to verify that do this: either look at the stats in the inspector interface or download the SummaryStats CSV file, which goes into a little more detail.

In this example, the customer had 927k view events in a week. If a week’s worth of traffic typically includes about 2M view events, that would mean the customer is losing about half their usual views with their recommender in place.

In this example, custom events are any user-defined events other than a view. These are usually things like add-to-cart clicks or social shares.

The item count represents the number of how many unique items were touched by an event in the last week. In this example, that number is 542. Determining whether this is an issue or not depends on context: if the customer’s entire catalog contains 3000 items, the item count shown here would be worrisome, since it means that less than 20% of the catalog has been viewed during the past week. If, however, their catalog contains closer to 650 items, this would be much less likely to represent an error.

Visitor count is the number of unique visitors came to the customer’s site. Validation failures lists the number of catalog items that were not available when Optimizely tried to visit their pages.

Check these stats for anything that stands out or does not seem right when you are setting up your collection.

Ideas for recommendations

There are many ways to use recommendations on your site. Below, we suggest a few retail and B2B use cases.

Retail sites:

  • In the checkout funnel, show accessories that complement the items a visitor is purchasing using a co-buy algorithm applied to add to cart events

  • On product detail pages, show alternative items that are related to the product a visitor is browsing using a co-browse algorithm applied to page view events

  • Highlight crowd favorites on the homepage using the popular algorithm on view and buy events

If you are using Optimizely to test on a checkout page, you might need to configure your site for PCI compliance.

B2B or lead generation sites:

  • Show visitors whitepapers, infographics, blog posts and other content based on their browsing behaviors

  • Suggest knowledge base articles or community posts to reduce support call volume

Experimentation ideas

Here are a few experimentation ideas to help you get the best performance from your recommendations implementation:

  • Try collecting different events: for example, add-to-cart, social share, interaction with a color picker, or description expansion all have strong potential for insights.

  • Experiment on the layout, positioning, and appearance of your recommendations, as well as the number of recommendations shown.

  • Experiment on the events going into the recommenders. A popularity algorithm based on view events will deliver different results from a popular item based on add to cart events.

  • Try applying different filters. For example, you could compare an unfiltered recommender against one with a similar price range filter, where the item being viewed sets a range for the recommended item price.

  • Experiment on the effects of different algorithms. Take care that the input types match, but you could compare the different results delivered by a co-browse, co-buy, or semantic algorithm.