Read the Optimizely Web Experimentation log

  • Updated

This topic describes how to:

  • Read the Optimizely Web Experimentation log to see whether an event passes or fails
  • Check whether an experiment or campaign is loading
  • Check if you qualify for an audience condition
  • Check if changes on a page are publishing

The Optimizely Web Experimentation log is a valuable resource. It allows you to "read Optimizely Web Experimentation's mind" by printing the execution of targeting and activation decisions, variation changes, events, and third-party integrations on a page in your browser's console. Use the Optimizely Web Experimentation log to investigate issues that you can't otherwise easily diagnose. 

The log can help you check:

  • If an experiment or campaign is loading correctly

  • If you qualify for an audience condition

  • If changes you make on a page are applied

Start by enabling the log with a query parameter or the JavaScript API.

Looking for a more technical resource? Find out more about the log in our developer documentation.

Enable the log with a query parameter

You can enable the log by adding a query parameter before you load the page. Here's how:

  1. Add the query parameter optimizely_log=info .

  2. Reload the page to read the log.

In Optimizely Web Experimentation, so you must append the optimizely_log parameter and reload the page to see the log. This minimizes the logging overhead for visitors and pageviews that don't request the log.

Enable the log with the JavaScript API

  1. Open your browser's developer console.

  2. Use the JavaScript API to reveal messages from the page load. Append the query parameter and push the log before the snippet loads to see information from the current page load.

    window['optimizely'] = window['optimizely'] || []; window['optimizely'].push({ type: 'log', level: 'info' });

  3. Other JavaScript on the page may have produced console output as the page was loading, so you'll need to find where the Optimizely Web Experimentation log starts.

    Search for a line that starts with, Setting log level to...

To modify the verbosity of the messages, replace info with one of the levels below (listed from least to most verbose):

  • off – no messages

  • error – only errors (in other words, unexpected conditions that might cause the snippet to run improperly)

  • warn – unusual conditions that might indicate a misconfiguration

  • info – recommended if you're trying to identify what's running and what's not

  • debug – recommended if you're trying to identify why something is or isn't running

Read the log

Messages are printed in the log in the same chronological order as the events that occur. The messages may be easier to follow if you refer to this flowchart. If you're enabling the log with a query parameter, append the query parameter before you load the page to see the log.

All log messages follow the same format. Here's an example:

The first part indicates the time at which the message was captured, in milliseconds, relative to the time at which the snippet started to execute:

The second part, Optly,  indicates that the message is from the Optimizely Web Experimentation snippet:

The optional third part indicates the general class of behavior that the log message describes. In our example, the message is about page evaluation:

If the optional third part is missing from the message, the message corresponds to the "Main" routine of the snippet.

The remainder is the actual text of the message:

This message may include references to JavaScript objects, which you can inspect using the console. In this case, the included array encodes the page URL Targeting conditions:

Troubleshoot with the log

Use the steps below to troubleshoot pages, campaigns, audience, traffic allocation and bucketing, and variation code with the Optimizely Web Experimentation log. Work through each step in order, according to the Optimizely Web Experimentation order of activation. 

Steps include a general log format, example log, and search string. The general log format shows which information the log message usually includes. The example log is an example of a real log message you might see. The search string helps you find the log message you need for each troubleshooting step.

You may see the word Null instead of Page, Campaign, Variation, Experiment, or Audience Name in your log message. This means that you have turned on the privacy setting Mask descriptive names in project code and third-party integrations. If this setting is on, use API Names to determine which IDs Optimizely Web Experimentation is referring to. 

Before you start troubleshooting, make sure to enable the log with a query parameter or the JavaScript API and reload the page. If you're using API Names to identify masked IDs, open the API Names settings for the campaign you are troubleshooting in another browser window.

Page troubleshooting

Pages must activate for experiments to activate.

First, check whether pages are active. If pages are active, the URL you are currently troubleshooting met the URL Targeting or activation conditions.

General log format {time} | Optly / Activated Page "{page name}" ({page ID})
Example log  134 | Optly / Activated Page "Home" (3536920000)
Search string Activated Page

Second, check for pages that failed to activate. If a page (or pages) failed to activate, the URL you are troubleshooting failed the URL Targeting or activation conditions.

General log format {time} | Optly / Page / Failed to activate view for "{page name}" ({page ID})
Example log  216 | Optly / Failed to activate view for "Home" (3536920000)
Search string Failed to activate view

Campaign troubleshooting

Campaigns that are not targeted to the page you're troubleshooting will not appear in the log. Here's how to check which campaigns have been activated for the page:

General log format {time} | Optly / Deciding Campaigns for Page "{page name}" ({page ID})
Example log  149 | Optly / Deciding Campaigns for Page "Home" (3536920208) : "Homepage Test" (7729960575), "Second Homepage Test" (8163204973), "Third Homepage Test" (8169565973)
Search string Optly / Deciding Campaigns for Page

Search for executing actions and expand the object to find different changeSet objects, including whether custom code has run on the page. Executing actions will also include what selector was used for the element and whether the attribute or CSS of the element was changed.

Audience troubleshooting

There are a couple of common scenarios to check when troubleshooting audiences. First, it's possible that a visitor doesn't meet the audience conditions. In this case, the log will show the visitor is in an empty object ([ ]), which represents the "everyone" audience condition.

General log format {time} | Optly / When deciding Campaign "{campaign name}" ({campaign ID}) visitor is in audiences: [ ] and not in audiences: ["{Audience Condition 1}","{Audience Condition 2}"]
Example log  138 | Optly / When deciding Campaign "My Campaign" (7729960575) visitor is in audiences: [ ] and not in audiences: ["Audience Condition 1","Audience Condition 2"]
Search string When deciding Campaign

Second, a visitor may meet one but not both audience conditions.

General log format {time} | Optly / When deciding Campaign "{campaign name}" ({campaign ID}) visitor is in audiences: ["{Audience Condition 1}", Object] and not in audiences: ["{Audience Condition 2}", Object]
Example log  138 | Optly / When deciding Campaign "My Campaign" (7729960575) visitor is in audiences: ["Audience Condition 1", Object] and not in audiences: ["Audience Condition 2", Object]

Third, a visitor may meet all audience conditions. In this case, the log will show the visitor is not in an empty object ([ ]).

General log format {time} | Optly / When deciding Campaign "{campaign name}" ({campaign ID}) visitor is in audiences: ["{Audience Condition 1}","{Audience Condition 2}"] and not in audiences: [ ]
Example log  193 | Optly / When deciding Campaign "My Campaign" (7729960575) visitor is in audiences: ["Audience Condition 1","Audience Condition 2"] and not in audiences: [ ]

Traffic allocation and bucketing troubleshooting

This step helps you identify which campaign or experiment and variation a visitor was bucketed into.

General log format {time} | Optly / Recording decision for Campaign "{campaign name}" ({campaign ID}) Object {bucketingId: "{x}", audienceIds: Array[{x}], globalHoldback: {x}, preferredVariationMap: Object, activationId: "{x"...} -> Object{layerId: "{x}", experimentId: "{experiment ID}", variationId: "variation id", isLayerHoldback: {true or false}}
Example log  157 | Optly / Recording decision for Campaign "Homepage Test" (11111111111) Object {bucketingId: "oeu100000000.1111111111", audienceIds: Array[0], globalHoldback: 0, preferredVariationMap: Object, activationId: "e2240b20-6531-4824-a111-9d7ccfc2cc00"…} -> 
Object {layerId: "9999999", experimentId: "11111111", variationId: "2222222", isLayerHoldback: false}
Search string Recording decision

To confirm traffic allocation and bucketing for a visitor, look for the values for these three elements in the log message:

  • layerId – the campaign or experiment the visitor was bucketed into

  • variationId – the variation the visitor was bucketed into

  • isLayerHoldback – if true, the visitor was not bucketed into the campaign or experiment even if a variation ID was assigned.

Variation code troubleshooting

This step helps you learn which variation code is being evaluated on the page. Variation code can be synchronous or asynchronous. Optimizely Web Experimentation applies campaign or experiment variation code immediately after the traffic allocation decision if the timing of the changes is set to synchronous, which is the default setting.

For synchronous variation code, which is loaded at the time the snippet loads:

General log format {time} | Optly / Executing actions for Campaign "{campaign name}" ({campaign ID}) Page "{page name}" ({page ID})
Example log  283 | Optly / Executing actions for Campaign "Homepage test" (7729960575) Page "Home" (3536920208)
Search string Executing actions

For asynchronous variation code, which is loaded after the snippet:

General log format {time} | Optly / Executing "load" with arguments: Object
Example log  2059| Optly / API / Executing "load" with arguments:  Object
Search string "load" with arguments

Variation code not being applied:

Below are some scenarios that would cause variation code to not apply. 

Visitor is not eligible for the A/B test or Campaign :

General log format {time} | Optly / NOT applying Variation for Campaign "{campaign name}" ({campaign ID})({reason})
Example log  182 | Optly / NOT applying Variation for Campaign "Homepage test" (818060000)(visitor is not eligible for any Experiments)
Search string NOT applying

Visitor was bucketed into the holdback:

General log format {time} | Optly / NOT applying Variation for Campaign "{campaign name}" ({campaign ID})({reason})
Example log  182 | Optly / NOT applying Variation for Campaign "Holdback experiment" (818060000)(visitor is in layer holdback)
Search string NOT applying Variation

How to find errors with variation code:

General log format {time} | Optly / Action / Failed to apply change:
{{Where the error is coming from)}
{(Reason for the error)}
 
Example log  182 | Optly / Action / Failed to apply change:
{dependencies: Array(0), type: "custom_code", id: "0282ED9C-A0E8-40AE-9C62-0EF40009FCDA", value: ƒ}
n {stack: "ExecError: ReferenceError: myfunction is not define…
Search string Failed to apply change