Glossary of Optimizely terminology

This glossary defines the terms and concepts found in Optimizely's Experimentation interface, articles, and Academy.

See Learn marketing terminology for general optimization-industry terms.

• A/A test – A comparison of the same webpage or mobile app experience with itself to test the accuracy of the testing tool. Read Run and interpret an A/A test for details.
• A/B test (or split test) – A comparison of the variations of a webpage (or pages) with each other and measurement of how the two variations perform against the defined metrics.
• A/B/n testing – A comparison of multiple versions of a webpage or mobile application with each other to determine which performs best.
• Account – Highest hierarchy level in your Optimizely plan. All projects, experiments, campaigns, variations, audiences, events, and collaborators are part of your Optimizely account.
• Administrator – Optimizely collaborator or user role that has access to all features of the account, including billing.
• Algorithm – The mathematical formula that the chosen strategy executes. You can use several algorithms and innumerable ways to tune the algorithm’s parameters for a strategy.
• Audience – A group of visitors who share a characteristic, such as the way they come to your page or the browser they use. You target experiments based on audiences.
• Average order value (AOV) – The average dollar amount a visitor spends each time they place an order on a website or mobile app. AOV is a common metric for e-commerce and retail sites.
• Body content – The main column of content on a webpage. Body content differs from the HTML term body, which represents all the content on a webpage.
• Breadcrumb – The specific category links visitors see at the top of a webpage that indicates the categorization pathway or trail a visitor follows to the current page. An example of a breadcrumb trail is Home > Clothing > Mens > Jeans. 
  See 10 common experiments and how to build them for experiments that involve breadcrumbs.
• Bucket – Visitors who see a given variation are bucketed into that variation. Similarly, a visitor can be bucketed into an experiment, meaning that the visitor is eligible to see the experiment's variations. See Bucket Testing for examples on how visitor bucketing works.
• Click event – An event that measures the number of times a visitor clicks a certain element on the page.
• Collaborator roles – Level of access for users within Optimizely. Collaborator roles include Administrator, Project Owner, Editor, and Viewer.
• Conversion – An action that a visitor completes. Examples are, make a purchase, complete a form, reach a page, and click a button.
• Conversion metric – A measurement of how effectively you convert your audience into paying customers.  
• Conversion rate – The number of conversions on a website, webpage, or mobile app per visitor. You can see the conversion rate for each metric you set on the Results page for your experiments.
• Conversion rate optimization (CRO) – The use of A/B testing to increase the conversion rate for a website, webpage, or mobile app.
• Call to action (CTA) – The primary button or link on a page that you want visitors to click.
• Content management system (CMS) – A platform that lets you edit content and publish to a website without coding an HTML page. With CMS platforms, you can centralize data editing, publishing, and modification from a single back-end interface. You can implement Optimizely on various CMS platforms.
• Cross-project events – A feature that lets you track events across your Optimizely Web projects. See Cross-project events for more information.
• Custom attribute – An attribute of the visitors that you add through Optimizely's API. Unlike Audiences, which have binary (yes/no) values, custom attributes can have multiple values.
• Direct entry – When a visitor enters a website's domain by typing in the address rather than by other means like clicking a search engine result or a referral link on a social media site. An example of a direct entry is a visitor typing www.optimizely.com in the URL address bar to go to Optimizely's website.
• Editor (collaborator or user role) – Optimizely collaborator or user role with access to create variations in a specific project, but not start or pause experiments. Administrators and Project Owners can grant access to a project to Editors.
• Editor (code) – Optimizely interface pane where you can edit your variations within an experiment. Optimizely has the Visual Editor and also lets you incorporate custom code.
• Engagement metric – A measurement of the extent to which a visitor makes a particular action. For example, average pageviews and time on site.
• Events – The success metric by which experiments are evaluated. See Difference between events for more information.
• Exact match – URL match type that runs an experiment or campaign only when a visitor loads the exact URL you specify.
• Experiment – Optimizely's term for an A/B, multivariate, or multi-page test. Also called a test or a campaign.
• Extensions – An Optimizely feature that developers can use to create custom features for non-technical teams. See Create reusable templates for custom features using Extensions for details.
• Fractional factorial testing – Testing that runs a limited number of variations that are the most different from each other.
• Full factorial testing – Testing that runs every possible combination as its own unique variation.
• Funnel (or sales funnel) – The path a visitor takes through a website or mobile app as they become a customer.
• Headline testing – Testing that develops multiple title variations for an article or other items published on the web to determine which title performs the best.
• Heatmap – Report that shows where visitors click on a page. Heatmaps display as an overlay. Color saturation directly correlates with number of visitor clicks.
• Hero image – A large image at the top of a webpage. Hero images are also called rotating heroes if they are slideshows that rotate through several images.
• Home page – Page in the Optimizely interface from which you can manage projects and experiments.
• Implementation (or snippet implementation) – The addition of the Optimizely snippet to your webpage. See Implement the one-line JavaScript snippet.
• Improvement (For A/B experiments) – Optimizely displays the relative improvement in conversion rate for the variation over the baseline as a percentage for most experiments.

  This is true for all A/B experiment metrics, including binary and numeric conversions.

  For example, in Optimizely Experimentation, a relative improvement interval of 1% to 10% means that the variation sees between 1% and 10% improvement over baseline. If the baseline conversion rate is 25%, you can expect the variation conversion rate to fall between 25.25% and 27.5%.

  See Confidence intervals and improvement intervals for more information. 

• IP address (Internet protocol address) – The numerical label assigned to any device on a network. IP targeting and IP filtering are based on visitors' IP addresses.
• JavaScript (or JS) – Computer programming language. The Optimizely snippet is written in JavaScript. You can include custom JavaScript in an experiment or use custom JavaScript to target experiments.
• jQuery – Optimizely snippet includes the JavaScript library and can be used to create variation code. Learn more about jQuery in Optimizely.
• Landing page optimization – Improvement of elements on a landing page to increase conversions.
• Lead generation – Generation of consumer interest for a product or service to make a sale. See Testing ideas for lead generation and B2B sites.
• Marketing technology stack – Group of technologies that a marketer uses to conduct and improve their marketing activities.
• Metric – Quantitative measurement of a visitor action. Metrics are based on events and let you measure differences in visitor behavior based on changes you make to your site in experiment variations or personalized experiences.
• Metrics builder – Natural-language interface for metrics in Optimizely that lets you define metrics that match your business needs, view the same event as multiple metrics, and decide the winning direction for a metric.
• Monthly unique visitors (MUVs) – Individual visitors who access your site within a 30-day window. Optimizely tracks unique visitors with a cookie and bills your account based on the number of MUVs who enter an experiment on your site.
• Multivariate testing (MVT) – Testing that runs multiple variations of specific elements. You select the elements and their variations, and the testing platform automatically generates the possible testing combinations. For example, 3x2 MVT means changing 3 elements with 2 variations per element. If a full factorial test is run, this results in 3^2 (i.e., 9) variations.
• Multi-armed bandit – A class of problems in probability theory where an agent decides how to best distribute resources among several options with initially unknown payout structures. The term multi-armed bandit derives from the bandit, who plays slots. Given a sack of coins and several slot machines with unknown payout ratios, the bandit must decide how to distribute the coins to get the highest payout. At first, the bandit does not know what each machine will pay out, but as they pull each one, they get a better understanding.
• Multi-page funnel testing – Testing that includes different changes across multiple pages to measure the difference in visitor behavior across a funnel (such as, product > cart > checkout) or across a whole site. In multi-page funnel testing, when a visitor sees a variation on one page, that visitor continues to see the variation across all other pages in the experiment.
• Online marketing – Strategy to leverage web-based channels to spread a message about a company’s brand, products, or services to potential customers.
• Optimization strategy – The optimization strategy defines the goal or objective a user chooses when enabling Stats Accelerator. Optimizely offers two optimization strategies – time minimization and regret minimization.
• Organic search – When a visitor clicks a non-paid link in a search engine results page to go to a website.
• Original (or baseline) – Experience that is used to compare against a variation in an experiment or campaign.
• Pageview event or pageview goal – An event or a goal that measures whether a visitor views a specified page.
• Pause – To stop running an experiment or campaign.
• Plan – Optimizely pricing and features package.
• Preview tool – Optimizely feature that lets you toggle between variations, evaluate code execution timing, and navigate your site without re-initiating to make sure your variations look and work the way they should. See Preview and publish your experiment for more information.
• Primary metric – The most important goal of your experiment—the action that you most want visitors to take. The primary metric determines which variation in your experiment is the winner.
• Project – The subsection of your Optimizely account. Each project has its own snippet, set of experiments, and set of collaborators. See Manage projects for more details.
• Project Owner – Optimizely collaborator or user role with access to all features for a specific project, except billing. The Project Owner role is similar to the Administrator role but does not have access to billing.
• Query parameter – Letters and numbers included in the query string portion of a URL that pass data to web applications. For example, ?item=true
  You can use query parameters to optimize based on paid ad campaigns and search engine marketing. You can also force behaviors with query parameters.
• Query string – String of query parameters, separated with an ampersand. For example, ?item1=true&item2=false
• Rail – Smaller column on either side of a webpage that is not part of the "body content."
• Redirect experiment – An experiment where one or more variations redirect the visitor to a different URL from the original page (sometimes referred to as a split URL test). Redirects are used when two entire pages are tested against each other, for example, in a landing page test or site redesign.
• Regret minimization – A framework that minimizes the number of times a new visitor sees a sub-optimal experience with respect to the primary metric. In doing so, the framework minimizes the quantified regret observed after the experiment's conclusion and maximizes the payout.
  Regret minimization is the traditional approach to solving a multi-armed bandit problem. 
• Regular Expressions (or RegEx) – The syntax used to match text strings. Regular Expressions (or RegEx) is an Optimizely match type you can use for URL Targeting.
• Results page – Section of the Optimizely interface where you can see the results of your experiments and campaigns and determine the winning variation. See The Experiment Results page for more details.
• Revenue per visitor (RPV) – The average dollar amount spent by each customer on a website or mobile app.
• Search engine marketing (SEM) – Paid search campaigns. In SEM, companies pay search engines like Google to show their paid links when certain search conditions are met.
• Search engine optimization (SEO) – Techniques to display a website or webpage higher in search results. Examples of SEO include incorporating keywords into content and making sure keywords appear high on the page. See A/B testing and SEO for more information.
• Secondary metrics – Next best actions after the event that's set as your primary metric. Secondary metrics typically occur up the funnel and are leading indicators of success. In Optimizely, the metrics you have set in places 2 through 5 are your secondary metrics.
• Segmentation or visitor segmentation – Analyzing a subset of traffic in an experiment's results to see if that subset's behavior differs from the average. On your Results page, you can segment results by audience or dimension, depending on your plan.
• Session – A session is a period of activity for a user. An existing session ends, and a new session begins after 30 minutes of inactivity on the site. The maximum session length is 24 hours. A new session automatically begins after 24 hours (even if the user was not inactive for 30 minutes). Learn more about how Optimizely counts conversions.
• Shopping cart abandonment – When a potential customer starts the check-out process for an online order but drops out of the process before completing a purchase.
• Simple match – URL match type that matches a given URL, without considering https://, query parameters, and other common URL modifications. A simple match is an Optimizely match type you can use for URL Targeting.
• Snippet (or Optimizely snippet) – Line of JavaScript code that you place on your page so you can use Optimizely. Learn more about the Optimizely Web Experimentation Snippet.
• Split URL test – See redirect experiment.
• Statistical significance – A measure of how unusual your experiment results would be if there was no difference between your variation and the baseline and the difference in lift was due to random chance alone.
• Stats Engine – Optimizely's proprietary way of calculating experiment results that lets you interpret results without waiting for a fixed test duration and reduces the chance of false positives.
• Substring match – URL match type that matches a string within a URL. For example, 'ample' is a substring match for 'example.com'. Substring match is an Optimizely match type you can use for URL Targeting.
• Taguchi method – A way to reduce the number of variations in a multivariate test (MVT) so that every variation does not have to be tested. Taguchi method MVT results in displaying the best predicted variation, even if that variation was not actually tested.
  
  Optimizely uses full factorial MVT instead of Taguchi method MVT.
• Targeting – Determining whether visitors are included in an experiment. In Optimizely, URL Targeting determines whether an experiment runs on a certain URL, and Audiences determine whether an experiment runs for visitors with certain characteristics.
• Time minimization – A framework that identifies and distributes visitors to variations with a relative likelihood of reaching statistical significance that is higher than other variations. Time minimization addresses a set of problems that are not defined as multi-armed bandit problems.
   
  Optimizely developed this framework, Stats Accelerator.
• Unique visitors – Distinct individuals who visit a page during a given period, regardless of how often they visit. Unique visitors are distinct from visits, which is the number of times anyone visits your site within a given period. A unique visitor may have several visits.
• URL (or web address) – Uniform Resource Locator is the web address of a website, as you see it in your browser's address bar.
• URL Targeting – Method to specify the pages where your experiment or campaign runs.
• User flow – The path taken by a prototypical user on a website or mobile app to complete a task.
• Value proposition – The value of your product or service to visitors and customers.
• Variation – An alternate experience that is tested against the original (or baseline) in an A/B test.
• Viewer – Optimizely collaborator or user role that can only view experiments and campaigns.
• Visitor – Individual person who goes to your website.
• Visits – The number of times anyone goes to your website within a given period.
• Visual Editor – Section of the Optimizely Editor that lets you edit your variations visually, using a WYSIWYG (or what you see is what you get) editor. See Visual Editor in Optimizely Web Experimentation and Optimizely Performance Edge for more information about the Visual Editor.
• Winner or winning variation – Best performing variation of a test based on the most important metrics. An experiment or campaign can have multiple winners if no single clear metric defines which variation will be implemented. See Take action based on the results of an experiment for more information.