Are you new to Optimizely or just getting started with experimentation? If you need to get familiar with Optimizely terminology, you're in the right place. This glossary defines the terms and concepts you'll find in Optimizely's interface, Knowledge Base articles, and Academy.
You can also check out our Optimization Glossary, which defines and describes some general optimization-industry terms.
Use Ctrl+F on PC or Command+F on a Mac to search this page for the exact term you're looking for.
|A/A testing||Comparing the same webpage or mobile app experience against itself to test the accuracy of the testing tool. Read this article for details about running and interpreting an A/A test.|
|A/B testing (or split testing)||Comparing one or more variations of a webpage (or pages) against each another and measuring how the two variations perform for specific defined metrics. Click here for a list of A/B testing ideas from our blog.|
|A/B/n testing||Comparing multiple versions of webpage or mobile app against each other to determine which performs best.|
|Account||Highest level of the hierarchy in your Optimizely plan. All of your projects, experiments, campaigns, variations, audiences, events, and collaborators are part of your Optimizely account.|
|Administrator||Optimizely collaborator or user role with access to all features of the account, including billing.|
|Algorithm||The mathematical formula used to execute against the chosen strategy. For a given optimization strategy, there are several possible algorithms we could use and an infinite number of ways to tune the algorithm’s parameters. See the FDR Control with Adaptive Sequential Experimental Design white paper for technical details.|
|Audience||Group of visitors who share a characteristic, such as the way they came to your page or the browser they use. You target experiments based on audiences.|
|Average order value (AOV)||Average dollar amount spent each time a visitor places an order on a website or mobile app. AOV is a common metric for e-commerce and retail sites.|
|Body content||Main column of content on a webpage. "Body content" is different from the HTML term "body," which means ALL the content that is on a webpage.|
|Breadcrumb||Increasingly specific category links visitors see at the top of a webpage that indicate the categorization pathway or trail a visitor followed to the current page. An example breadcrumb trail is Home > Clothing > Mens > Jeans.
For ideas for experiments that involve breadcrumbs, check out this article.
|Bucket||Visitors who see a given variation are "bucketed" into that variation. Similarly, a visitor can be "bucketed" into an experiment, meaning that the visitor is eligible to see the experiment's variations. For an example of how visitor bucketing works, see this page.|
|Click event||Event that measures whether a visitor clicked a certain element on the page.|
|Collaborator roles||Level of access for users within Optimizely. Optimizely collaborator roles include Administrator, Project Owner, Editor, and Viewer.|
|Conversion||Action or event that a visitor did or did not do. Examples of conversions include making a purchase, completing a form, reaching a page, and clicking a button.|
|Conversion rate||Number of conversions on a website, webpage, or mobile app divided by the total number of visitors. On the Results page for your experiments and campaigns, you'll see the conversion rate for each metric you have set.|
|Conversion rate optimization (CRO)||Using A/B testing to increase the conversion rate for a website, webpage, or mobile app.|
|Call to action (CTA)||Primary button or link on a page that you want visitors to click.|
|Content management system (CMS)||Platform that allows you to edit content and publish to a website without coding an HTML page. With CMS platforms, you can centralize data editing, publishing, and modification from a single back-end interface. You can implement Optimizely on various CMS platforms.|
|Cross-project events||Optimizely feature that helps you track events across all of your Optimizely Web projects. For more information about cross-project events, click here.|
|Custom attribute||Attribute of your visitors that you add through Optimizely's API. Unlike Audiences, which have binary (yes/no) values, custom attributes can have multiple values.|
|Direct entry||When a visitor enters a website's domain by typing in the exact address rather than by another means like clicking a search engine result or a referral link on a social media site. An example of a direct entry is a visitor typing "www.optimizely.com" in the URL address bar to go to Optimizely's website.|
|Editor (collaborator or user role)||Optimizely collaborator or user role with access to create variations in a specific project, but not start or pause experiments. Administrators and Project Owners can grant access to a project to Editors.|
|Editor (code)||Section of the Optimizely interface where you can edit your variations within an experiment. Optimizely has the Visual Editor and also allows you to incorporate custom code.|
|Engagement metric||Average or counting metric that tabulates to what extent a visitor did a particular actionin contrast to a conversion metric, which tabulates whether a visitor did or did not do something. Example engagement metrics include average pageviews and time on site.|
|Events||The success metric by which experiments are evaluated. Click here to learn more about the differences between events and metrics.|
|Exact match||URL match type that runs an experiment or campaign only when a visitor loads the exact URL you specify.|
|Experiment||Optimizely's term for an A/B, multivariate, or multi-page test. Also called a test or a campaign.|
|Extensions||Optimizely feature developers can use to create custom features for non-technical teams to use. Click here for details about Extensions.|
|Full factorial testing||Testing that runs every single possible combination as it's own unique variation. Full factorial testing is different from the Taguchi method (or fractional factorial testing), which runs a limited number of variations based on the variations that are expected to be the most different from each other.
The benefit of full factorial testing is there is no guesswork. The downside is that by running so many variations, you need a lot of traffic to run the test.
|Funnel (or sales funnel)||Path a visitor takes through a website or mobile app as they move toward becoming a customer.|
|Headline testing||Developing multiple title variations for an article or other item published on the web to determine which title performs the best.|
|Heatmap||Report that shows where visitors click on a page. Heatmaps are usually displayed as an overlay, with more intense colors indicating more visitor clicks.|
|Hero image||Large image at the top of a webpage. Hero images are sometimes called "rotating heros" if they are a slideshow that rotates through several images.|
|Home page||Page in the Optimizely interface from which you can manage projects and experiments.|
|Implementation (or snippet implementation)||Adding the Optimizely snippet to your webpage. Check out this article for more information on implementing the snippet for Optimizely.|
|Improvement (or lift)||Relative difference in performance of one variation vs. another, expressed as a percentage.
For example, if one campaign converts at 5% and the alternate converts at 10%, that represents 100% improvement. If the alternate converts at 20%, that represents 400% improvement.
Here's how improvement is calculated:
(Winning conversion rate % - Old conversion rate %) - Old conversion rate % = % Improvement
|IP address||Internet protocol address: the numerical label assigned to any device on a network. IP targeting and IP filtering are based on visitors' IP addresses.|
|Landing page optimization||Improving elements on a landing page to increase conversions.|
|Lead generation||Generating consumer interest for a product or service with the goal of turning that interest into a sale. Check out our ideas for lead generation testing.|
|Marketing technology stack||Group of technologies that a marketer uses to conduct and improve their marketing activities.|
|Metric||Quantitative measurement of a visitor action. Metrics are based on events and help you measure differences in visitor behavior based on changes you make to your site in experiment variations or personalized experiences.|
|Metrics builder||Natural-language interface for metrics in Optimizely that allows you to define metrics that more closely match your business needs, view the same event as multiple metrics, and decide the winning direction for a metric.|
|Monthly unique visitors (MUVs)||Individual visitors who access your site within 30-day window. Optimizely tracks unique visitors with a cookie and bills your account based on the number of MUVs who enter an experiment on your site. Check out this article for details about MUVs.|
|Multivariate testing (MVT)||Testing that runs multiple variations of specific elements. The experiment is defined by selecting the elements and their variations, and the testing platform automatically generates the possible testing combinations. For example, 3x2 MVT means changing 3 elements with 2 variations per element. If a full factorial test is running, this will result in 3^2 (i.e., 9) variations.|
A class of problems in probability theory where an agent must decide how to best distribute resources among several options with initially unknown payout structures. The term "multi-armed bandit" derives from a person, the “bandit,” playing slots. Given a sack of coins and several slot machines with unknown payout ratios, the bandit must decide how to distribute the coins to get the highest payout. At first, the bandit does not know what each machine will pay out, but as she pulls each one she gets a better understanding.
See the definition for regret minimization below for a solution to the multi-armed bandit problem.
|Multi-page funnel testing||Testing that includes different changes across multiple pages to measure the difference in visitor behavior across a funnel (such as product > cart > checkout) or across a whole site. In multi-page funnel testing, when a visitor sees a variation on one page, that visitor will continue to see the variation across all other pages in the experiment.|
|Online marketing||Strategy for leveraging web-based channels to spread a message about a company’s brand, products, or services to potential customers.|
|Optimization strategy||The optimization strategy defines the goal or objective a user chooses when enabling Stats Accelerator. We offer two optimization strategies: Accelerate Learnings and Accelerate Impact. These are often referred to as time minimization and regret minimization frameworks, respectively.|
|Organic search||When a visitor clicks a non-paid link in search engine results to go to a website.|
|Original (or baseline)||Experience that is used to compare against a variation in an experiment or campaign.|
|Pageview event or pageview goal||Event or goal that measures whether a visitor views a specified page.|
|Pause||Stop running an experiment or campaign.|
|Plan||Optimizely pricing and features package.|
|Preview tool||Optimizely feature that lets you toggle between variations, evaluate code execution timing, and navigate your site without re-initiating to make sure your variations look and work the way they should. Check out this article for information about the Optimizely Preview tool.|
|Primary metric||Most important goal of your experiment—the action that you most want visitors to take. The primary metric determines which variation in your experiment is the winner.|
|Project||Subsection of your Optimizely account. Each project has its own snippet, set of experiments, and set of collaborators. Learn more about projects in Optimizely.|
|Project Owner||Optimizely collaborator or user role with access to all features for a specific project, except billing. The Project Owner role is similar to the Administrator role, but does not have access to billing.|
|Query parameter||Letters and numbers included in the query string portion of a URL that pass data to web applications. They look like this: ?item=true
You can use query parameters to optimize based on paid ad campaigns and search engine marketing. You can also force behaviors with query parameters.
|Query string||Sring of query parameters, separated with an ampersand. They look like this: ?item1=true&item2=false|
|Rail||Smaller column on the left or right side of a webpage that is not part of the "body content."|
An experiment where one or more variations redirects the visitor to a different URL from the original page (also sometimes referred to as a "split URL test"). Redirects are often used when testing two entire pages against each other, for example in a landing page test or site redesign.
|Regret minimization||A framework or strategy that seeks to minimize the number of times a new visitor is shown a sub-optimal experience with respect to the primary metric. In doing so, the framework minimizes the level of quantified “regret” observed after the conclusion of the experiment and maximizes the payout.
Regret minimization is the traditional approach to solving a multi-armed bandit problem. In Stats Accelerator, this is called Accelerate Impact.
|Regular Expressions (or RegEx)||Syntax used to match text strings. Regular Expressions (or RegEx) is an Optimizely match type you can use for URL Targeting.|
|Results page||Section of the Optimizely interface where you can see the results of your experiments and campaigns and determine the winning variation. For more details, see this article about the Results page in Optimizely.|
|Revenue per visitor (RPV)||Average dollar amount spent by each customer on a website or mobile app. RPV is a common metric for e-commerce sites.|
|Search engine marketing (SEM)||Paid search campaigns. In SEM, companies pay search engines like Google to show their paid links when certain search conditions are met.|
|Search engine optimization (SEO)||Techniques performed to get a website or webpage to display higher in natural search results. Examples of SEO include incorporating keywords into content and making sure keywords appear high on the page. For information about A/B testing and SEO, check out this article.|
|Secondary metrics||"Next-best" actions that you want visitors to take, after the event that's set as your primary metric. Secondary metrics typically occur "up the funnel" and are leading indicators of success. In Optimizely, the metrics you have set in places 2 through 5 are your secondary metrics.|
|Segmentation or visitor segmentation||Analyzing only a subset of traffic in an experiment's results to see if that subset's behavior is different from the average. On your Results page, you can segment results by audience or dimension, depending on your plan.|
|Session||A session is a period of activity for a user. An existing session ends and a new session begins after 30 minutes of inactivity on the site. The maximum session length is 24 hours, at which point a new session automatically begins (even if the user wasn’t inactive for 30 minutes). Learn more about how Optimizely counts conversions.|
|Shopping cart abandonment||When a potential customer starts the check-out process for an online order but drops out of the process before completing a purchase.|
|Simple match||URL match type that will match a given URL, without taking into account https://, query parameters, and other common URL modifications. Simple match is an Optimizely match type you can use for URL Targeting.|
|Split URL test||See redirect experiment.|
|Statistical significance||Likelihood that the difference in conversion rates between a given variation and the baseline is not due to chance.|
|Stats Engine||Optimizely's proprietary way of calculating experiment results that allows you to interpret results without waiting for a fixed test duration and reduces the chance of false positives.|
|Substring match||URL match type that will match a string within a URL. For example, 'ample' is a substring match for 'example.com'. Substring match is an Optimizely match type you can use for URL Targeting.|
|Taguchi method||Reducing the number of variations in a multivariate test (MVT) so that not every single variation has to be tested. Taguchi method MVT results in displaying the "best predicted" variation, even if that variation was not actually tested.
Optimizely uses full factorial MVT instead of Taguchi method MVT.
|Targeting||Determining whether visitors will be included in an experiment. In Optimizely, URL Targeting determines whether an experiment will run on a certain URL, and Audiences determine whether an experiment will run for visitors who have certain characteristics.|
A framework or strategy that aims to identify and distribute visitors to variations with a relative likelihood of reaching statistical significance that is higher than other variations. To be technically precise, time minimization addresses a set of problems that are not defined as multi-armed bandit problems.
Optimizely developed this framework. In Stats Accelerator, it is called Accelerate Learnings.
|Unique visitors||Distinct individuals who visit a page during a given period, regardless of how often they visit. Unique visitors are distinct from visits, which are the number of times anyone has gone to your site within a given period. A unique visitor may have several visits.|
|URL (or web address)||Abbreviation for Uniform Resource Locator. The web address of a website, as you would see it in the address bar of your browser.|
|URL Targeting||Method of specifying the pages where your experiment or campaign will run.|
|User flow||Path taken by a prototypical user on a website or mobile app to complete a task.|
|Value proposition or "value prop"||Essence of the value that your product or service provides to visitors and customers.|
|Variation||Alternate experience that is tested against the original (or baseline) in an A/B test.|
|Viewer||Optimizely collaborator or user role that can only view experiments and campaigns.|
|Visitor||Individual person who goes to your website.|
|Visits||Number of times anyone has gone to your website within a given period. A unique visitor may have several visits.|
|Visual Editor||Section of the Optimizely Editor that allows you to edit your variations visually, using a WSYIWYG (or "what you see is what you get") editor. See this article for more information about the Visual Editor in Optimizely.|
|Winner or winning variation||Best-performing variation of a test based on the most important metrics. An experiment or campaign may have multiple winners if there is no single clear metric that defines which variation will be implemented. Click here for details about determining the winner or winning variation.|
If you're looking for a certain term that isn't listed here, please let us know by clicking the blue "Feedback?" tab on the right side of this page.