Improve AI-readiness with llms.txt

  • Updated

Optimizely Configured Commerce automatically generates an llms.txt file within your site's Spire CMS. This file functions similarly to robots.txt and serves as a configuration point for integrating with AI features, letting you specify which content should be made available to large language models (LLMs). By default, the generated llms.txt file is empty, meaning no content is exposed to LLMs unless you explicitly configure it.

You should work with Optimizely Opal for maximum benefits and support. When you point Opal to your website, Opal can provide suggested copy based on the requirements found on  https://llmstxt.org/ using prompts like Based on the requirements found on https://llmstxt.org/, create a llms.txt file for the site [paste domain of website here].

Best practices

An llms.txt file acts as a concise, expert guide for LLMs, helping them understand your website's most important content quickly. 

Keep these things in mind:

  • Start with a clear title and summary – Begin each file with a main heading (# Site Name) and a brief, one-sentence summary in a blockquote (> Summary...) to give the LLM immediate context.
  • Be a curator, not a collector – Think of this file as a curated list of your site's "greatest hits." Instead of listing every page like a sitemap, select only the most critical resources that a user (or LLM) would need to get started.
  • Prioritize clarity and brevity – Use simple language and keep descriptions short and to the point. The goal is to provide high-value information that can be processed quickly, without the clutter of complex HTML and scripts.
  • Use descriptive links – Describe its purpose when you add a link to give the LLM context about why it is important.
    • Good –  - [Quickstart Guide](url): A 5-minute tutorial for new users
    • Less good – – [Quickstart](url)
  • Use the ## Optional section strategically – Place links to secondary or less critical information under an ## Optional heading. This lets an LLM focus on the core content first and access deeper information if needed.
  • Link to LLM-friendly content – Point your links to clean, Markdown-formatted versions of your pages (ending in .html.md) when possible. This makes the content easier for an LLM to read and understand.

Edit the llms.txt file

  1. Go to the Admin Console.
  2. Click View Website and toggle the Content Editor to On.
  3. Select the desired website from the list of websites.
  4. Go to Llms.txt in the Page Tree.
  5. Edit the file as desired.
LLMS file.png