What is the llms.txt file and does it help SEO?
The llms.txt is a Markdown text file, usually placed at the root of a website, that summarizes what the site is and links its most useful pages so that a language model or an AI agent can find them when it needs to answer. It is an open proposal from 2024, not a standard, and Google Search does not use it.
Key takeaways
- The llms.txt is a proposal by Jeremy Howard published in September 2024: a Markdown index so that an agent finds what it needs when answering, not for training models.
- It does not replace robots.txt: it neither allows nor blocks any bot, it only summarizes the site and links its pages.
- Google clarified on June 15, 2026: Google Search does not use llms.txt files, and having one neither improves nor hurts your visibility.
- Shopify serves a /llms.txt by default on stores, which mirrors the content of /agents.md unless the theme changes it.
- Who really reads it is not something the proposal says: your server logs say it.
If someone has offered you "an llms.txt to show up in ChatGPT," it helps to separate two questions that tend to be sold together: what the file is, which is well defined, and what effect it has on your visibility, which is another matter. The llms.txt is not robots.txt under another name or an XML sitemap for AI. It is a specific proposal, with an owner and a date, and Google has already said what its search engine does with it.
What llmstxt.org proposes
The proposal was published by Jeremy Howard on September 3, 2024, and a second version came out in August 2026, revised with what was learned in two years of use.[1]llmstxt.org (Jeremy Howard). The /llms.txt file, v2 It starts from a problem anyone who has pasted a website into a chat knows: pages are made for people, they wrap the information in navigation, ads and JavaScript, and turning them back into clean text is hard and imprecise. And a model's context window, even though it has grown, is still too small for most entire sites.[1]llmstxt.org (Jeremy Howard). The /llms.txt file, v2
The answer is a file at /llms.txt with a fixed Markdown format:[1]llmstxt.org (Jeremy Howard). The /llms.txt file, v2
- An H1 title with the site's name, which is the only required part.
- A blockquote with a short summary.
- Optional paragraphs with more detail.
- H2 sections with lists of Markdown links, each with an optional note.
- By convention, an "Optional" section for secondary material.
The proposal adds a second piece: that the pages an agent might need have a clean Markdown version at the same URL, ending in .md.[1]llmstxt.org (Jeremy Howard). The /llms.txt file, v2 Version 2 allows adding .md to the full URL or replacing the extension, links those versions with rel="alternate" type="text/markdown" and points to the llms.txt with rel="describedby", and clarifies that an llms.txt can sit in a subfolder: it covers the pages under its path, and the most specific one wins.[2]llmstxt.org. Changes in v2 It is the idea behind Markdown for agents.
Two clarifications from the proposal itself. The first, what it is for: its author expected it to be useful mainly at answer time, not for training models.[1]llmstxt.org (Jeremy Howard). The /llms.txt file, v2 In version 2, an AI agent consults the llms.txt to find what it needs and then follows the links it cares about.[2]llmstxt.org. Changes in v2 The second, what it is: an informal summary open to community feedback, not a standard approved by any body.[1]llmstxt.org (Jeremy Howard). The /llms.txt file, v2 The proposal cites OpenAI, Anthropic and Google (for the Gemini API) among those who publish an llms.txt for their developer documentation.[1]llmstxt.org (Jeremy Howard). The /llms.txt file, v2 Publishing your own is not the same as having yours read, and that is the confusion that moves the most money.
What effect it has on SEO
Here the answer has a date. On May 15, 2026, Google published its guide to optimizing for the generative AI features of Search.[3]Google Search Central. Latest documentation updates It says you do not need to create machine-readable files, AI text files, markup or Markdown to appear in Google Search.[4]Google Search Central. Optimizing your website for generative AI features on Google Search On June 15, 2026 it added a note about llms.txt files:[3]Google Search Central. Latest documentation updates Google Search does not use them, and creating them neither helps nor hurts your site's visibility or rankings, because it ignores them. The same note says there is no harm in keeping one for other services or systems.[4]Google Search Central. Optimizing your website for generative AI features on Google Search The guide is about Search with its generative features, so it holds for AI Overviews and Google AI Mode too.
Chrome treats it as optional. Lighthouse, Chrome's audit tool, has an agentic browsing section with an llms.txt check: it only flags the page if the server returns an error when the file is requested. If the file does not exist and returns a 404, the check is marked "not applicable," because offering it is optional for now, and it does not affect the score.[5]Chrome for Developers. llms.txt (Lighthouse, agentic browsing audits)
For the rest of the assistants, the question is empirical. Each company documents its agents and what they respect, as OpenAI does with OAI-SearchBot, GPTBot and ChatGPT-User,[6]OpenAI. Overview of OpenAI Crawlers but who actually requests your /llms.txt, and how often, only your server's access log tells you.
In a store
On Shopify you do not start from zero. Shopify manages a /llms.txt by default on stores, which mirrors the content of /agents.md, the document the platform treats as the main one for agents to discover the store. The lookup order is: an llms.txt.liquid template from the theme if it exists, otherwise agents.md.liquid, and otherwise the text Shopify generates. The documentation recommends not creating llms.txt.liquid unless you need the llms.txt to say something different from /agents.md, because then keeping it up to date becomes your job.[7]Shopify Dev Docs. llms.txt.liquid Those templates can be customized since May 28, 2026.[8]Shopify developer changelog. Customize /llms.txt, /llms-full.txt and /agents.md
This is what a small photography gear store's llms.txt would look like, in a fictitious example:
# Example Tripods
> Online store of tripods and ball heads for photography and video, shipping across the United States.
Prices in US dollars. Free returns for 30 days.
## Categories
- [Travel tripods](https://store.example/collections/travel-tripods.md): models under 1.5 kg folded
- [Ball heads](https://store.example/collections/ball-heads.md): maximum load, weight and plate type
## Optional
- [Guide to quick-release plates](https://store.example/pages/quick-release-plates.md)
What is useful is in the links: if they point to heavy HTML pages, the file saves little. The proposal wants them pointing to content that is easy for a model to read.[2]llmstxt.org. Changes in v2 Clione, for its part, generates an llms.txt and a Markdown document per product, category, collection and page, served on the store's own domain: how Clione serves files for agents tells how.
What people often get wrong
The llms.txt neither blocks nor allows anything. Bots' access rules go in robots.txt, which follows the Robots Exclusion Protocol,[9]IETF (Koster, Illyes, Zeller and Sassman). RFC 9309: Robots Exclusion Protocol and each company documents which agent respects it and what it uses it for, as OpenAI does with its search agent and its training one.[6]OpenAI. Overview of OpenAI Crawlers If you want your catalog not to be used to train a model, or to appear in an AI search, the place to say so is robots.txt. An llms.txt only summarizes and links, and an agent consults it when it needs to.[1]llmstxt.org (Jeremy Howard). The /llms.txt file, v2
How to check it
- Open
yourdomain.com/llms.txtin the browser. If your store is on Shopify and you have never touched it, you will see the text the platform generates from/agents.md.[7]Shopify Dev Docs. llms.txt.liquid - Read the file the way an agent would: an H1, a one-line summary and links that work and lead to the pages that actually answer your customers, not to the home page and the legal notice.
- Run Lighthouse in Chrome and look at the agentic browsing section. A server error when requesting the file is the only thing the check flags.[5]Chrome for Developers. llms.txt (Lighthouse, agentic browsing audits)
- Search your server or CDN logs for
/llms.txtand note which agents request it. Before believing the name, check the IP: OpenAI, for example, publishes its agents' addresses in JSON files.[6]OpenAI. Overview of OpenAI Crawlers - Do not look for it in Search Console: Google Search ignores the file, so no Google report will show that you have it.[4]Google Search Central. Optimizing your website for generative AI features on Google Search
Related concept: robots.txt
Both files live at the root and end in .txt, and that is where the resemblance ends. robots.txt is a protocol published as RFC 9309 by the IETF in 2022, which tells crawlers which URLs they may request, and the standard itself warns that those rules are not a form of access authorization.[9]IETF (Koster, Illyes, Zeller and Sassman). RFC 9309: Robots Exclusion Protocol llms.txt is a reading guide: the proposal distinguishes it from robots.txt, which indicates what automated access is considered acceptable, because it is used on demand, when an agent needs information. And it distinguishes it from the sitemap, which does not link model-readable versions and accumulates more than fits in a context.[1]llmstxt.org (Jeremy Howard). The /llms.txt file, v2 One decides who comes in; the other, where to start reading once inside.
The comparison of llms.txt, sitemap and structured data puts the three files side by side, and the topic's reading path orders the rest of the terms, starting with indexing.
Synonyms
- English
- llms.txt file
- Español
- archivo llms.txt
Sources
- [1] llmstxt.org (Jeremy Howard). The /llms.txt file, v2. Accessed . ↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑
- [2] llmstxt.org. Changes in v2. Accessed . ↑ ↑ ↑
- [3] Google Search Central. Latest documentation updates. Accessed . ↑ ↑
- [4] Google Search Central. Optimizing your website for generative AI features on Google Search. Accessed . ↑ ↑ ↑
- [5] Chrome for Developers. llms.txt (Lighthouse, agentic browsing audits). Accessed . ↑ ↑
- [6] OpenAI. Overview of OpenAI Crawlers. Accessed . ↑ ↑ ↑
- [7] Shopify Dev Docs. llms.txt.liquid. Accessed . ↑ ↑
- [8] Shopify developer changelog. Customize /llms.txt, /llms-full.txt and /agents.md. Accessed . ↑
- [9] IETF (Koster, Illyes, Zeller and Sassman). RFC 9309: Robots Exclusion Protocol. Accessed . ↑ ↑
Change history
- First English version.
This definition is published under CC BY 4.0. Reuse this definition freely, crediting Clione with a link to this page.
Related terms
robots.txt
What the robots.txt file is, how Googlebot, GPTBot and OAI-SearchBot read it, why it can't remove a page from the index and how to review your store's.
agents.md
What agents.md is: the AGENTS.md format used in code repositories and the /agents.md file Shopify serves on every store for shopping agents.
Markdown for agents
What serving a Markdown version of your pages to AI agents means: .md URLs, content negotiation, Cloudflare's feature and what Google says.
XML sitemap
What an XML sitemap is, which tags Google uses and which it ignores, which URLs in your store belong in it, and how to check it in Search Console.
Web crawler
What a web crawler is, how it discovers your store's product pages, what it respects in robots.txt, and how to check in your logs who is visiting.
Indexing
What indexing is, how it differs from crawling, why Google does not store all your product pages and how to read its status in Search Console.
Make your catalog easy for bots to read
Clione fills every product page with what bots read best: the JSON-LD graph with its structured data, the meta description and the FAQs. You review and press Push; then we check your live store so it all stays published.