# llms.txt vs sitemap vs structured data: what each one does > llms.txt, XML sitemaps and structured data on a store: who defines each, who reads it, what it does for your product pages and what Google says about llms.txt. URL: https://clione.ai/comparisons/llms-txt-sitemap-structured-data/ Lang: en | Criterion | [llms.txt](/glossary/llms-txt/) | [XML sitemap](/glossary/xml-sitemap/) | [Structured data](/glossary/structured-data/) | | ----------------------------- | ---------------------------------------------------------------- | -------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | | What it is | A Markdown index of the site, with a summary and links[^llmstxt] | A list of the URLs you want search engines to know about[^google-sitemaps] | Information about a page in a standardized format[^google-sd-intro] | | Who defines it | A proposal by Jeremy Howard, from 2024[^llmstxt] | The sitemaps.org protocol, backed by Google, Yahoo! and Microsoft[^sitemaps-org] | The [Schema.org](/glossary/schema-org/) vocabulary, and each search engine on top[^schemaorg-about] | | Where it lives | One file, at `/llms.txt` | One file or an index of files, at `/sitemap.xml` | Inside each page; Google recommends [JSON-LD](/glossary/json-ld/)[^google-sd-intro] | | What it says about your store | Where to start reading | Which URLs exist and when they changed | What's on each [product page](/glossary/product-page/): name, price, availability | | Who says they read it | Agents that look it up when answering[^llmstxt] | Google, Bing and other search engines[^google-robots-spec] | Google, and Microsoft for its AI systems[^microsoft-ai-search] | | In Google Search | Ignored[^google-ai-guide] | A hint, with no guarantee of crawling[^google-build-sitemap] | Eligibility for [rich results](/glossary/rich-results/); not required for its AI[^google-ai-guide] | | What it doesn't do | Allow or block bots | Guarantee that a URL gets indexed | Guarantee that a rich result shows up | If you got here comparing llms.txt, sitemaps and structured data, chances are someone pitched llms.txt to you as "the sitemap for AI," or as the thing that replaces the markup on your product pages. It's neither. These are three pieces with different owners, different readers and different jobs: **the sitemap tells search engines which URLs exist, structured data says what's on each page, and llms.txt suggests where agents should start reading**. The llms.txt proposal frames it the same way: it's meant to sit alongside the standards that already exist, not to replace them.[^llmstxt] ## Who defines each one **llms.txt is a proposal, not a standard.** Jeremy Howard published it on September 3, 2024, with a second version in August 2026, and its own page presents it as a proposal to standardize the use of an `/llms.txt` file.[^llmstxt] **The sitemap is a protocol the search engines back.** The format is defined by sitemaps.org, at version 0.90, published under a Creative Commons license and, according to the site itself, supported by Google, Yahoo! and Microsoft.[^sitemaps-org] **Structured data has two layers.** The first is the vocabulary, [Schema.org](/glossary/schema-org/): a community activity founded by Google, Microsoft, Yahoo and Yandex, and run by a group with representatives from those companies, one from the W3C and a few individuals who have made significant contributions.[^schemaorg-about] The second is the requirements of whoever reads it. Google, for example, treats the three possible formats equally as long as the markup is valid and follows each feature's documentation, and recommends whichever is easiest to maintain, which in most cases is JSON-LD.[^google-sd-intro] ## Who reads each one **Search engines read the sitemap, and they say so.** Google describes it as a file that search engines read to crawl your site more efficiently.[^google-sitemaps] Google, Bing and the other major search engines accept a `sitemap:` line in [`robots.txt`](/glossary/robots-txt/) that points to the file and isn't tied to any particular agent.[^google-robots-spec] Bing went further in July 2025: with [AI search](/glossary/ai-search/) in [Copilot](/glossary/copilot/) under way, it called sitemaps a foundational signal for making sure every URL on your site is covered.[^bing-sitemaps] **Structured data has declared readers too.** Google uses the structured data it finds to understand the content of your pages.[^google-sd-intro] Microsoft, on its advertising platform's blog, writes that [schema.org markup](/glossary/schema-markup/) helps search engines and AI systems understand your content.[^microsoft-ai-search] **With llms.txt, we know who it's meant for, not who reads it.** According to the proposal, an [AI agent](/glossary/ai-agent/) consults it when it needs information, at the moment of answering; its author expected it to serve that purpose more than model training.[^llmstxt] What the proposal can't tell you is who actually requests it. Google Search has already answered that it doesn't use it, as you'll see below, and the page where OpenAI documents its crawlers explains how to control them with `robots.txt` and never mentions llms.txt.[^openai-bots] For your store, the answer is in your server logs. ## What each one says about your store **The sitemap gives addresses.** Each URL goes in its own tag, with the date it last changed if you choose to provide it. Google ignores `` and ``, and uses `` only if it's consistently and verifiably accurate.[^google-build-sitemap] Bing doesn't use the first two either, and asks that `` reflect the last change to the page's content, not the date the file was generated.[^bing-sitemaps] It says nothing about what a product page sells or at what price. **Structured data gives the content.** Google defines it as a standardized format for providing information about a page and classifying its content.[^google-sd-intro] On a product page, that means its name, its price, its currency and its availability, in named fields. And it comes with a rule the other two don't have: Google asks you not to mark up content that isn't visible to people reading the page,[^google-sd-policies] and, for its AI features, that structured data match the visible text.[^google-ai-features] **llms.txt gives a route.** It carries a title, a summary and lists of links to the most useful pages, ideally to clean Markdown versions of them, the idea behind [Markdown for agents](/glossary/markdown-for-agents/).[^llmstxt] The proposal draws the line against the sitemap: a sitemap lists everything indexable for search engines, while llms.txt offers a selection for a model. A sitemap usually doesn't link to the versions a model reads well or to pages on other sites, and it almost always covers more than fits in a model's [context window](/glossary/context-window/). llms.txt can also point to the structured data the site uses, to help interpret it.[^llmstxt] In one line: **the sitemap says where; structured data, what; llms.txt, where to start.** ## What Google says about llms.txt Google has put it in writing, in its documentation. On May 15, 2026, it published its guide to optimizing for generative AI features on Search, with a section that busts myths about [AEO (Answer Engine Optimization)](/glossary/aeo/) and [GEO (Generative Engine Optimization)](/glossary/geo/), and on June 15, 2026, it added a note on llms.txt files.[^google-updates] The guide says: > "You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them."[^google-ai-guide] And a sentence later: > "It's completely fine if you decide to create and maintain LLMS.txt files (or other similar files) for other services or systems that use these files. Doing so will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them."[^google-ai-guide] Elsewhere in the same guide, "creating unnecessary AI text files (like llms.txt)" is listed among the tactics you can ignore for Google Search.[^google-ai-guide] Three things to keep in mind when you read it: - **Seeing your llms.txt crawled doesn't contradict any of this.** The guide warns that Google may discover, crawl and index many kinds of files besides HTML, and that this doesn't mean it treats them in any special way.[^google-ai-guide] - **It's about Search, not all of Google.** That includes its generative AI features, such as [AI Overviews](/glossary/ai-overviews/) and [AI Mode](/glossary/google-ai-mode/).[^google-ai-guide] Other parts of Google treat the file differently: Lighthouse, Chrome's auditing tool, calls llms.txt an emerging convention and, if your server doesn't have one, marks its check as not applicable, because offering it is optional for now.[^lighthouse-llms] And the [Gemini](/glossary/gemini/) API documentation publishes its own llms.txt.[^gemini-llms] - **Structured data gets different treatment in the same guide.** It isn't required for generative AI search and there's no special schema.org markup to add, but Google says it's a good idea to keep using it as part of your SEO, because it helps with being eligible for rich results.[^google-ai-guide] Lined up, here's what Google Search says about the three: **it ignores llms.txt, recommends structured data and treats the sitemap as a hint**.[^google-build-sitemap] ## llms.txt vs sitemap: where each one stops None of the three does what it's sometimes credited with. - **A sitemap doesn't index anything.** It helps search engines discover your URLs, but it doesn't guarantee that all of them will be crawled and make it through [indexing](/glossary/indexing/),[^google-sitemaps] and submitting it is only a hint: Google doesn't even guarantee it will download it.[^google-build-sitemap] - **Structured data guarantees nothing in the results.** Google doesn't promise that your markup will appear in its results, even when the page is correctly marked up according to its own test.[^google-sd-policies] - **llms.txt neither grants nor removes permissions.** Who may crawl your store is decided in `robots.txt`, which tells automated tools what access is considered acceptable; llms.txt is used on demand, when an agent needs information.[^llmstxt] - **None of them tells you whether your published product page carries what you saved.** That's [publication verification](/glossary/publication-verification/), and how it differs from validating the markup is covered in the [validate vs verify](/comparisons/validate-vs-verify/) comparison. **llms.txt is not the sitemap for AI.** The proposal itself says a sitemap is no substitute for an llms.txt, because it lists everything indexable rather than the versions a model reads well,[^llmstxt] and Google Search, which does use your sitemap as a hint, ignores llms.txt.[^google-ai-guide] It doesn't work the other way around either: dropping the sitemap or the markup on your product pages because "you already have an llms.txt" leaves Google without the two pieces it says it uses. ## On a Shopify store If your store runs on Shopify, two of the three pieces already exist without you lifting a finger. **Shopify generates a `sitemap.xml` at the root of every domain**, which links to separate sitemaps for products, collections, blogs and pages, with each product's main image.[^shopify-sitemap] And by default it manages a `/llms.txt` that mirrors the content of [`/agents.md`](/glossary/agents-md/), the document the platform treats as the main one for agents to discover the store; its documentation recommends against creating your own template unless you need llms.txt to say something different, because keeping it up to date then becomes your job.[^shopify-llms-liquid] Structured data doesn't sit at a URL on the root: it lives on each product page. The guide to [auditing a product page's structured data](/guides/audit-product-page-structured-data/) explains how to read it and compare it with what the page shows. ## Pick where to start You don't choose between llms.txt, a sitemap and structured data: each one answers a different question, and they don't all carry the same weight for your store. - **If Google is what worries you**, start with the sitemap and structured data: they're the two Google Search says it uses, and llms.txt changes nothing in its Search.[^google-ai-guide] - **If your product pages don't show up in search**, the sitemap is a hint, not a fix.[^google-build-sitemap] First find out why they aren't indexed. - **If you want a specific agent to understand your catalog**, an llms.txt won't hurt your rankings in Google and is little work if your platform already generates it,[^google-ai-guide][^shopify-llms-liquid] but only your logs will tell you whether that agent requests it. - **If someone offers you llms.txt as a replacement for the sitemap or the markup**, ask them which agent reads it and how they're going to check. Keeping two of these pieces up to date on a catalog that keeps changing is a separate job, and there are tools that do it for you. Clione, for example, generates an llms.txt and a Markdown document for every product, category, collection and page, served on the store's own domain; publishes a JSON-LD graph on each product page with the organization, the website, the breadcrumbs and the product; and then reads the published product page and alerts you when a verified signal disappears. The page on [Clione's files for agents](/features/llms-txt/) has the details. To keep going, the [crawling and indexing reading path](/learn/crawling-and-indexing/) puts the terms in this comparison in order, along with the ones behind them.