In beta, with limited placesOpen to everyone in Q1 2027

↑↓ to move · Enter to open · Esc to close

Crawling and indexing

What is Markdown for agents?

Markdown for agents is the practice of offering, alongside a page's HTML version, a version of the same content in Markdown for AI agents and language models to read. It is served at its own URL ending in .md or at the same URL, when the agent asks for it with the Accept: text/markdown header.

Written by
Reviewed by
Published 7 min read
Takeaway:

Key takeaways

  • It is the same page in Markdown, without menus, scripts or layout, so an agent reads it with less leftover text.
  • There are two routes: a URL of its own ending in .md, as llmstxt.org proposes, or HTTP content negotiation with the Accept: text/markdown header.
  • What is standard is content negotiation (RFC 9110) and the text/markdown type (RFC 7763); Cloudflare's automatic conversion is a vendor feature, in beta since February 2026.
  • Google says you do not need Markdown to appear in its Search, generative AI features included.
  • The Markdown version has to say the same as the page: showing bots different content to manipulate results is cloaking.

Markdown is a syntax for writing formatted text with simple signs: hash marks for headings, asterisks for bold. "Markdown for agents" is not about writing that way but about serving every page of your site in that format too, so an AI agent reads it without the menu, the scripts or the layout. It is not llms.txt, which is an index of the site, and it is not Shopify's agents.md, which describes the store: it is a clean version of each page. It matters because a product page in HTML carries far more code than content, and the agents that read the web already ask for this version.

What is standard and what is not

Three pieces of very different origin live under the same name, and it pays to separate them before deciding anything.

Content negotiation, which is standard. HTTP allows one address to have several representations of the same resource. Content negotiation works like this: the client says what it prefers with the Accept headers, and the server picks the representation according to those preferences. The response's Vary header says which request headers it took into account when choosing.⁠[1]IETF (Fielding, Nottingham and Reschke). RFC 9110: HTTP Semantics The text/markdown type has existed as a registered type since March 2016: RFC 7763 describes it as a family of plain-text syntaxes that can be converted to HTML, and it requires declaring the character set with the charset parameter.⁠[2]IETF (S. Leonard). RFC 7763: The text/markdown Media Type With those two pieces, an agent can send Accept: text/markdown to the same URL as your product page and get Markdown back.

URLs ending in .md, which are a proposal. The llms.txt proposal suggests that the pages an agent might need have a clean Markdown version at the same URL, with .md added at the end (page.html.md) or replacing the extension (page.md); URLs with no file name add index.html.md or index.md. So a client can find them, it recommends linking them with rel="alternate" type="text/markdown", in the HTML or in an HTTP Link header.⁠[3]llmstxt.org (Jeremy Howard). The /llms.txt file, v2 It is a convention with an author and a date, not a standard from any standards body.

Automatic conversion, which is a vendor feature. On February 12, 2026, Cloudflare introduced Markdown for Agents, in beta.⁠[4]Cloudflare Blog (Celso Martinho and Will Allen). Introducing Markdown for Agents If you turn it on for your zone, when a request arrives with Accept: text/markdown, Cloudflare fetches the HTML from your server, converts it to Markdown and delivers it. It is available at no cost on the Pro, Business and Enterprise plans, only converts HTML and does not convert responses larger than 2 MB. The response carries Content-Type: text/markdown; charset=utf-8, a Vary that includes Accept so the cache stores the two versions separately, a token estimate in x-markdown-tokens, and it loses the ETag and Last-Modified headers.⁠[5]Cloudflare Docs. Markdown for Agents

Piece What it is Who defines it
Accept: text/markdown at the same URL HTTP content negotiation IETF, RFC 9110 and RFC 7763
URL ending in .md and rel="alternate" A convention for finding the clean version The llmstxt.org proposal
Markdown for Agents HTML-to-Markdown conversion on the network Cloudflare, beta feature

The case for it is size. Cloudflare measured its own announcement post: 16,180 tokens in HTML and 3,150 in Markdown, 80% fewer. And it reported that widely used coding agents, such as Claude Code and OpenCode, already send that header when requesting a page.⁠[4]Cloudflare Blog (Celso Martinho and Will Allen). Introducing Markdown for Agents Less leftover text leaves more room in the model's context window for what matters.

What Google says

For Google Search, the answer is short: you do not need to create machine-readable files, AI text files, markup or Markdown to appear in Search, generative AI features included, because Search does not use them.⁠[6]Google Search Central. Optimizing your website for generative AI features on Google Search Markdown for agents is for the other readers: the assistants and agents that request pages to answer or to act.

There is a line not to cross. Google defines cloaking as presenting different content to users and to search engines with the intent to manipulate rankings and mislead users, and gives as an example inserting text or keywords only when the one requesting the page is a search engine.⁠[7]Google Search Central. Spam policies for Google web search A Markdown version that says the same as the page is not that. One that adds arguments, prices or promises the page does not have comes close to it.

In a store

A fictional example: a tripod's product page, requested by an agent using content negotiation.

curl -s -D - -H "Accept: text/markdown" https://example-tripods.com/products/travel-tripod
HTTP/2 200
content-type: text/markdown; charset=utf-8
vary: Accept

# Carbon travel tripod

Folding tripod for mirrorless cameras, with a ball head.

- Price: 189.00 USD
- Availability: in stock

## Frequently asked questions

### Does it fit in carry-on luggage?
Yes, folded it is smaller than a standard carry-on backpack.

What matters is not the format but what it carries: the name, the price, the availability and the answers to the buyer's doubts, without the menu, the footer or the cart. That text has to match the HTML product page; if the price changes in one, it changes in the other. Clione generates one Markdown document per product, with its name, its details, a buying guide, its keywords and its frequently asked questions, and serves it on the store's own domain.

Warning:

What people often get wrong

Turning on Cloudflare's conversion also publishes a signal about how your content can be used. By default, every Markdown response carries the content-signal header with ai-train=yes, search=yes, ai-input=yes, meaning the content can be used to train models, for search results and as agent input, unless your server sends its own header.⁠[5]Cloudflare Docs. Markdown for Agents If you closed the door to training crawlers in your robots.txt, check that this signal does not say the opposite. What to let through and what not is the AI bot blocking decision.

How to check it

  1. Request a product page with the Accept: text/markdown header, with curl as in the example, and look at the response's Content-Type. If text/html comes back, your site does not negotiate the format.⁠[1]IETF (Fielding, Nottingham and Reschke). RFC 9110: HTTP Semantics
  2. Try the proposal's convention: the same URL with .md at the end. A 404 only says you do not use it.⁠[3]llmstxt.org (Jeremy Howard). The /llms.txt file, v2
  3. Look in the product page's HTML for a <link rel="alternate" type="text/markdown">, or in the response headers for a Link with that type: that is how an agent learns the clean version exists.⁠[3]llmstxt.org (Jeremy Howard). The /llms.txt file, v2
  4. If the version exists, compare it with the page: price, availability and text have to match.⁠[7]Google Search Central. Spam policies for Google web search
  5. If you use Cloudflare, check the Vary header (it has to include Accept) and the content-signal header, and decide whether the default signal is the one you want.⁠[5]Cloudflare Docs. Markdown for Agents

llms.txt and Markdown for agents come from the same proposal, but they do different jobs. llms.txt is a single file that summarizes the site and links its pages; the Markdown versions are those pages, one by one, cleaned up. The proposal joins them: llms.txt should link model-readable versions, and each Markdown version can point to the llms.txt that covers it with rel="describedby".⁠[3]llmstxt.org (Jeremy Howard). The /llms.txt file, v2 One is the index; the other, the content.

To keep going, the content negotiation entry explains the HTTP mechanism in more detail, and this topic's reading path orders the rest of the terms.

Synonyms

English
Markdown for agents, markdown version of a page
Español
versión en Markdown para agentes

Sources

  1. [1] IETF (Fielding, Nottingham and Reschke). RFC 9110: HTTP Semantics. Accessed . ↑ ↑
  2. [2] IETF (S. Leonard). RFC 7763: The text/markdown Media Type. Accessed . ↑
  3. [3] llmstxt.org (Jeremy Howard). The /llms.txt file, v2. Accessed . ↑ ↑ ↑ ↑
  4. [4] Cloudflare Blog (Celso Martinho and Will Allen). Introducing Markdown for Agents. Accessed . ↑ ↑
  5. [5] Cloudflare Docs. Markdown for Agents. Accessed . ↑ ↑ ↑
  6. [6] Google Search Central. Optimizing your website for generative AI features on Google Search. Accessed . ↑
  7. [7] Google Search Central. Spam policies for Google web search. Accessed . ↑ ↑

Change history

  • First English version.

This definition is published under CC BY 4.0. Reuse this definition freely, crediting Clione with a link to this page.

Make your catalog easy for bots to read

Clione fills every product page with what bots read best: the JSON-LD graph with its structured data, the meta description and the FAQs. You review and press Push; then we check your live store so it all stays published.