What Is llms.txt, and Does It Work? What the Evidence Says in 2026

llms.txt is a proposed plain Markdown file at the root of a website that gives AI systems a curated, readable map of the site. Does it work? On the evidence available in November 2026, there is no public confirmation that major AI search products use it to find or cite pages, and Google says its own Search does not.

That does not make it worthless, and it costs an afternoon, not a budget. It does mean you should not expect it to move rankings or citations, and should spend effort on the things that clearly do matter first. Sources were read on 6 November 2026.

What is llms.txt?

The proposal comes from Jeremy Howard and was first published on llmstxt.org in September 2024; the page I read shows a revision dated August 2026. The idea: a model reading a website struggles with navigation menus, scripts and sprawling page structure, so give it a short file that lists the pages that matter, in clean text, with a one-line description of each.

The spec describes a Markdown file with these parts, in order: a required H1 with the site or project name (the only mandatory part); a blockquote summary; optional free text with more detail; and zero or more H2 sections that contain lists of links, each link with an optional description after a colon. A section named "Optional" is, by convention, secondary material an agent can skip when it needs a shorter context. The file can sit at the site root or at a path inside the site to cover that area.

The proposal also suggests offering clean Markdown copies of key pages at the same address with .md added, and linking them with rel="alternate" type="text/markdown". It is a proposal from one author, not a standard published by a standards body.

How do you build an llms.txt file?

1. Pick the pages. Choose the 15 to 40 pages that answer what customers and assistants ask most: main services or products, pricing explainers, policies, a few strong guides. Leave out thin pages, tag archives and checkout flows.
2. Write the header. Start with # Your Company, then a blockquote of one or two plain sentences saying what you do, where, and for whom. Facts that never change belong here: legal name, location, contact route.
3. Group links under H2 headings such as Services, Guides, Company. Format each line as a Markdown link followed by a colon and a short description: - [Page title](https://example.com/page): what it answers.
4. Keep descriptions factual. Do not stuff keywords or make claims you would not put on the page itself. A model reading contradictory text trusts neither.
5. Publish at /llms.txt as plain text, return a 200 status, and mention it in your sitemap process so it stays current. Add a reminder to update it when pages are added or removed.
6. Optionally add Markdown versions of your best pages if your CMS can generate them cheaply.

For reference, Website Prism publishes one at websiteprism.com/llms.txt. When I read it, it had a title and summary followed by seven sections and about 90 links covering services, case studies, articles and company pages. We publish it because it is cheap and harmless, not because we can show it changes citations.

Do AI systems actually use llms.txt?

Here is what can be checked.

• Google. Google's guidance on generative AI features in Search says you do not need to create new machine-readable files, AI text files, markup or Markdown to appear in AI Overviews or AI Mode. Its optimisation guide names llms.txt among misconceptions, says Google Search does not use such files, and says creating them for other services is fine and will neither help nor harm visibility in Google Search.
• Google's John Mueller. Search Engine Journal reported on 2 June 2026 that John Mueller called llms.txt "purely speculative" for now, said no AI systems use it, and said that if an AI platform asks for the file, site owners can consider making one. That is a reported statement about the AI industry in general, not a measurement. I could not find a primary page with the original wording.
• OpenAI, Anthropic, Perplexity and others. I found no official documentation from them saying their crawlers or assistants read llms.txt to choose sources. OpenAI's bots page describes its crawlers and robots.txt controls, and does not mention llms.txt.

The honest summary: evidence of use is limited, and the one major search company that has addressed it says its search does not use it. Absence of documentation is not proof that no product ever reads the file, and that may change. If a tool you care about asks for one, make one.

What actually helps AI systems cite your site?

These have clearer support than a new text file.

Let the right crawlers in

OpenAI documents separate crawlers: OAI-SearchBot, which surfaces sites in ChatGPT's search features; GPTBot, which relates to model training; and ChatGPT-User, which fetches pages when a person asks. Each can be allowed or blocked independently in robots.txt. If you block OAI-SearchBot, OpenAI says you can still appear as a plain link but your site is kept out of its search answers. A blanket block added to stop training can therefore also remove you from AI search. Check your robots.txt, your CDN's bot rules and any firewall settings, because many sites block AI crawlers by accident. Google's own documentation recommends allowing crawling for Search eligibility in the same way.

Write answer-first pages

Put the direct answer in the first two sentences, use question-style headings, and keep facts specific and dated. Assistants quote passages that stand alone. Tables for comparisons and short definition paragraphs get lifted more easily than long narrative. Our guide to getting cited by ChatGPT and Google AI Overviews walks through that page structure.

Keep structured data honest

Google says no special schema.org markup is required for AI features, and that markup should match visible text. Structured data still helps machines understand organisation details, products and authorship, so use it where it is accurate and skip it where it is not. Our WordPress and Shopify SEO service covers the technical side, including markup that matches the page.

Earn mentions elsewhere

Assistants often draw on what other sites say about a brand. Reviews, directory listings, industry articles and community answers give a model independent confirmation. This is an observed pattern in how answers are sourced, not a guarantee, and no vendor publishes a rule for it.

What mistakes make an llms.txt useless?

• Listing every URL. A file with thousands of links is just a sitemap in a different format; the point is curation.
• Describing pages with marketing slogans. Short, literal descriptions tell a reader what question each page answers.
• Letting it go stale, with links to pages that now redirect or return errors.
• Putting private or unfinished pages in it. The file is public.
• Treating it as access control. It does not block anything; use robots.txt and authentication for that.

Should you publish an llms.txt?

Yes if the cost is small, you have a clean page list, and you can keep it current. Skip it, or delay it, if your robots.txt blocks AI crawlers, your key pages are thin, or nobody will maintain the file. Fix those first. Tracking matters too: check your server logs for requests to /llms.txt from the user agents above so that your decision rests on your own data, not on opinion. Our GEO and AI search visibility service covers the audit steps that sit around this, from crawl access to page structure.

Frequently asked questions

What is llms.txt in simple terms?

A Markdown file at yoursite.com/llms.txt with a site summary and a curated list of key page links, meant to help language models find the important content quickly.

Does llms.txt improve Google rankings?

No. Google's guidance says Google Search does not use such files and that having one neither helps nor harms rankings or visibility.

Does ChatGPT read llms.txt?

I found no official OpenAI documentation saying so. OpenAI documents its crawlers and robots.txt controls instead. Allow OAI-SearchBot if you want to appear in ChatGPT search.

How is llms.txt different from robots.txt?

Robots.txt tells crawlers what they may access. llms.txt is a content guide; it grants or blocks nothing, and a crawler can ignore it.

Is it worth the effort?

For a clean, small site it is a low-cost addition. Treat it as optional housekeeping, behind crawl access and strong answer-first pages.