Key takeaways
- What it is: llms.txt is a Markdown file at yourdomain.com/llms.txt that summarises your site and links to your most useful pages for AI tools.
- Google doesn't use it: Google says Search itself doesn't use llms.txt files, and you don't need one to appear in its AI features.
- AI search barely reads it: In Ahrefs' study of 137,210 domains, 97% of llms.txt files received zero requests in May 2026.
- Where it helps: Sites with developer documentation, because coding agents fetch the file more often than AI search tools do.
llms.txt gets sold as two things: "robots.txt for AI" and a quick way into ChatGPT's answers. It's neither. It sets no rules for crawlers, and the AI search tools have shown little interest in reading it.
You've probably been told you need one, maybe by a plugin, a developer or an agency quote. So should you add it? Will it help your SEO? Spend a week on it for the wrong reasons and the work that actually shapes AI answers waits another week.
This guide explains what llms.txt is and what the evidence says about who reads it. Then it covers which sites should still make one, and how to write one in under an hour. It's based on the llms.txt proposal itself, Google's and Chrome's documentation, and a June 2026 Ahrefs study of 137,210 websites.
What llms.txt is
llms.txt is a plain-text file, written in Markdown, that sits at yourdomain.com/llms.txt. It gives large language models (LLMs) and AI agents a short summary of your site and a curated list of links to the pages that matter most.
Jeremy Howard of Answer.AI proposed it in September 2024. The proposal describes it as a way "to provide information to help agents use a website". It has since been revised as version 2, which also lets you place an llms.txt in a subfolder, such as /docs/llms.txt.
It's easy to confuse with two files you probably already have:
| File | What it does | Does it set rules? |
|---|---|---|
| robots.txt | Tells crawlers which parts of your site they may access | Yes. Major search and AI crawlers follow it |
| sitemap.xml | Lists the URLs you want search engines to find | No. It's a hint for crawling |
| llms.txt | Summarises your site and links your key pages for AI tools | No. It's a suggestion, and few AI tools ask for it |
The practical point: if you want to allow or block AI crawlers, that still happens in robots.txt. llms.txt can't do it.
Does llms.txt help SEO or AI visibility?
Not with Google, and there's little evidence it helps with other AI search tools. Google's guide to generative AI search says you don't need new machine-readable or AI text files, and that "Google Search itself doesn't use them". It adds that keeping one for other services is fine.
The bigger question is whether anyone else reads it. In June 2026, Ahrefs checked 137,210 websites that use its analytics tool. 28% published an llms.txt file. Of those files, 97% received zero requests during May 2026.
The biggest share of requests came from SEO audit tools checking whether a site has the file. AI search barely showed up:
AI search bots made just 1.1% of requests to llms.txt files.
Share of all requests to llms.txt files, by type of visitor. About 38,000 sites with an llms.txt file, May 2026. Bars start at zero.
Source: Ahrefs, llms.txt study, published 15 June 2026. Not all categories shown. A request shows the file was downloaded, not that it was used.
Two caveats from Ahrefs itself. Its customers lean technical, so real adoption is probably lower than 28%. And a fetch only proves a file was downloaded. Even the small AI share is a ceiling on how much the file is used, not a floor.
Who should still make one
Sites whose readers include developers or AI agents. In the Ahrefs data, Claude Code, Anthropic's coding assistant, was the second most active named bot, ahead of every AI search and assistant bot. Coding tools use llms.txt to find documentation fast.
| Your site | Worth making one? |
|---|---|
| Software, API or developer documentation | Yes. Coding agents do request it, and it helps them find the right pages |
| SaaS with a large help centre | Probably. It's cheap, and AI agents may use it to answer product questions |
| Service business, clinic, hotel or agency | Optional. It does no harm, but don't expect more AI visibility from it |
| Online shop | Optional. Your product pages and Merchant Center feed matter far more |
How to create an llms.txt file
If you decide to make one, it takes under an hour. The only required part is a title. Everything else is a short summary and lists of links.
- Start with a title: one line beginning with # and your site or company name.
- Add a one-line summary: a line beginning with > that says what you do and for whom.
- Group your key pages: a heading beginning with ## for each group, then a list of links written as [Page name](URL), each with a short note after a colon.
- Add an Optional section: a ## Optional heading for pages an AI tool can skip if it's short on space.
- Publish it at the root: save it as llms.txt in plain text and upload it so it loads at yourdomain.com/llms.txt.
- Keep it current: update it when you add, move or remove key pages. A file full of dead links is worse than none.
Here's what a finished file looks like for an example payroll software company:
# Example Payroll
> Payroll and HR software for companies with 10 to 500 staff in Southeast Asia.
## Product
- [Features](https://example.com/features): what the software does
- [Pricing](https://example.com/pricing): plans and prices in USD
## Help
- [Getting started](https://example.com/docs/start): set up your first payroll run
- [API reference](https://example.com/docs/api): endpoints for developers
## Optional
- [Blog](https://example.com/blog): guides and company newsWhat to do instead for AI visibility
If your goal is to be named in ChatGPT, Perplexity or Google's AI answers, three things matter far more than llms.txt.
- Let AI crawlers in: check that robots.txt allows OAI-SearchBot and PerplexityBot, and that your host doesn't block them. A site those crawlers can't reach can't be cited.
- Answer questions clearly: put a direct one or two sentence answer under each heading, and cover the obvious follow-up questions.
- Get mentioned elsewhere: AI answers lean heavily on what other sites say about you, so reviews, press and industry lists count.
Our guide to getting your brand cited in AI answers walks through each step. If you're wondering how this fits with your existing SEO, see AEO vs SEO: what changes and what doesn't.
That's also why we don't charge for llms.txt files on our AEO service. It isn't the power-up it's sold as. We spend that time on the work above.
FAQs
Is llms.txt mandatory?
No. llms.txt is a voluntary proposal, not a web standard or a requirement from any search engine. Google says its Search doesn't use the file, and Chrome's Lighthouse marks a missing llms.txt as not applicable because providing one is optional. If you don't have one, make sure yourdomain.com/llms.txt returns a normal 404 page rather than a server error.
Will an llms.txt file help my SEO?
Not with Google. Google's guide to generative AI search says you don't need AI text files and that Google Search itself doesn't use them. There's also little sign that AI search tools rely on the file: in Ahrefs' study of 137,210 domains, 97% of llms.txt files received zero requests in May 2026. Your time is better spent on crawler access, clear answers and mentions on trusted sites.
Where do I put the llms.txt file?
Put it at the root of your domain, so it loads at yourdomain.com/llms.txt, the same way robots.txt does. Save it as plain text written in Markdown. The latest version of the proposal also allows an llms.txt in a subfolder, such as /docs/llms.txt, covering the pages under that path. Whatever you choose, link only to pages that are public and up to date.
What is the difference between llms.txt and robots.txt?
robots.txt sets rules: it tells crawlers which parts of your site they may access, and the major search and AI crawlers follow it. llms.txt sets no rules. It's a Markdown summary of your site with links to your most useful pages, offered as a guide for AI tools. Blocking or allowing AI crawlers still happens in robots.txt, not in llms.txt.
Does ChatGPT read llms.txt?
OpenAI hasn't said that ChatGPT uses llms.txt when it answers. Its crawlers do sometimes fetch the file. In Ahrefs' May 2026 data, GPTBot, OpenAI's training crawler, made 4.51% of all requests to llms.txt files. OAI-SearchBot, which finds pages for ChatGPT search, made 0.74%. A fetch shows the file was downloaded, not that it shaped an answer.
Guy founded Wincon Marketing in Bangkok after nearly ten years in SEO, content and paid media, both agency-side and in-house. He moved early into AEO as AI assistants started answering buyers' questions, and now helps brands get found on Google and in AI answers.
Connect on LinkedIn →