What llms.txt is, with an example
An llms.txt file is a plain-text map of a website written for AI systems: a short description of the site followed by links to its most useful pages, ideally as markdown. It was proposed in September 2024 by Jeremy Howard. It is a convention a site may follow, and no search engine requires it.
Each sounding opens its section
- 01 What the file is for 65 words
- 02 How the file is laid out 115 words
- 03 Markdown copies of pages 43 words
- 04 A live example 51 words
- 05 What it does not do 56 words
- 06 Should you add one? 71 words
- 07 Questions people ask 126 words
- 08 Free competitor gap report 108 words
- 09 More in this series 78 words
- Words
- 713
- Site 366–1,117 · median 749.5
- Sections (H2)
- 9
- Site 3–14 · median 9
- Internal links
- 15
- Site 1–36 · median 19
- File
- 70.3 kB
- HTML, uncompressed · of 83.6 kB budget · 0 scripts
What the file is for
A web page is built for a browser: menus, scripts, styling and the content somewhere among them. An AI system reading the page has to dig the content out. An llms.txt file gives it a shortcut: a single plain-text file that says what the site is and links to the pages that matter, preferably in a clean text form.
How the file is laid out
The proposal sets a simple order, written in markdown.
- A top-level heading with the name of the site or project. This is the only required part.
- A short summary, written as a quotation block.
- Any further paragraphs of explanation.
- Sections under second-level headings, each a list of links with a note beside each link.
A small file looks like this.
# Example Plumbing
> Plumbing and heating repairs in Leeds, with fixed call-out times.
## Services
- [Boiler repair](https://example.com/boiler-repair.md): what is covered and what it costs
- [Emergency call-outs](https://example.com/emergency.md): hours and areas
## About
- [Contact](https://example.com/contact.md): phone, address and opening hours
Markdown copies of pages
The proposal recommends that the links in the file lead to markdown versions of pages, published at the page’s address with .md added. A markdown copy carries the page’s words and headings with none of the furniture around them.
A live example
This site publishes one. Open hadalmedusa.com/llms.txt to see it: the name, a one-paragraph summary, then every page and guide with its markdown copy beside it. It is generated from the same data as the pages, so it never lists a page that does not exist.
What it does not do
Three limits are worth knowing before adding one.
- The file controls nothing. Access is still set in
robots.txt. - Google says its AI features require no such file.
- Reading it is each tool’s choice. It helps the agents that look for it and changes nothing for the ones that do not.
Should you add one?
An llms.txt file costs little and does no harm, and it suits a site whose pages are already clear and well organised. It is not a substitute for that work. A site that is slow, thin or closed to crawlers gains nothing from a map, so the order is: make the site readable first, as how ChatGPT search chooses websites sets out, then add the file.
Questions people ask
Does llms.txt help SEO?
Not in Google. Google states that its AI features need no new machine-readable files or AI text files. The file is read by AI agents and tools that choose to look for it, so its value is in being easy for those systems to use.
Where does the llms.txt file go?
At the root of the site, so that it answers at /llms.txt. The proposal also allows one in a subfolder, describing only the pages beneath it.
What is the difference between llms.txt and robots.txt?
A robots.txt file tells crawlers which addresses they may fetch. An llms.txt file grants and refuses nothing: it points AI systems at the content most worth reading.
Free competitor gap report
An llms.txt file is a small part of how a site reads to a machine. Name up to three competitors and we measure how your site and theirs read to search engines and answer engines, then send you where theirs are easier to use.
- Speed and technical health, side by side
- On-page SEO and site structure
- The content they are found for and you are not
- Who holds the searches that matter in your market
- A prioritised list of what to fix first
- Address
- contact@hadalmedusa.com
- Include
- your site; up to three competitors; where you compete
More in this series
- How to check if ChatGPT recommends your business
- How to get ChatGPT to recommend your business
- How ChatGPT search chooses the websites it cites
- Should you block AI crawlers?
- How to see ChatGPT traffic in Google Analytics
- AI Overviews and falling clicks: how to check, and what to do
- Generative engine optimisation: what it means and what works
- AI search and answer engines: the whole topic
- All guides
- On-site SEO: what the service includes