Guide
What is llms.txt and how do you use it for your documents?
llms.txt is a proposed standard: a Markdown file at /llms.txt that gives AI models a short, curated map of a site — a title, a summary and lists of links to the most useful content, ideally in clean Markdown. For documents, it works as an index that tells agents which files exist, what each one is, and where to read a text version instead of parsing PDFs.
What the proposal says
The llms.txt proposal (llmstxt.org) addresses a simple problem: websites are built for browsers, with navigation, scripts and layout that waste a model’s limited context. A small Markdown file at a known address lets a model find what matters quickly.
- Location: /llms.txt at the root of a site (the proposal also allows it under a subpath).
- An H1 with the name of the project or site — the only required part.
- A blockquote with a short summary.
- Optional paragraphs or lists with more detail.
- H2 sections containing lists of links in the form [name](url), optionally followed by a short note.
- An “Optional” section for secondary links that can be skipped when context is short.
- Pages are encouraged to offer a clean Markdown version at the same URL with .md added.
What it looks like for a set of documents
For a company’s documents, llms.txt is a table of contents for AI. A short example:
# Example Co — Investor documents
> Current investor materials for Example Co. Figures are checked against the source against source files.
## Documents
- [Q3 2026 investor update](https://example.com/docs/q3-update.md): quarterly results, unaudited
- [Pitch deck, October 2026](https://example.com/docs/deck.md): company overview and round terms
## Optional
- [Archive of earlier updates](https://example.com/docs/archive.md)
Good practice for a document llms.txt
- Link to text versions (.md), not to PDFs — the point is to save agents from parsing layouts.
- Write one line per document that says what it is and its period: “Q3 2026, unaudited” beats “Update”.
- List only current versions; move superseded ones to Optional or remove them.
- Keep URLs stable so that agents and people who saved them land on the latest version.
- Do not list documents you do not want public. llms.txt is a public file; anything restricted needs real access control.
- Update it whenever you add or replace a document.
What llms.txt does not do
It is a proposal, not a guarantee: it helps agents that look for it, and you should not assume every AI tool reads it. It also says nothing about whether the linked content is correct. If a linked Markdown file was produced by a converter that dropped a column header, llms.txt points agents straight to the error.
So llms.txt is the index; the quality of what it points to is a separate problem.
How The Papers uses it
Every AI Data Room in The Papers has an llms.txt at /{room}/llms.txt and an index at /{room}/index.md. They list the room’s documents and point to each one’s AI Version in Markdown (/{room}/{doc}.md) and JSON (/{room}/{doc}.json), each checked against the source before publishing. Unlisted and private documents are not exposed to the public. Updating a document keeps its URL, so the index stays valid, and analytics show when an AI agent read the llms.txt.
FAQ
- Is llms.txt the same as robots.txt?
- No. robots.txt tells crawlers what they may access; llms.txt suggests what is worth reading and where to find clean versions.
- Do ChatGPT and Claude read llms.txt?
- It is a proposed convention; support varies by tool and is not guaranteed. It costs little to publish and helps agents that use it.
- Should llms.txt link to my PDFs?
- Prefer Markdown versions. Linking to PDFs brings back the extraction problems llms.txt is meant to avoid.
- Can I have an llms.txt for just one folder?
- The proposal allows llms.txt under a subpath, covering pages below it — useful for a documents section or a data room.
Give your documents an llms.txt index that points to fidelity-checked text versions.
Sources
Updated 2026-10-27