What is llms.txt?
llms.txt is a plain-text Markdown file served at the root of a domain that gives language models a curated, human-written map of the site: what the organisation is, what it offers, and which URLs matter most. It is to AI assistants roughly what sitemap.xml is to search crawlers — a shortcut past guesswork.
The premise is simple. When an assistant fetches your homepage it gets navigation, cookie notices, hero copy written for atmosphere rather than information, and a footer. It then has to infer what you do. An llms.txt file skips that inference: you tell it, in the plain declarative language models parse most reliably, and you point at the pages that carry the substance.
The format
There is no committee and no validator. The convention that has settled is Markdown, in this order:
- An
#H1 with the name of the site or organisation. - A
>blockquote of one or two sentences summarising what it is. This is the single most-read line in the file — write it as though it is the sentence an assistant will repeat. - Optional prose sections for anything a model needs to get right: legal entities, addresses, what you do and do not sell.
##sections containing lists of links in the form- [Title](URL): one-line description.
Serve it at https://example.com/llms.txt as text/plain. Many sites also publish llms-full.txt, which contains the actual text of the important pages concatenated, so a model can read the substance in one fetch instead of crawling. This site publishes both.
The four mistakes
1. Writing marketing copy
"We are a passionate team of innovators delivering best-in-class solutions" tells a model nothing it can use in an answer. "Siolfur is a growth studio working across GEO, SEO, web development, brand and content, operating from Italy and India since 1987" can be quoted directly. Write facts: what, where, for whom, since when.
2. Dumping the entire sitemap into it
The value of the file is curation. If it lists four hundred URLs it is a sitemap with worse formatting. Pick the twenty pages that would let a stranger understand the business, and leave the rest to sitemap.xml.
3. Letting it go stale
A hand-maintained file drifts within two releases. Generate it at build time from the same data that renders the pages — that way a change to your services or your registered address updates the file automatically, and the two can never disagree. Contradicting yourself is worse than saying nothing, because a model that finds conflicting facts about an entity tends to drop the entity.
4. Publishing it while blocking the crawlers
More common than it sounds. A carefully written llms.txt behind a robots.txt that disallows GPTBot and ClaudeBot is a letter left in a locked room. Check both files together, and check them against each named agent rather than assuming the wildcard covers it.
Is it worth doing?
Does llms.txt actually do anything yet?
Adoption is partial. No major assistant has committed to reading it universally, and it is not a ranking factor in any published sense. It costs twenty minutes, it cannot hurt, and it forces you to write down what your company does in the plain language a model needs — which is worth doing regardless.
That last point is the real argument. Most companies that sit down to write an llms.txt discover they cannot state plainly what they sell and to whom. Fixing that fixes the homepage too.