GEO
What Is llms.txt and Should Your Site Have One?
Key takeaways
- llms.txt is a markdown proposal for guiding AI agents; only a top-level heading is required.
- It supplements robots.txt and sitemaps rather than replacing them.
- The proposal's own page says AI labs publish llms.txt for their own developer docs; it does not claim that providers use it to rank or cite other sites.
- Google says there are no additional requirements or special optimizations needed for AI Overviews and AI Mode.
What llms.txt is
llms.txt is a proposal to standardise a file at /llms.txt that gives AI agents a concise guide to a website. The author, Jeremy Howard, describes it as a markdown document with brief background information, guidance and links to more detailed markdown files. The specification page lists version 2 as published on September 3, 2024.
The required format
The proposal sets an order for the contents:
- An optional byte-order mark.
- An H1 heading with the name of the site or project. This is the only required element.
- A blockquote with a short summary.
- Optional free-form sections with more detail.
- Zero or more H2 sections, each containing a markdown list of links in the form name, URL and optional notes.
Because it is plain markdown, it is easy for people and language models to read, and easy to generate automatically from a list of pages.
How it differs from robots.txt and sitemaps
Robots.txt tells crawlers what they may fetch. A sitemap lists URLs for discovery; Google says submitting one is a hint, not a guarantee, and that it uses lastmod only when it is consistently and verifiably accurate. llms.txt does neither job. It is a curated reading guide: the handful of pages and short descriptions you would want an agent to see first. Treat it as a complement, not a substitute.
What is proven and what is not
The proposal's page states that AI labs publish llms.txt files for their own developer documentation, naming OpenAI, Anthropic and Gemini. That is evidence of use for documentation, not a statement that those providers read llms.txt on other sites or that it changes how a site is cited. Separately, Google's guidance on AI features says there are no additional requirements to appear in AI Overviews or AI Mode, nor other special optimizations necessary. Nothing in these sources shows llms.txt improving rankings or citations, so expectations should stay modest.
Should your site have one?
It costs little: a short file and a few minutes. It can help a person or tool that does read it, and it forces you to decide which pages are your most important. The reasonable stance is to publish one if maintenance is automated, keep it accurate, and not expect a measurable lift. If it would go stale or take real effort, put the time into the content and technical basics Google lists instead.
A practical template
Start with the site name as an H1, add a one-sentence blockquote describing who you are, then list your main pages, your most useful articles and your profile links under H2 headings. Keep each description to one line and update the file whenever you publish.
Frequently asked questions
Is llms.txt an official web standard?
No. It is described by its author, Jeremy Howard, as a proposal to standardise a file that helps agents use a website.
Does llms.txt improve rankings or AI citations?
The sources reviewed do not show that. The proposal says AI labs publish llms.txt for their own docs, and Google says no special optimizations are needed for AI Overviews or AI Mode.
What is the only required part of llms.txt?
An H1 heading with the name of the site or project. Everything else in the format is optional.
Sources
- The /llms.txt file — llmstxt.org (Jeremy Howard)
- AI features and your website — Google Search Central
- Build and submit a sitemap — Google Search Central
Keep reading
See how AIrecommend.ai helps businesses get found and recommended by AI assistants.