Does llms.txt Improve AI Citations?
Oct 24,2026
By SEO ANALYSER
AI search guide
llms.txt is a community proposal, not a ranking signal, and Google says its Search features do not use it. Here is what the specification asks for, what the evidence shows, and when the file is worth publishing.
Short answer
No public evidence shows that llms.txt raises AI citations
llms.txt is a Markdown file proposed in September 2024 to give AI agents a short guide to a site. Google states that Google Search does not use it, and the server-log and citation analyses reported so far found no sign that major AI crawlers request it or that publishing it changes citation frequency. The file is cheap to publish and has narrow, real uses for agents already on a site, but it should not be budgeted as a visibility lever.
Specification
What the llms.txt proposal actually specifies
The proposal, authored by Jeremy Howard and updated since its first publication, defines a Markdown file served from the site root or any subpath, with the most specific file taking precedence. Its structure is strict: an H1 with the site or project name is the only required element, followed by an optional blockquote summary, optional free text without headings, and any number of H2 sections that hold link lists. Each entry is a name and URL with an optional note. By convention, an H2 section called Optional marks links an agent can skip when context is tight.
The intended consumer is an agent working at inference time, fetching a compact guide instead of parsing full HTML pages. The proposal also suggests clean Markdown versions of pages at the same URL with .md appended. Nothing in the specification covers access control, ranking or citation, and it has no conformance test, so a published file is a convention that each consumer may treat differently.
Stated positions
What the parties involved have said
Only one search provider has addressed the file in its own documentation, and the sources reviewed for this guide contain no official statement from the other major AI providers.
| Party | Position | Source type |
|---|---|---|
| Google Search | Does not use llms.txt; no special files or markup are needed for its generative AI features | Google Search Central optimisation guide |
| llms.txt proposal | Convention for agents at inference time; no ranking or citation claims | The specification itself |
| Other AI providers | No official confirmation of reading the file found in the sources reviewed | Not applicable |
Evidence
What independent analyses found
The empirical work comes from agencies and tool vendors rather than from the AI providers, and the figures below were retrieved through secondary reporting, so they are directional rather than audited. Two designs dominate.
Who requests the file
One analysis covering roughly 900 domains over about seven months logged only a little over a thousand requests for llms.txt-family files, with most coming from a commercial data aggregator and ordinary browsers. It reported no requests from the crawlers of the major AI labs.
Whether the file changes citations
A study across roughly 300,000 domains tested whether publishing llms.txt correlated with how often models cited a site. It found no measurable link, and a model predicting citations did slightly better without the llms.txt variable.
Both designs carry limits. Logs miss on-demand fetches that do not identify a crawler and traffic that never reaches the origin, and a null correlation shows any effect sits below the noise floor, not that none exists. The honest reading is absence of evidence from sources with commercial interests, plus a direct statement from Google that its Search features ignore the file.
Real uses
Where the file does get consumed
The documented value sits outside search ranking. Developer documentation sites publish llms.txt so that coding assistants and retrieval pipelines can load a curated index in one request, and the pattern is common in tooling that feeds documentation into an agent's context. In those settings the file behaves like a table of contents the publisher controls, which is why it appears mostly on documentation and developer-product sites. For a content or local-business site, the same reasoning applies only if agents are actually visiting to complete a task, such as finding policy, pricing or purchase instructions.
Log check
How to test llms.txt demand on a specific site
- 01
Publish a valid file
Follow the order in the specification and serve it as plain text or Markdown with a 200 status from the root.
- 02
Filter server logs by path
Search raw access logs for requests to
/llms.txtand/llms-full.txt. Edge or CDN logs are needed where caching hides origin hits. - 03
Group requesters by user agent and IP
Separate verified AI crawlers, aggregators, security scanners and browsers. A request from a browser or scanner says nothing about AI use.
- 04
Compare against normal page fetches
Check whether the same agents fetch ordinary pages without ever requesting the file, which is the pattern the reported analyses describe.
- 05
Track citations before and after
Record a fixed prompt set before publishing, then repeat it later. The method in the guide to AI citation tracking keeps the comparison fair, and any change still needs to be weighed against other edits made in the same period.
Decision
When publishing llms.txt is worth the effort
Cost is low and so are the proven returns, which makes the decision about maintenance rather than risk. A file is justified when a site has structured documentation, an API or product content that agents are expected to read, and an owner who will keep the link list in step with the site. A file left unmaintained points agents at stale or removed URLs, so it should be generated from the same source as the sitemap where possible.
It does not replace crawl controls or discoverability. Access still belongs in robots.txt, and discovery by AI systems still depends on indexable pages, internal links and content worth citing, which is the subject of the wider guide to how to measure AI visibility.
If citations rise after publishing the file, other changes made in the same period are the more likely cause. Reports should list llms.txt as a low-cost experiment, not as an attributed result.
SEO Analyser
Test the effect with measured citations
SEO Analyser's AI Visibility module tracks citations, share of voice and answer inclusion across AI engines such as ChatGPT, Claude and Perplexity, which gives a before-and-after baseline for any change, including a new llms.txt file. Free for 7 days.
Start free trialFrequently asked questions
llms.txt FAQs
Does Google use llms.txt?
No. Google's optimisation guidance for its generative AI features says Google Search does not use llms.txt files, and no special markup is needed to appear in them.
Is llms.txt an official web standard?
No. It is a community proposal published in September 2024, with no conformance test, so each consumer decides whether and how to read it.
Where should the file be placed?
At the site root, or at a subpath to cover the pages beneath it. When several files apply, the most specific one is meant to be used.
Can llms.txt replace robots.txt?
No. The proposal describes a content guide for agents, not an access rule, so crawl permissions stay in robots.txt.
How can llms.txt requests be checked?
By filtering access logs for the file path and grouping requesters by user agent and IP, with edge logs used where caching hides origin requests.
Summary
Publish llms.txt as a cheap, maintained index for agents, not as a citation tactic. Google says its Search features ignore it, reported logs show no major AI crawler requesting it, and any claimed uplift should be tested against a measured baseline.
References
Sources and further reading
- llms.txtThe /llms.txt file (specification)
- GoogleGoogle guide to optimising for generative AI features in Search
- SEO AnalyserAI Citation Tracking Guide
- SEO AnalyserThe Complete Robots.txt Guide
- SEO AnalyserHow AI Visibility Is Measured
- SEO AnalyserAI Visibility
