2026-07-29
Does llms.txt Actually Do Anything? What the Data Says in 2026
Agencies are selling llms.txt as an AI-search essential. Google says it does not use it, and crawler data shows almost nothing does. Here is the evidence.
Short answer: no, not for AI search. As of mid-2026, no major AI provider uses llms.txt to decide what to cite or recommend. Google has said so directly. Crawler data confirms it. If a marketing company is selling you llms.txt as the thing that will get you into ChatGPT answers, they are selling you a file that almost nothing reads.
That is an uncomfortable thing for us to publish, because we put an llms.txt on this site and on the sites we build. We will explain why we still do at the end — but the sales pitch that has grown up around it does not survive contact with the evidence.
What llms.txt is supposed to be
The idea, proposed in late 2024, was reasonable: a plain-text file at the root of your site — yoursite.com/llms.txt — that gives AI systems a clean, curated summary of what your business is and which pages matter. Think of it as robots.txt for meaning rather than permission.
It solved a real-sounding problem. Crawlers have to wade through navigation, cookie banners, and boilerplate to find the substance of a page. A curated file skips all that.
The problem is not the idea. The problem is that adoption by the companies that would have to read it never happened.
What the platforms actually say
Google has been the most explicit. In July 2025, Google’s Gary Illyes confirmed Google does not support llms.txt and had no plans to. John Mueller, also at Google, compared it to the keywords meta tag — a reference every SEO understands, because the keywords meta tag was abandoned decades ago after it became pure spam bait.
Google then made it official documentation. On 15 June 2026, Google added a note to its AI-optimization guidance stating plainly that llms.txt files are not required for Google Search.
OpenAI has never claimed to use it either. Its crawler documentation does not mention llms.txt at all. GPTBot does fetch the file occasionally, which some agencies present as proof of adoption — but a bot requesting a URL is not the same as that file influencing how ChatGPT sources, ranks, or cites anything.
What the crawler data shows
Claims are one thing. Traffic is another.
An analysis of more than 500 million AI bot visits over a 90-day window found that 408 of them requested an llms.txt file. That is roughly eight ten-thousandths of one percent of AI crawler traffic.
A separate look at published files found that 97% of llms.txt files receive zero AI requests — while adoption of the format itself has risen 8.8×. More people are publishing it. Almost nothing is reading it.
| Claim you may have heard | What the evidence shows |
|---|---|
| “Google uses llms.txt to understand your site” | Google says it does not support it and does not plan to |
| “ChatGPT reads your llms.txt” | OpenAI’s crawler docs never mention it; occasional fetches are not usage |
| “It’s a required AI-search file” | Google’s own guidance says it is not required |
| “Everyone’s crawler is fetching it” | 408 requests out of 500M AI bot visits |
Why the pitch spread anyway
Three reasons, and none of them are conspiracy.
It is cheap to sell. Writing an llms.txt takes twenty minutes. It produces a visible deliverable a client can open in a browser. Compare that to rebuilding a site’s rendering so crawlers can read it without executing JavaScript, which takes weeks and produces nothing a client can point at.
It sounds current. “We’ll set up your llms.txt for AI search” signals that an agency is paying attention to 2026 rather than 2019. The signal is doing the work, not the file.
Nobody can easily disprove it. AI citations are hard to attribute. If a client starts appearing in ChatGPT six months later, the llms.txt gets credit alongside everything else that happened.
What actually determines whether AI cites you
The research here is more useful than the folklore. A Princeton and Georgia Tech team published GEO: Generative Engine Optimization at KDD 2024, testing nine optimization tactics across a 10,000-query benchmark. The methods that moved visibility were about the content itself:
- Adding quotations from credible sources — up to 41% improvement, the strongest single tactic tested
- Adding relevant statistics — around 37% improvement
- Citing sources — roughly 30–40%
- Keyword stuffing — no improvement, and in places actively negative
Read that list again. Every winning tactic is about making a page more quotable and verifiable. None of them is a configuration file.
Alongside that, three structural things matter, and they are the boring ones:
- Can a crawler read the page without running JavaScript? If your content only appears after a framework hydrates, much of what fetches your site sees an empty shell.
- Is there schema markup? JSON-LD states unambiguously that you are an HVAC contractor, in these towns, offering these services. Without it, a system is inferring that from prose.
- Are there off-site signals? AI systems lean heavily on sources they already trust. User-generated content and established publications carry disproportionate weight — Reddit alone is cited at roughly 40% frequency across major AI engines.
So why does this site still publish one?
Because it costs almost nothing and it is not useless — just useless for the thing it is usually sold for.
llms.txt does get read by a category of tools that is not AI search: IDE agents like Cursor and Copilot, MCP servers, and in-product AI assistants that fetch documentation on demand. If your business publishes technical documentation those tools might consume, the file has a genuine job.
For an HVAC contractor trying to get named when a homeowner asks ChatGPT who to call, it does not. Publish it if you like — it takes twenty minutes and harms nothing. Just do not pay anyone a monthly retainer for it, and do not let it substitute for the work that does move the needle.
The test worth running instead
Rather than checking whether you have an llms.txt, check whether a crawler can actually see your content. In your browser, view the page source — the raw HTML, not the inspector. If your services, service areas, and phone number are not in that raw HTML, they may not exist as far as a large share of AI crawlers are concerned.
That single check tells you more about your AI-search readiness than any file you could add to your root directory.
Sources: Google Search Central AI-optimization guidance (June 2026); Gary Illyes and John Mueller public statements (July 2025); Ahrefs analysis of AI crawler behaviour (January 2026); PPC Land analysis of llms.txt adoption versus AI bot request logs; Aggarwal et al., GEO: Generative Engine Optimization, KDD 2024.