A lot of site owners added llms.txt in 2025 because it sounded sensible. Now it's 2026, and people want receipts. Does the file actually do anything, or is this another fleeting web standard that got loud for a quarter and then quietly died?
Short answer: it depends which AI platform you care about. For three of the four dominant ones — ChatGPT, Claude, and Perplexity — there's documented evidence it works. For Google, Gary Illyes said the quiet part loud. And he wasn't entirely wrong.
Here's the full breakdown.
What Gary Illyes Actually Said (and What He Didn't)
In July 2025, Google's Gary Illyes compared llms.txt to the keywords meta tag — a field Google stopped respecting in 2009 after spammers loaded it with irrelevant terms. That comparison got quoted everywhere, usually without the context around it.
The context: Illyes was specifically warning against using llms.txt to influence Google Search rankings. He didn't say the standard was useless. He didn't say AI platforms should ignore it. He drew a historical parallel as a caution against abuse — which is meaningfully different from a verdict on the standard.
The deeper irony is that llms.txt was never designed for Google Search. Jeremy Howard proposed it in late 2024 specifically for language models — systems that need to understand your site's content, not rank you for keywords. Google Search and AI assistants have entirely different jobs. Applying Google's ranking logic to llms.txt is like evaluating a sitemap by whether it helps your social media reach.
So Illyes was correct within a narrow frame: if you're adding llms.txt hoping to rank higher on Google, stop, because it won't do that. But that's not what the file is for.
The Evidence Scorecard: Platform by Platform
ChatGPT (OpenAI)
OpenAI's GPTBot crawler reads llms.txt files. More telling: OpenAI maintains its own llms.txt on their documentation site — a signal that the company believes in the standard enough to implement it internally. GPTBot uses llms.txt as a navigation layer, letting it identify priority pages without relying entirely on internal link density.
Verdict: High confidence. OpenAI uses the standard themselves. Their crawler processes it.
Claude (Anthropic)
Anthropic runs one of the most detailed llms.txt implementations online. ClaudeBot uses the file to prioritize crawl depth and improve citation accuracy. When Claude cites a specific page rather than just paraphrasing a category, llms.txt structure is part of why that targeting is possible.
Verdict: High confidence. Anthropic's file is actively maintained and publicly verifiable.
Perplexity
Perplexity has been the most explicit of all three. The company confirmed that PerplexityBot assigns higher indexing priority to pages listed in an llms.txt file. Community data from multiple site owners shows increased Perplexity-referral traffic after implementation — not universal, but consistent enough to be a real pattern rather than noise.
Verdict: High confidence. Directly confirmed by Perplexity, supported by community reports.
Google has not adopted llms.txt. Googlebot ignores the file. If Google Search rankings are your only concern, llms.txt has no effect in either direction — no benefit, no penalty.
Verdict: Not applicable for Google rankings. The file is simply not processed.
Bing / Microsoft Copilot
Microsoft is evaluating the standard. No confirmed implementation as of mid-2026.
What "Working" Actually Looks Like in Practice
The question "does it work" needs a specific outcome to be answerable. Here's what the data actually shows:
Crawl depth improves. Without llms.txt, a bot has to infer page importance from link structure, word count, and anchor text patterns. With the file, you're explicitly telling it which pages matter. This has a measurable effect on pages that don't rank by link signals — tools, API docs, case studies, and resources that live three or four clicks from your homepage.
Citation accuracy goes up. When Claude or ChatGPT references your site in a response, they ideally link to the specific page that contains the relevant information. llms.txt descriptions help the model map a query to the right page rather than defaulting to your homepage. Developer community feedback backs this up, though controlled before/after studies are limited.
AI-referred traffic grows over time. Multiple site owners in developer communities reported measurable traffic increases from AI platforms within 30–60 days of adding llms.txt. This isn't a guaranteed outcome — sites with no existing AI search presence won't see dramatic changes overnight. But among sites already receiving some AI-referral traffic, the pattern is consistent enough to be worth noting.
No documented downside. There is no recorded case of llms.txt hurting a site's Google rankings, creating crawl conflicts, or reducing traffic. The risk exposure is zero. The upside is real but variable. That's a different risk profile from most optimization tactics, where tradeoffs cut in both directions.
The Adoption Signal That Says More Than Case Studies
As of early 2026, over 2,000 GitHub repositories reference llms.txt. Anthropic, OpenAI, Vercel, Supabase, and Stripe's documentation team all maintain the file.
When the companies that build AI crawlers use a standard themselves, that's a stronger signal than any individual traffic report. These teams have direct visibility into what their crawlers process. They built the parsers. They maintain the files. The fact that they bother is meaningful.
Why Some People See Limited Results
The most common reason llms.txt underdelivers is file quality, not the standard itself. A bare-minimum file with five links to tag archive pages won't move the needle. A well-structured file with accurate descriptions of genuinely useful pages changes how AI systems model your site.
What belongs in the file:
- Core product and feature pages
- Your highest-quality blog posts (not all of them — your best ones)
- Documentation entry points
- Resource pages that demonstrate subject matter depth
- Key landing pages
What doesn't belong:
- Tag, category, and author archive pages
- Pages under 300 words
- Pages behind authentication
- Admin paths or internal tooling
Optimal size: 20–50 entries. Above 100, you create noise and dilute the priority signal.
Update frequency: Monthly is sufficient. Refresh any time you publish significant new content.
The Verdict
llms.txt works for AI platforms that have adopted it. The evidence for ChatGPT, Claude, and Perplexity is strong enough to justify implementation for any site that cares about AI search visibility. Setup takes under 10 minutes. There is no ranking downside. The upside grows as AI search traffic grows relative to traditional search.
Gary Illyes was right that llms.txt won't influence Google rankings — and right that abusing metadata fields invites problems. Neither of those points is a reason to skip implementing a standard that three major AI platforms actively use.
The real question isn't whether llms.txt works. It's whether you want to be indexed accurately when someone asks ChatGPT, Claude, or Perplexity about your category.
If the answer is yes, the implementation takes less time than reading this article took. Generate your llms.txt file free here.
Frequently Asked Questions
Will llms.txt hurt my Google rankings?
No. Googlebot does not process the file. There is no documented case of llms.txt causing ranking penalties or crawl issues with Google.
How long before I see results from llms.txt?
Community reports suggest 30–60 days for measurable changes in AI-platform referrals. The timeline depends on how frequently AI crawlers index your domain and whether your site already has some AI search presence.
Does llms.txt matter if I only have a small site?
Yes, but the impact is proportionally smaller. The standard adds most value when you have enough content that a crawler would struggle to identify your most important pages without guidance.
Can I generate my llms.txt file automatically?
Yes. Tools like llms-txt-generator.net build a properly formatted file from your sitemap or URL list in minutes, without manual formatting or technical setup.
Is llms.txt an official standard?
It's a community-proposed specification by Jeremy Howard, not a W3C or IETF standard. However, adoption by Anthropic, OpenAI, and Perplexity gives it practical authority regardless of formal ratification status.