ChatGPT Said It Worked Two Weeks Ago, Now It Calls It a Joke: A Fake Standard About Office Cats Broke All Four Proofs the GEO Industry Sells

ChatGPT Said It Worked Two Weeks Ago, Now It Calls It a Joke

On 4 August 2026, Mark Williams-Cook published How cats.txt showed llms.txt evidence is GEO astrology. His motivation was blunt: he was tired of watching "an AI bot fetched it" and "ChatGPT said it helps" get treated as evidence that llms.txt does anything. So he invented cats.txt — a "web standard" requiring a plain text file at your domain root declaring your office cats, their job titles, their breeds, and a mandatory affection metric called PurrLevel scored out of ten. He wrote a formal draft specification, published it, and tested it against the four proofs the industry uses to sell llms.txt. All four passed. PerplexityBot, GPTBot, ClaudeBot and Googlebot all queued up to fetch it. Google indexed it. Google's AI Overview gave a confident bulleted answer, with a citation, about a fictional cat named Odd, job title "Render Cat", PurrLevel 5/7. ChatGPT explained at length that cats.txt "could help AI systems trust, summarize, and cite your content more accurately." Search Engine Journal reported it on 7 August, and it headlined the industry weekly on 11 August.

This blew up now because two conditions arrived at once. First, llms.txt deployment finally reached statistical significance. Second, server log analysis became cheap enough for anyone to run. Ahrefs' study across 137,000 domains found roughly 28% (38,360 domains) publish a valid llms.txt — the authors flag this as an upper bound, since Ahrefs' customer base skews more technical than the web at large — but 97% of those ~38,000 files received zero requests in May 2026. The top three requesters were SEO audit tools at 21.7%, unidentified bots at 14.9%, and general web crawlers at 13.1%. AI retrieval bots — the ones that answer user queries inside AI search products — accounted for 1.1% of total requests. Google's position hardened over the same period: the late-May AI optimization guide states, in a section literally titled "mythbusting", that machine-readable files like llms.txt are not needed to appear in generative AI search, and John Mueller put it plainly on Bluesky: "no AI system currently uses llms.txt … It's super-obvious if you look at your server logs."

The problem is not one file. It is the measuring stick shared by an entire category of GEO tactics. Other data from the same period shows the same fracture. Similarweb's 2026 Generative AI Landscape report found 65% of URLs cited by ChatGPT sit two or three folders deep, while 58.8% of the referral traffic lands on homepages. Previsible analysed 6.77 million AI-referred sessions across 166 sites and found 28.8% arrive at internal site search pages. Cited pages and visited pages are separate populations, and most GEO dashboards draw them as one line. Meanwhile the direction Google is actually spending engineering effort on is WebMCP — a browser API co-edited by Google and Microsoft engineers, incubated in a W3C community group, already running an origin trial in Chrome, that lets a site declare its own features as callable tools for AI agents. The category is splitting in half: engineering actions you can verify in a log, and rituals sold with a story.

For small and mid-sized businesses and freelance studios, the practical takeaway is not "delete your llms.txt." It is that every GEO proposal on your desk should pass one test first: if the content on this page were nonsense, would this piece of evidence still hold? If it would, it is not evidence. Below: where each proof fails, what three groups of readers should do today, which tools genuinely verify anything, and how to check it yourself for nothing.

What Happened, With the Full Numbers

Start with why each of the four "proofs" fails, because they will be copied wholesale onto the next GEO tactic.

• Proof 1: AI crawlers fetched it. Fetching things is a crawler's entire job description, not an endorsement. The cats.txt logs filled with GPTBot, ClaudeBot, PerplexityBot and Googlebot, all dutifully requesting a file describing the professional responsibilities of cats. catstxt.org even runs a live filtered log viewer so you can watch it happen. A fetch proves the URL exists and returns a 200.
• Proof 2: Google indexed it. Google has been indexing text files for two decades. Being in the index is a statement that a URL exists and contains words — not a verdict on truth, usefulness or sanity. cats.txt is indexed, and Search Console will cheerfully offer to let you claim it and see ranking data.
• Proof 3: A model repeated a fact found only in the file. That is exactly what ordinary retrieval-augmented generation does. The model searches, lands on a URL that ranks because it is indexed, and reads what is on it. At that moment your llms.txt is functioning as a web page, not as a standard.
• Proof 4: ChatGPT says it works. A model telling you something is a good idea only proves that a lot of text online says it is a good idea. Williams-Cook calls this the convergence problem: the model returns a running average of the discourse. About two weeks after launch, ChatGPT said cats.txt "can potentially help you rank in both search engines and LLM-driven systems." Once the joke was widely written up, the same question returned "it's a joke." The file never changed.

The scale numbers matter too. In the Ahrefs data, of roughly 38,000 valid llms.txt files only about 1,100 received any requests at all, and 96% of those requests came from bots. Broken down, GPTBot accounted for 4.51% of requests to those files, ClaudeBot 0.80%, DeepseekBot 0.02%. In other words, when your dashboard shows "llms.txt is being crawled," the most probable source is the audit tool you are paying for. Independent replication points the same way: on 9 August the technical blog Cittago published its own server logs showing 4 requests to llms.txt in 23 hours, none of them from an AI crawler.

The same Ahrefs dataset contains three numbers that are harder to argue with than the 97%:

• AI never goes looking for it. Of requests to llms.txt files that do not exist, 98% came from humans (mostly SEOs checking on competitors) and the AI bot share was zero. So "publish a file so AI can discover you" does not hold up in the data — AI fetches it only when a link or an instruction says it exists.
• Slackbot fetched llms.txt more often than PerplexityBot did. A chat app's link-preview bot out-fetched one of the AI search engines the file was supposedly designed to court.
• 12% of requests come from tools studying the standard itself. GEO/AEO audit tools at 5.8%, llms.txt directories and validators at 3.6%, research crawlers at 2.7%. The largest single research crawler identifies itself as prompt-injection-survey/1.0 — someone is systematically studying llms.txt as a prompt injection surface, precisely because agents are designed to trust it.

Google also took both sides in the same week: the late-May guide says you do not need it, and days later the Chrome team shipped an llms.txt check inside Lighthouse's experimental Agentic Browsing audits. That audit produced 22 requests in the entire dataset — roughly 1 in 1,000.

What Three Groups of Readers Should Do Today

Brand owners and SMB leaders

• Lay out this quarter's GEO/AEO proposal and ask one question per line item: "if this page were false, would this evidence still hold?" Where it would, ask the vendor to re-evidence it with server logs or first-party reports.
• Do not delete your llms.txt because of this article. It costs almost nothing to keep. What should be cut is the service fee attached to it as a headline deliverable.
• Set a purchasing rule: any AI visibility performance report must contain at least one column of first-party data (server logs, Search Console, GA4). A report that is entirely third-party sampling is not a report.

Marketers and SEO practitioners

• Stop using "I asked ChatGPT whether this is right" as validation. The answer flips with the volume of discourse, and the same model will contradict you in six months.
• Report "crawled", "indexed", "cited" and "generated revenue" as four independent columns, never as one score. Similarweb's 65% versus 58.8% is direct evidence those columns move separately.
• Run a landing page audit for AI referral traffic. Since most of it arrives at the homepage and internal search pages, treat those two as conversion pages, not brochure covers.

Developers and agencies

• Turn your client's AI crawler server logs into a queryable table. Minimum fields: timestamp, user-agent, source IP, URL, status code, response size. This is the only place any GEO claim can be falsified.
• Warn clients that logs only show bots that honestly declare themselves. Fetches from residential IPs or spoofed user-agents never appear, so the log is a lower bound, not the whole picture.
• Start evaluating WebMCP. It uses navigator.modelContext to expose existing front-end functionality as agent-callable tools; for sites with clean forms the work is far smaller than standing up a back-end MCP server. Crucially, whether a tool was called is something you can verify.

Tool Comparison: Which Ones Prove Something Was Actually Read

Only one dividing line on this table matters: the top three measure what machines actually did, the bottom three measure whether you show up when a self-chosen prompt is asked. The first group is falsifiable; the second can only ever confirm. The most expensive line item in a GEO proposal usually hangs off the second.

What Nobody Puts in the Pitch Deck

• This is not a claim that llms.txt is harmful — the expensive part was never the file. Williams-Cook explicitly says he is not arguing llms.txt will never work. The cost is not in 2KB of text; it is in the service fee charged for packaging it as a proven lever into AI answers, plus the work you did not do instead.
• The "97% zero requests" number deserves the same scrutiny. Ahrefs' sample comes from its own customer base, which skews technical, and server logs only see crawlers that declare an honest user-agent. Fetches via residential proxies or spoofed agents are invisible. It is strong counter-evidence, not an absolute.
• The "it's being crawled" line in your dashboard is probably your own tool. SEO audit tools were the single largest requester of llms.txt files at 21.7%. Paying for a tool to check a file, then reading in a report that the file gets traffic, is a closed loop you funded.
• The real readership is coding agents, not search. In the Ahrefs classification, AI agents and agentic infrastructure are the largest AI category at 10.5%, and Anthropic's Claude-Code out-fetched every AI retrieval bot, every AI assistant and every training crawler. Mueller describes the file as a "temporary crutch, perhaps to save some tokens" for AI coding tools. So if your customers use Claude Code or similar to source recommendations, the file has a genuine use — that use is just not AI search visibility.
• There is a real security dimension. Agents are built to trust the contents of llms.txt, and the largest research crawler in the dataset calls itself prompt-injection-survey/1.0. If you publish one, treat it like code: version control, restricted edit rights, alerts on unauthorised changes, plain links and descriptions rather than anything instruction-shaped, and human review of anything a platform auto-generates.
• The convergence problem contaminates every "ask the AI to audit us" workflow. That includes asking ChatGPT to review your SEO, Gemini to assess your content strategy, or Claude to set your priorities. The model returns the current average of opinion, not a judgment. Opinion turns; your site changes do not turn back.

The No-Subscription Version

Everything here runs on tools you already have, in an afternoon:

• Step 1: Get the logs. Shared hosting usually exposes them under cPanel's Raw Access. On a VPS, read /var/log/nginx/access.log. If you use Cloudflare, the free tier already shows an AI crawler breakdown.
• Step 2: Count llms.txt first. Use grep 'llms.txt' access.log | awk '{print $1, $12}' | sort | uniq -c | sort -rn to see who fetched it in the last 30 days and how often. For most SMB sites the answer will be zero or single digits.
• Step 3: Count the real content pages. Run the same command against your three most important service page URLs and compare request counts for GPTBot, ClaudeBot, PerplexityBot and OAI-SearchBot. That number is the actual foundation of your AI visibility.
• Step 4: Find the pages nobody fetches. Diff the URLs present in the logs against your sitemap and list every page no AI crawler has ever touched. Fix their load speed, robots.txt rules and internal linking before you touch llms.txt.
• Step 5: Add a first-party control. The Search Console generative AI performance report is free first-party impression data. Put it side by side with your log fetch counts. When they disagree, believe these two, not the sampling tool.

• ☐ Downloaded the last 30 days of server logs
• ☐ Counted actual requests to llms.txt and their sources
• ☐ Counted AI crawler requests to the top three service pages
• ☐ Listed pages never fetched by any AI crawler
• ☐ Connected the Search Console generative AI report as a control
• ☐ Ran every GEO proposal line item through the "would it hold if the content were false" test

FAQ

So should I publish an llms.txt or not?

Keeping one is fine — just do not pay a service fee for it, and manage it like code (version control, restricted edit rights, plain links and descriptions, nothing instruction-shaped). Its only evidenced readership is AI coding agents: Claude-Code out-fetched every AI retrieval bot. So if your customers use coding agents to source recommendations, it has a use. If your goal is visibility in ChatGPT or AI Overviews, the data says it is decoration. The test is simple: ask the vendor to prove with server logs that an AI retrieval bot reads it.

My logs show GPTBot fetching llms.txt. Isn't that evidence?

No. Fetching is a crawler's job description, not an endorsement. cats.txt — a fake file about cats — was fetched by GPTBot, ClaudeBot, PerplexityBot and Googlebot too. A fetch proves the URL exists and responds. It proves nothing about the content being understood, trusted, or used in an answer.

Why doesn't "I asked ChatGPT if this works" count as validation?

Because the model returns a running average of online discourse, not a judgment. Two weeks after cats.txt launched, ChatGPT said it could help you rank. Once enough people wrote that it was a prank, the same question returned "it's a joke" — with the file unchanged. An optimisation you make today on the model's advice may be judged wrong by the same model in six months.

If llms.txt isn't the answer, what is Google actually backing?

WebMCP. Co-edited by Google and Microsoft engineers and incubated in a W3C community group, it uses the browser's navigator.modelContext API to let a site declare its features as tools an AI agent can call, instead of the agent screenshotting your page and guessing where the buttons are. It is in origin trial in Chrome with very little real deployment so far, but the key difference is that whether a tool was called is something you can check.

I'm a small company with no engineers. What's the cheapest way to verify anything?

Start with the Search Console generative AI performance report — free and first-party. Then ask your host or site maintainer for the last 30 days of raw access logs and search them for GPTBot, ClaudeBot and PerplexityBot to see which pages they fetched. Those two steps cost nothing and replace the core conclusion of most GEO reports.

My Take

The mainstream reaction is "llms.txt is dead, move to the next thing." My read is that the thing that should die is not the file — it is the habit of using an AI's answer as validation — and it gets more dangerous over the next 12 to 18 months, not less. Convergence is accelerating, because as more SEO content is itself AI-written, a growing share of the "industry consensus" the model reads is the model's own older output. You ask for an opinion and get your own post from six months ago, laundered and returned with total composure. The value of cats.txt is not that it mocked a file. It is that it ran that loop end to end in two weeks, in public.

The second call is less popular: the GEO service category will next grow a counter-business — evidence auditing, meaning checking whether what the previous agency sold has any server log behind it. When a category needs someone to audit the books, its expansion phase is over.

For a studio like ScriptWalker, the opportunity is not reselling another AI visibility subscription. It is two deliverables. First, a GEO evidence audit: walk every AI optimisation line item against server logs and produce a "supported / unsupported / unverifiable" list. Second, a crawler log reporting pipeline the client can actually read, with Search Console's first-party impressions in the same table. SaaS cannot do either, because both require touching the client's hosting. Price it as a one-off audit plus a monthly retainer. What it delivers is "stop paying for this" — and in Taiwan's marketing budgets, almost nobody is willing to sell that yet.

Sources

• Mark Williams-Cook: How cats.txt showed llms.txt evidence is GEO astrology, 4 August 2026 (first-party)
• Mark Williams-Cook: cats.txt Specification Draft v1.0 (first-party)
• Ahrefs: We Analyzed 137K Sites: 97% of llms.txt Files Never Get Read (first-party research)
• Google Search Central: AI optimization guide (first-party)
• Chrome for Developers: Lighthouse Agentic Browsing audits: llms.txt (first-party)
• John Mueller: Bluesky post (first-party)
• catstxt.org: live log viewer (first-party)
• Similarweb: 2026 Generative AI Landscape (first-party)
• Previsible: AI Traffic Report, July 2026 (first-party)
• Search Engine Journal: How Cats.txt Showed LLMs.txt Evidence Is GEO Astrology, 7 August 2026
• Search Engine Journal: 97% Of llms.txt Files Got No Requests, Ahrefs Data Shows
• DesignRush: Cats.txt Breaks the GEO Evidence Bar, 8 August 2026
• Anicca: Weekly search marketing update, 11 August 2026
• Cittago: Does llms.txt work? 4 requests in 23 hours, zero AI, 9 August 2026
• Search Engine Roundtable: Google Search Team Does Not Endorse LLMs.txt Files
• InfoQ: WebMCP Standard Proposal for Agentic Web Actuation Now Available in Chrome

Want Someone to Audit the Books on Your GEO Proposal?

If you have an AI optimisation proposal on your desk and cannot say which line items are backed by server logs, we can run a GEO evidence audit and set up a monthly AI crawler log report. Get in touch:

• Email: [email protected]
• Phone: 0916-224-047
• LINE: @ufv9089p