llms.txt in Practice: Why the AI Visibility File Doesn't Deliver (Yet)

Website owners have been hearing the same advice for months: add an llms.txt file so AI systems can better find your content. The idea sounds plausible — a kind of robots.txt for language models. But before your team invests another hour, it's worth looking at what the data actually show.

What llms.txt Promises

The concept comes from developer Jeremy Howard: a structured Markdown file in the root directory of a website that guides AI crawlers to the most important content — stripped of HTML clutter. Around 10 percent of larger websites have now implemented it, including companies like Anthropic, Cursor, and GitBook. Anyone building or planning AI solutions for their business quickly encounters this advice. The promises are significant: better discoverability by ChatGPT, Gemini & Co., more citations in AI answers, a head start over competitors without the file.

What Three Independent Studies Actually Found

The findings are clear — and sobering for anyone who has already set up the file:

• Ahrefs (May 2026, 137,000 websites): 97% of all llms.txt files received zero requests from AI bots — crawlers simply don't find or use the file.
• SE Ranking (300,000 domains): No statistically significant correlation between having an llms.txt and citation frequency in AI answers. Removing the variable from the prediction model actually improved its accuracy.
• Trakkr (37,894 domains): No measurable citation advantage over websites without the file.

Google's liaison to the webmaster community, Gary Illyes, confirmed that Google does not support llms.txt and has no plans to. His colleague John Mueller compared it to the keywords meta tag from the early 2000s: high effort, low impact. Source: 1clickreport.com, September 2026

Measurement comes before optimization — what you can't see in AI answers, you can't improve.

What Actually Works — Based on Evidence

Businesses that want visibility in AI answers should focus on factors with documented correlations. A sound technology strategy and IT consulting approach starts exactly here — with what demonstrably works:

• Brand mentions: External mentions of your brand correlate three times more strongly with AI citations than traditional backlinks — PR work pays off for LLM visibility too.
• Citable content: Question-and-answer structures, self-contained paragraphs (understandable without context), and clear headings make content quotable for language models.
• Content freshness: Regularly updated, fact-based content is cited more often by LLMs than outdated pages.
• Technical foundations: Fast load times, clean server rendering, and structured data (Schema.org) improve indexability — not because of llms.txt, but because these fundamentals characterise well-functioning websites.

The Real Challenge: Measure Instead of Guess

The llms.txt debate reveals a structural problem in GEO (Generative Engine Optimization): recommendations circulate before the data exists. This encourages actionism — creating files, adjusting metadata, restructuring content — without knowing whether any of it actually changes your visibility. For an overview of why LLM visibility already matters for your business today, see our post How Visible Is Your Business in AI Answers?.

The smarter starting point: understand what is happening today. Does your company appear when potential customers ask ChatGPT, Gemini, or Grok about your services? Which competitors are mentioned instead? Tools like Vjus.ai — developed by NoviCogi — check exactly these questions daily and provide the baseline without which every tactical decision is guesswork. Only once measurement is in place can you assess whether llms.txt, content restructuring, or PR work actually move the needle on your AI visibility.

AI integration and digital visibility for mid-sized businesses — NoviCogi

This article was created with AI assistance and editorially reviewed.