llms.txt is a proposed file for guiding AI crawlers. Here's what Google has actually confirmed about it, and when it's worth building one.
llms.txt is a plain-text file placed at a website's root that's meant to give AI models a structured, curated summary of a site's key pages, similar in concept to an XML sitemap but written for AI systems rather than search crawlers. The honest, current answer on whether a site needs one is: probably not for the reason most guides claim. Google's own Search Central documentation states plainly that site owners do not need machine-readable files, AI-specific markup, or an llms.txt file to appear in Google Search, and that maintaining one will not harm visibility but will not help it either.
This is worth being precise about, since the marketing around llms.txt has often overstated it. Gary Illyes confirmed at Google Search Central Live that Google does not support llms.txt and has no plans to, and John Mueller has compared it directly to the old keywords meta tag, a self-declared signal that search engines learned to ignore because site owners control it and it's therefore easy to manipulate. When Google's own developer documentation briefly carried an llms.txt file in December 2025, Mueller clarified on the record that it was for agent functionality, not search, an internal platform rollout, not an endorsement.
One independent test is worth citing directly here: out of 62,100 AI bot requests to one domain, exactly 84 went to its llms.txt file, about 0.1%, performing worse than an average content page on the same site. The only consistent visitor to the file in that test was a technology-detection bot cataloguing which files exist, not an AI system using it to inform an answer.
This appears to hedge case by case, not a settled yes: Lighthouse, Chrome's auditing tool, now includes an llms.txt check in an experimental "agentic browsing" category, separate from search ranking entirely, aimed at browser-based AI agents completing tasks on a user's behalf, not at search or chat-based citation. AEO tracking tools like Peec.ai and AthenaHQ reportedly parse llms.txt when profiling a brand's AI visibility, so it may support monitoring workflows even without a confirmed citation benefit. Some practitioners argue for early adoption anyway on the logic that web standards get adopted because publishers start serving them first, the way XML sitemaps did in 2005, not because usage data proves value on day one.
Given the current, honest state of the evidence, the higher-leverage work is the same regardless of llms.txt: crawlable, well-structured HTML, genuine first-party data and expertise signals, and answer-first content, covered in more depth in how to structure content so AI search engines cite it.
This is part of AI Search & SEO. For the content side of this work, see how to structure content so AI search engines cite it. See it applied in a real project in AI visibility: from invisible to citable.
Want help figuring out what actually deserves your time in AI search optimization? Get in touch.