My experiement with llms.txt, which i used on one of my most successful projects, Telegaon (For a clear view, open the image in a new tab)
Website owners are always searching for ways to increase their online presence as the popularity of AI platforms like ChatGPT, Gemini, Claude and Perplexity increases.
The llms.txt file has recently become a trending topic, because it is a proposed standard for large language models (LLMs) to easily recognize important information on a specific website.
However, does having an llms.txt file increase your chances of being included in AI results?
The short answer is No, that it’s not completely clear yet.
According to a study by the SEO firm Ahrefs, 28% of sites publish a valid llms.txt file, but 97% of sites have received zero requests from LLMs or search engines. No bots, no humans, nothing.
I used the llm.txt file in one of my projects (Telegaon), but it didn’t help as much as I expected. Citations from LLMs like ChatGPT are still the same.
While robots.txt has become an industry standard for search engines, llms.txt has not been established as a universal web standard by major AI companies.
Therefore, as of this moment, there is no evidence to suggest that including an llms.txt file will lead ChatGPT or any other LLMs to give your website more citations or have it rank higher in the search results of an AI tool.
The quality of your web content and how well it has been structured will ultimately be the most important factors in determining if your site will be referenced by an AI, or whether an AI will use your site as a resource.
In order to help ensure that your site is referenced by the AI systems, publishing original research, clearly responding to a users’ requests, implementing good technical SEO practices, and showing true expertise will all increase the likelihood that your site will be referenced in the AI’s results. Additionally, building topical authority, utilizing descriptive headings, improving internal linking, and keeping your content fresh and updated are strategies that help your site rank well with search engines as well as AI-based results.
llms.txt does not signify an obsolete file type. If AI crawlers use it as a means to locate preferred content in the future it will suddenly be of great benefit. However, using this file format does not lessen the need for fundamentally good SEO techniques.
The most important aspect of creating visibility through AI searches is producing useful, high-quality content and not seeking shortcuts. AI models are created to find data that are useful, dependable, and properly structured for value.
Long term, creating quality content and having valid technologies will produce greater long-term results than using an untested and experimental file type that is not yet popular.