AI Search & GEO

What Is llms.txt and Does It Matter?

By BacklinkPrice Team · Nov 18, 2026 · Updated Nov 20, 2026 · 12 min read
Illustrated cover for What Is llms.txt and Does It Matter?

llms.txt is a small Markdown file that gives AI tools a summary of a site and a list of its key pages. Google Search ignores it, so it helps documentation sites far more than search visibility.

llms.txt is a small Markdown file at a website root that gives AI tools a short summary of a site and a list of its most useful pages. It does not control search rankings, and Google Search ignores it, so it matters mainly for software documentation and coding agents, not as a shortcut to AI citations as of November 2026.

Key takeaways

  • llms.txt is a proposal from 2024, not a web standard, and its second version followed two years of use, according to llmstxt.org in 2026.
  • Google Search Central said in 2026 that Google Search does not use llms.txt files, and keeping one will neither help nor hurt visibility in Google Search.
  • The studies show that almost no one reads the file. Ahrefs studied 137,210 domains in June 2026 and found 97% of published llms.txt files received zero requests in May 2026.
  • SE Ranking analysed nearly 300,000 domains and found no measurable link between having an llms.txt file and AI citation frequency.
  • Voltik reported in 2026 that adding one takes about 10 minutes, so it is a cheap test for documentation sites and an optional extra for most other sites.

What llms.txt is

llms.txt is a Markdown file placed at a website root, usually at /llms.txt, that offers a short summary of a site plus links to the pages an AI agent should read first. Jeremy Howard published the proposal in 2024, and the second version on llmstxt.org in 2026 reflects two years of use by thousands of sites.

The idea came from a simple problem. Web pages are built for people, so they wrap facts in menus, ads, and scripts. An AI agent that lands on a site must strip all of that away before it can read the substance, and a large site will not fit in a model context window at all.

A short map solves that problem for the agent, not for the search engine. The file stays small enough to fit in context, and the detail lives behind its links, which the agent fetches only when it needs them. That design is why the format suits LLM SEO work on documentation sites more than it suits a normal marketing site.

The proposal is not a standard from a web body, and no major search engine has adopted it. It is a convention that any site owner can follow, and any agent can choose to read or ignore.

For Google Search, the answer is no, and Google has put that answer in writing. Its guide to generative AI features says site owners do not need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search, because Google Search itself does not use them, according to Google Search Central in 2026.

That guide also sets out what does decide whether a page can appear in AI Overviews or AI Mode. A page must be indexed and eligible to appear in Google Search with a snippet, and it must meet the Search technical requirements. A file cannot substitute for that eligibility.

The same point applies to citations in ChatGPT and other answer tools. No study so far has shown that adding the file raises citation frequency, and the largest studies point the other way. That is the core finding for anyone weighing the file as a generative engine optimization tactic in 2026.

AI Overviews draw on pages from the Search index, not on a separate AI file. That is why the same eligibility rules apply, and why our guide to how Google AI Overviews choose sources starts with indexing and snippet eligibility, not with a new file.

What do the studies show about AI citations?

The studies show that almost no one reads the file. Ahrefs studied 137,210 domains in June 2026 and found that 28% published a valid llms.txt file, yet 97% of those files received zero requests in May 2026. Only about 1,100 of the domains with a file saw any traffic to it at all, according to Ahrefs in 2026.

Of the requests that did arrive, about 96% came from bots, and named AI tools made up 19.5% of them. AI agents accounted for 10.5% of requests, training crawlers for 5.3%, assistants for 2.5%, and retrieval bots for only 1.1%, according to Ahrefs in 2026.

SE Ranking analysed nearly 300,000 domains and found the file on 10.13% of them, with no measurable link between having it and how often a domain was cited in AI answers. Removing the file as a factor made its prediction model more accurate, according to SE Ranking.

When an llms.txt file is worth your time

An llms.txt file is worth your time when an agent already reads your documentation to do a job. Coding assistants fetch these files while helping someone use a library or an API, and that is the one audience with measured use in the studies published in 2026.

It is also worth a short test when your site is a reference work, such as a help centre, a course library, or a product manual. In those cases the file acts as a table of contents for an agent that a visitor has sent to find an answer, not as a way to win new visitors.

For a normal business site, the file is optional. It costs little, it does no harm in Google Search, and it may help a future agent, but no evidence in 2026 shows that it changes whether ChatGPT cites you. Your effort goes further on the pages themselves and on the sources that decide how ChatGPT chooses which sources to cite.

The decision turns on who reads your site. If developers and their coding tools are your readers, publish the file and keep it current. If buyers and search visitors are your readers, treat the file as a 10 minute task at the end of a longer list.

How to create an llms.txt file

Creating the file takes a text editor, a list of your best pages, and a few minutes. The steps below follow the format set out in the proposal on llmstxt.org in 2026.

Step 1: Pick the pages that matter

List the 5 to 15 pages an agent most needs, such as your product overview, pricing, getting started guide, API reference, and main help articles. Leave out thin pages, duplicates, and pages that exist only for navigation.

Step 2: Write the file in Markdown

Start with an H1 line holding your site or project name, since that is the only required part of the file. Add a blockquote of one or two sentences that sums up what the site does and who it serves.

Then add H2 sections, each with a short list of links in Markdown form, with a few words after each link that say what the page covers. Put secondary links under an H2 headed Optional, so an agent short on context can skip them.

Link to the cleanest version of each page you have. The proposal suggests offering a Markdown version of a page at the same address with .md added, and pointing to it with a link relation, according to llmstxt.org in 2026.

Check that every link returns a live page. A dead link in a file meant to guide an agent wastes the agent time the file was meant to save, and it is the first thing to check after you publish.

Step 4: Publish it at the root

Save the file as llms.txt and place it at your site root, so it lives at yourdomain.com/llms.txt. You can also place one under a path such as /docs/llms.txt, where it covers only the pages under that path, according to llmstxt.org in 2026.

The proposal also allows a file at any path, with the most specific file applying where more than one covers a page. For most sites, one file at the root is enough.

Step 5: Test it with an agent

Ask a coding agent or chat assistant a question about your product, giving it only your llms.txt file as a starting point. The proposal itself suggests this test, and it shows at once whether the file points at the right pages, according to llmstxt.org in 2026.

If the agent follows a wrong link or misses a key page, fix the file. Keep the test short, and repeat it after any large change to your site, since a stale file misleads the few tools that read it.

What to put in the file

A good llms.txt file is short, factual, and current. It names the site, sums it up in a sentence or two, and then lists the pages that answer the questions an agent is most likely to be asked.

Keep each link note to a few words. A note should say what the page covers, not sell it, and it should match what the page itself says. If the page changes, change the note in the same edit, or the file will drift from the site it describes.

The table below shows how llms.txt differs from the two files it is most often confused with. The three files serve different readers, and none of them replaces the others.

FileWho reads itWhat it does
llms.txtAI agents, on demandGives a short summary and a list of key pages
robots.txtSearch and AI crawlersStates which parts of a site crawlers may fetch
sitemap.xmlSearch enginesLists the pages a site wants indexed

The proposal states that llms.txt complements robots.txt rather than replacing it. Robots.txt controls access, while llms.txt offers context for the content that access allows, according to llmstxt.org in 2026.

A sitemap is a full list, and it often covers far more pages than an agent can read in one pass. An llms.txt file is a short list, chosen by the site owner, and it can also link to outside pages that help explain a topic, which a sitemap does not do.

Tools, time and cost

You need no paid tool to create an llms.txt file. A plain text editor is enough, and the file itself costs nothing to host, since it is a few kilobytes of text on your own site.

Several platforms generate the file for you. Mintlify and GitBook create one for documentation sites they host, Yoast SEO and AIOSEO offer a generator for WordPress sites, and Wix generates one for every Wix site, according to llmstxt.org in 2026.

Plugins also exist for VitePress and Docusaurus, and a Drupal recipe adds support for sites on Drupal 10.3 and later, according to llmstxt.org in 2026. If your platform already makes the file, check what it includes before you trust it, since a generated list may include pages you would not have chosen.

Time is the only real cost. Voltik reported in 2026 that writing a file by hand takes about 10 minutes, and a generated file takes less. The larger cost is upkeep: review the file when you add or remove key pages, or it will point agents at pages that no longer exist.

How to measure results

Start with your server logs, since they show who actually fetched the file. Search your access log for requests to /llms.txt, group them by user agent, and count them over a month. If the count is zero, your file is unread, and that matches what Ahrefs found for 97% of files in May 2026.

The log check also shows who the readers are. In the Ahrefs study from June 2026, SEO audit tools made up 21.7% of requests to llms.txt files, the largest single group, ahead of every AI tool. A file that is fetched only by audit tools is not reaching AI agents.

Evil Martians logged its own site from April 2026 and counted about 593 direct fetches of llms.txt and llms-full.txt, of which AI crawlers made 75. Named AI assistants fetched the file only 47 times, and Claude Code made 43 of those, according to Voltik in 2026.

For Google, use Search Console instead of the file. Google points site owners to the Generative AI performance report in Search Console to see how content performs in its generative AI features, according to Google Search Central in 2026.

Do not judge the file by rankings or citations alone. Those move for many reasons, and no study in 2026 has tied a change in them to adding the file. Judge it by fetches in your logs, and by whether an agent test reaches the right page.

Access still matters more than the file. The crawlers that feed AI answers must be able to fetch your pages at all, so check that your robots.txt allows them before you spend time on llms.txt, as explained in our guide on whether AI crawlers follow links.

Common mistakes

  • Treating the file as a ranking factor: Google Search ignores it, so a file cannot lift a page that is not indexed, eligible, and useful on its own merits.
  • Copying your sitemap into the file: a list of every page defeats the point, since the file must stay small enough for an agent to read in one pass.
  • Linking to pages that no longer exist: check every link after you publish, and again after a site redesign, since dead links waste agent time.
  • Hiding key facts in the file only: put prices, names, and facts on the human pages too, so visitors, crawlers, and agents all read the same source of truth.
  • Blocking the crawlers that matter: llms.txt cannot allow a crawler in, so a robots.txt block on a search bot keeps your pages out of AI answers whatever the file says. See our guide on whether you should block GPTBot.
  • Paying for llms.txt as a service: the file takes minutes to write, and no vendor can promise citations from it, so a large fee for one is a warning sign.

Frequently Asked Questions

Does llms.txt help with SEO?

No, there is no evidence that llms.txt helps with SEO. Google Search Central said in 2026 that Google Search does not use the file, and the largest log and citation studies from 2026 found no link between having one and being cited more often.

Do ChatGPT, Claude, or Perplexity read llms.txt?

No major AI company has said its search product reads llms.txt to choose sources. Their crawler documentation describes robots.txt controls, and the log studies from 2026 show retrieval bots making only a tiny share of requests for the file. That position held as of November 2026.

Should I add an llms.txt file to my website?

Add one if you publish developer documentation, an API, or a large help centre, since coding agents are the readers with measured use. For a small business site, it is optional, takes about 10 minutes, and should not be expected to change citations.

Is llms.txt the same as robots.txt?

No, the two files do different jobs. Robots.txt tells crawlers which parts of a site they may fetch, and the major AI crawlers respect it. Llms.txt is a suggested reading list with no power to allow or block anything. The file cannot change either rule.

How do I check whether anyone reads my llms.txt file?

Check your server access log for requests to the /llms.txt path over a month, grouped by user agent. If the count is zero, nothing fetched it. If audit tools are the only visitors, the file is not reaching AI agents.

Sources

For a wider view of how AI tools judge a site, read our guide to AI visibility.

BacklinkPrice Team
Written by
BacklinkPrice Team
Link building and backlink pricing research

The BacklinkPrice Team researches backlink prices, publisher sites and link building tools. We check prices and plans against each provider's own pages and date them, so you know how current the numbers are.

More from BacklinkPrice Team

Related guides