llms.txt: what it is and whether your site needs it

What is llms.txt, and what is it meant to do?

llms.txt is a proposed plain-text file for a website’s root directory that points language models toward useful site information. It is intended to provide a concise map of key resources, not to replace the pages themselves.

The proposal at llmstxt.org describes a lightweight format that can include a short site introduction, links, and optional descriptions. A site owner might use it to surface an API reference, documentation, product details, or a set of carefully chosen guides. A model or tool would still need to fetch and interpret the linked material for the file to be useful.

Think of the file as an editorial index:

• It tells a reader where important information lives.
• It can prioritize canonical, readable resources over secondary pages.
• It does not make an inaccessible page accessible or correct a misleading page.

The proposal is not the same as a universal web standard enforced by crawlers. Before adopting it, decide what problem you expect it to solve: improving navigation for model-assisted research, making documentation easier to locate, or testing an emerging convention. That purpose should determine both the file’s contents and how you evaluate it.

Is llms.txt needed for your website?

Most sites do not need llms.txt as a prerequisite for search visibility. Consider it when your site has valuable, stable material that is difficult to locate through ordinary navigation, especially technical documentation, product references, or a large knowledge base.

It is a lower priority when the site is small, its key information is already easy to find, or the team cannot keep another index accurate. In those cases, invest first in clear page structure, crawlable links, useful content, and accessible documentation. Those improvements help human visitors as well as automated systems.

Use this decision check before creating the file:

• A clear use case: Name the resources you want a model-assisted reader to find.
• Stable destinations: Prefer canonical pages that will remain useful over time.
• An owner: Assign someone to review links and descriptions when the site changes.
• A way to assess value: Decide what you will observe, such as referral visits, user feedback, or whether supported tools retrieve the intended pages.

If you cannot identify a specific audience or maintenance owner, skip the file for now. For a broader view of technical visibility work, see technical AEO and the AI search visibility hub.

What evidence supports llms.txt in SEO?

There is a plausible discovery use case for llms.txt, but that is different from evidence that the file improves rankings or citations. The proposal explains a suggested format; its existence does not show that major search engines or AI services use the file as a ranking signal.

For llms.txt SEO claims, separate three questions. First, can a crawler request the file? Second, does a particular service say it reads or uses it? Third, can you observe a meaningful change that is attributable to the file? A successful request only answers the first question. It does not establish that a system uses the contents or changes how it presents your pages.

Google has not documented llms.txt as a search-ranking requirement. Do not treat publishing it as a substitute for content quality, crawlability, or clear page information. Likewise, a model producing an answer that resembles your content is not, by itself, proof that it read the file.

A grounded test keeps expectations modest:

• Record the file’s publication and revision in your own change log.
• Check available server logs for requests to the file, without assuming a request means adoption.
• Track relevant referral sources and user feedback over time.
• Compare observations with other site changes before drawing a conclusion.

This is an emerging convention worth testing when the cost of upkeep is low, not an established SEO lever. For related work, compare the scope of AI search monitoring with a broader AI search audit.

LLMs.txt vs schema.org: what is the difference?

llms.txt and schema.org markup serve different purposes. llms.txt is a human-readable index of selected resources; schema.org provides structured descriptions attached to web content in formats that software can parse.

A useful distinction is navigation versus description. An llms.txt file can point a reader toward an organization page or documentation section. Structured data can describe properties of a specific page or entity using defined types and fields. Neither format makes inaccurate source content reliable, and neither should be used to disguise information that is absent from the visible page.

Choose based on the job:

The formats can coexist, but they should agree. If the file links to a product page, that page should remain the authoritative source for product claims. Structured data should describe the page accurately rather than introduce extra claims. For a deeper look at how structured markup fits AI search, read the schema markup guide.

Do not add schema just because llms.txt exists, or create llms.txt to compensate for missing structured data. Start with the information need, then use each format only where it makes that information easier to find or interpret.

How to write an llms.txt file that stays useful

A useful llms.txt file is short, selective, and accurate. Write it as a guide to your best sources, not as a second website or a dump of every URL you own.

Start by choosing the audience and the material they need. A developer may need API documentation and integration guides; a prospective customer may need product descriptions, policies, and support resources. Select canonical pages for that audience, and use descriptions that state what each page actually contains.

A practical drafting checklist:

• Add a brief description of the site or resource collection.
• Group related links under clear headings where that helps readers scan.
• Prefer stable, public URLs with substantive page content.
• Use concise link labels and plain descriptions, without promotional claims.
• Check that linked pages are accessible and do not require an account to read.
• Exclude duplicate, outdated, private, or thin pages.

Do not assume that a particular model will parse every heading or follow every link. Keep the file understandable to a person, and make sure the linked pages stand on their own. Review the proposed llms.txt format as a starting point, then check the current expectations of any tools you intend to test.

If you use a CMS plugin, including one associated with Yoast, confirm what it actually generates and where it publishes the file. A plugin label is not evidence that the output is correct. Inspect the resulting text, links, and live URL before calling the work complete.

How to implement llms.txt on your site

Implement llms.txt by publishing a plain-text file at the location your team intends to use, then checking its contents and accessibility from outside your editing environment. The work is small, but validation matters: a draft in a repository is not the same as a live file.

Before publishing, confirm who can approve the selected links and descriptions. Avoid including confidential material, internal URLs, or claims that have not been approved for public use. Keep the file consistent with canonical page versions, and make sure redirects do not lead to unrelated content.

After publication, validate the actual response in a browser or command-line client. Check that the URL resolves, the response is readable text, and each listed destination opens as expected. If the site uses a CDN, cache, or deployment pipeline, confirm that the live version is the one you reviewed.

Use this implementation sequence:

• Draft the index from approved, public source pages.
• Review links, wording, and ownership with the relevant content or product lead.
• Publish the file at the chosen root-level path.
• Test the live file and linked pages from outside the CMS.
• Add a review trigger to the content release or documentation process.

For the surrounding strategy, see the guide to getting cited in ChatGPT. A file can support clearer discovery, but the cited page still needs to answer the reader’s question directly and accurately.

What llms.txt cannot control, and how to maintain it

llms.txt can communicate your preferred reading path, but each platform controls its own crawling, parsing, indexing, and answer generation. A platform may ignore the file, interpret it differently, or change its behavior; publishing it cannot guarantee that a specific page will be fetched, cited, or shown in an answer. That platform-specific uncertainty is the main limit to account for before investing in extensive maintenance.

Keep the file small enough to review whenever important pages change. Use a simple ownership rule: the team responsible for a listed resource should tell the file owner when its URL, purpose, or public status changes. Remove links that have become stale rather than letting the index accumulate.

A lightweight maintenance routine should check:

• Whether each listed page still exists and remains publicly readable.
• Whether the description still matches the page’s actual contents.
• Whether a new canonical resource has replaced an older one.
• Whether the file’s live version matches the reviewed version.

Judge the file by whether it is accurate and useful as an index, then look for observable signs of use without treating them as proof of influence. If your team is reviewing broader AI visibility, compare the file with your pages and entity information in an AI visibility audit. Keep the decision reversible: if maintaining the file creates work without a clear use, revise or remove it and focus on improvements to the underlying pages.

Prices

Starting prices in USD. Custom bundles and volume discounts on request. Payment in USDT, USDC, BTC, ETH, SOL, TON or your project token.

How it works

1. Set the purposeName the audience and the information you want them to find. If there is no concrete use case, postpone the file.
2. Select authoritative pagesChoose a small set of stable, public resources. Exclude duplicate, outdated, restricted, or low-value destinations.
3. Draft and reviewWrite a concise introduction and accurate link descriptions. Ask the owners of the linked content to check them.
4. Publish and validatePublish the plain-text file at the chosen location, then test the live response and every listed destination.
5. Monitor and maintainAssign an owner and review the file when key pages change. Track observable use without assuming it proves ranking impact.