Do you need an llms.txt file?

Mrs Chan was told her bakery needs an llms.txt

On Tuesday, a man told Mrs Chan that ChatGPT cannot see her bakery because it has no llms.txt. He offered to write one, for a fee.

She bakes cakes. She has no idea what an llms.txt is. A few years ago the advice was page titles, a Maps listing and reviews, for people who typed "bakery near me". Now people ask ChatGPT or Google's AI Mode for an eggless birthday cake in Tiong Bahru, ready to collect on Sunday, and she wants to be the answer. So she asks what every owner asks. Am I missing something? Are the other bakeries getting the Sunday orders because they have the file?

No. The AI that answers the cake question reads her robots.txt and the text on her pages. The file can wait. Here is the order I would give her.

Spend your first hour on three other things

Do these three first, in this order.

1. Open your robots.txt, the small file that tells automated visitors which pages they may read. The crawlers that fetch pages to answer a question must be allowed by name. Each vendor names its training and retrieval crawlers separately ³⁴⁵⁶, so a blanket "block the AI bots" rule can shut out the ones that cite you. A blocked crawler never gets as far as your words.
2. View the page source, not the page. Your headings, text and links must be in the HTML the server sends, before any JavaScript runs. If the source is an empty shell, no file at the root will fill it.
3. Say who you are once, in a form machines can read. Add one block of business details with your legal name, your address and links to your real profiles.

An earlier article walks through all three with what we found on 21 local sites. Only when those are done does it make sense to think about a plain-text index.

Then write the file, if you want one

llms.txt is a file at the root of your site, written in Markdown: a heading with the site's name, a short summary, then lists of links to the pages worth reading, one line on each. Jeremy Howard of Answer.AI proposed it on 3 September 2024 ². The problem it solves is real: a language model with a small context window cannot read a whole documentation site, so a curated index tells it where to look.

Here is an example of the shape, for an invented firm. It is not a client's file.

# Example Plumbing Co

> Licensed plumbers for homes and small offices. Call-outs, repairs and installations.

## Services
- [Repairs](https://example.com/repairs/): leaks, blocked drains, water heaters
- [Installations](https://example.com/installations/): new fittings and bathrooms

## Contact
- [Contact and service area](https://example.com/contact/): phone, WhatsApp, districts covered

A useless version lists every URL on the site, or repeats the homepage slogan. Your sitemap already lists everything, so a good index is the opposite: the few pages a reader needs, one honest line each, and every link working. Allow ten minutes.

Who reads it, and who says so

All four vendors publish an llms.txt on their own documentation sites. Publishing one is not reading one, and most sales pitches for the file run the two together.

The confirmed readers are coding assistants pointed at a documentation site: a developer's tool reading your API reference. If your buyers are developers, the file earns its place early. If your buyers are patients, diners or tenants, nothing published says any AI assistant reads it on their behalf.

Google's verdict, in its own words

Google's guide on optimising for its generative AI features has a section on common AEO and GEO misconceptions. On llms.txt files and special markup: "Google Search itself doesn't use them." On rewriting for machines, it says no special style is needed for generative AI. On structured data: "Structured data isn't required for generative AI search" ¹.

Not required is not the same as not useful. The same guide sends you to Google's structured-data documentation for rich results, which is why the Organization block in step three stays on the list.

Median AI score 78 with the business declared, 59 without

Of the 21 small-business sites in Hong Kong and Singapore we scanned on 9 September 2026, 2 published an llms.txt. Of those 2, one scored a C on the AI axis and the other an F. What separated the higher AI scores from the lower ones in this batch was whether the business is declared in markup: the median AI score was 78 where the primary entity was declared and 59 where it was not.

One of our own seed sites, a Hong Kong agency, publishes the file with a link that returns an error, which our L3 check caught.

Our rubric does give the file a small weight on the AI axis, under module M21: present, correctly headed, links that resolve. We kept that because a file that exists should be correct, and the rubric says what it does and does not prove.

What not to spend money on, and what to do instead

Do not pay anyone for an llms.txt as a visibility fix, and do not buy a plugin whose pitch is that it makes AI assistants read your site. Nothing published says it does.

What I tell an owner is simple: the file is a tidy habit, not a strategy. We generate one for every AI-Ready client because it costs minutes, cannot hurt, and is ready the day an engine adopts it. We do not say it makes ChatGPT, Claude or Perplexity read your site.

The old checklist, page titles and a Maps listing, still matters. It answers less of the question each year, because the question is now a sentence. Making the site itself readable to AI assistants is the move, and it is the work we do. The free scan shows Mrs Chan, or you, which of the three steps above is missing.

Questions people ask

Will Google penalise me for having an llms.txt?

No. Google says Search does not use the file ¹; it does not say the file is a problem. A text file at your root that nobody fetches is harmless. Keep the links in it working, since a broken link is still a broken link on your site.

Should llms.txt list every page?

No. The proposal is a curated index: the pages a reader needs, one line each, with an optional section for pages that can be skipped ². Your sitemap already lists everything. If you want the full text in one place, that is what llms-full.txt is for.

Is llms-full.txt different?

It is the companion: the whole site as text in one Markdown file, so a model can load it in a single read ². Useful for a documentation site, large and pointless for a restaurant. Ours holds our tiers, every FAQ answer and the changelog, generated from the same data as the pages.

Does Claude honour llms.txt?

Anthropic's crawler documentation describes robots.txt and does not mention llms.txt ⁴. Claims that Claude follows the file circulate in SEO articles and are not in Anthropic's own material. Treat them as unsourced until Anthropic says otherwise.

What should I do first instead?

Run the free scan and read the three findings. In our batch the common gaps were headings served by JavaScript, a business never declared in markup, and content without an author. Each of those changes what an AI assistant can read today. llms.txt changes nothing until someone reads it.

Sources

1. Google Search Central, Optimizing your website for generative AI features on Google Search, 10 July 2026. Accessed 13 September 2026.
2. Jeremy Howard, Answer.AI, The /llms.txt file, 3 September 2024. Accessed 13 September 2026.
3. OpenAI, Overview of OpenAI crawlers, undated. Accessed 13 September 2026.
4. Anthropic, Does Anthropic crawl data from the web, and how can site owners block the crawler?, 7 April 2026. Accessed 13 September 2026.
5. Google Search Central, Overview of Google crawlers and fetchers, 14 July 2026. Accessed 13 September 2026.
6. Perplexity, Perplexity crawlers, undated. Accessed 13 September 2026.

Our own counts come from our batch of 9 September 2026, 21 sites, one visit each, scored with the published rubric. Not a market survey.