llms.txt vs sitemap.xml: What a Charity Website Actually Needs
Key takeaways
- sitemap.xml and llms.txt are not alternatives. One lists the pages you want search engines to find; the other offers a short curated summary for a language model to read while it answers a question.
- Google's documentation on its AI features states plainly that you do not need to create new machine-readable files or AI text files to appear in AI Overviews or AI Mode.
- Ahrefs studied 137,000 domains and found that of the roughly 38,000 publishing a valid llms.txt, 97% received no requests for the file at all during May 2026.
- The sitemap is still the file that decides whether you can be cited, because Google requires a page to be indexed and eligible for a snippet before an AI answer can link to it.
- Most charity websites are small enough and well enough linked internally that Google says they may not need a sitemap at all, which is the opposite of what most suppliers imply.
- llms.txt is a Markdown file with one required line. If anyone quotes you a project fee to produce one, that is the finding, not the file.
Summary
llms.txt and sitemap.xml are not two versions of the same file. A sitemap tells search engines which pages exist and is still the file that decides whether an AI answer can cite you, because Google requires a page to be indexed before it can appear. llms.txt proposes something different: a short, curated summary of your site for a language model to read at the moment it answers a question. The evidence that anything reads it is weak, Google's own documentation says you do not need it, and Ahrefs found 97% of published files received no requests at all. It is still worth writing, for reasons that have nothing to do with rankings, and it should take an afternoon rather than a budget line.
A funder's researcher asks an AI assistant what your charity does. Somebody on your team hears about this, and the answer that comes back is a year out of date, so a supplier proposes adding an llms.txt file to fix it.
That is roughly how this file arrives in most organisations. It usually arrives with a quote attached.
The file is real, the idea behind it is reasonable, and almost everything said about what it will do for you is not supported yet. It is worth twenty minutes to understand what it is, because the difference between llms.txt and the sitemap you already have is the difference between what machines can find and what they can understand.
The two files answer different questions
A sitemap answers "what pages are here?". llms.txt answers "what is this organisation, and what should I read first?".
That is the whole distinction, and everything else follows from it.
| File | What it is for | Who reads it | Does a charity site need one |
|---|---|---|---|
| robots.txt | Stating which automated visitors may fetch what | Crawlers, by long-standing convention | Yes, and check what it currently blocks |
| sitemap.xml | Listing the URLs you want found, and when each changed | Search engine crawlers | Useful, and less essential than most suppliers imply |
| llms.txt | Summarising the site and pointing at the few pages that matter | Mainly developer and agent tooling today | Optional, cheap, and not a visibility strategy |
What sitemap.xml does
Google describes a sitemap as "a file where you provide information about the pages, videos, and other files on your site", and says it "tells search engines which pages and files you think are important". It is a discovery aid. It can also carry the date a page last changed, which is how a crawler knows to come back.
Two things about it are worth knowing before anyone bills you for sitemap work.
The first is that it guarantees nothing. Google's guidance is explicit that a sitemap "doesn't guarantee that all the items in your sitemap will be crawled and indexed". Submitting one is a request, not an instruction.
The second is that your site may not need one. Google's own documentation says sitemaps matter most for large sites, for new sites with few inbound links, and for sites with substantial video or image libraries, and that a site of around 500 pages or fewer which is "comprehensively linked internally" may not need one, because a crawler can reach every page from the homepage. Most charity websites are well inside that threshold.
This matters because the sitemap gets blamed for problems it did not cause. When a charity tells me its annual report cannot be found, the cause is almost never the sitemap. It is a page nothing links to, a document trapped in a PDF, or a heading that does not say what is underneath it.
What llms.txt proposes
llms.txt was proposed by Jeremy Howard in September 2024, and the specification is now at version two. It is a Markdown file at the root of your site. The specification is unusually short, and the only required element is a single H1 naming the project or organisation.
The reasoning is about context rather than crawling. A language model answering a question does not want your entire site. It wants a short orientation it can read in the moment, which is why the specification describes the file as used on demand during inference rather than for training. The specification makes the contrast with sitemaps itself: sitemaps list all indexable pages for search engines, while llms.txt "offers a curated overview for LLMs", and a sitemap usually covers more content than fits in a context window.
It is a sensible idea. The problem is the next section.
Almost nothing reads it
This is the part that does not appear in the pitch.
Google's documentation on its AI features states that "You don't need to create new machine readable files, AI text files, or markup to appear in these features", and that "There are no additional requirements to appear in AI Overviews or AI Mode, nor other special optimizations necessary". That is Google describing its own AI surfaces, and it is about as direct as platform documentation gets.
The usage data points the same way. Ahrefs studied 137,000 domains in its own analytics product and found roughly 38,000 publishing a valid llms.txt file. Of those, 97% received no requests for the file at all during May 2026. Ahrefs notes that its sample skews towards sites that pay attention to AI, so adoption across the web is likely lower still. Ryan Law, writing up the study, answered the question of whether a site should create one with "In my opinion, no, not yet".
Where the file genuinely gets used is narrower and less exciting than AI search: developer documentation and coding agents, where a tool is pointed at a product's docs deliberately.
I publish an llms.txt on this site. It lists every resource I have written, because the file is a useful map of what exists and it costs nothing to maintain. I have no evidence that it has changed anything, and I would not present it to a client as a reason to expect more visibility.
So should your charity publish one?
Yes, on one condition: that it takes an afternoon and nobody charges you for it.
The case for publishing is not rankings. It is that you are writing down, in one place and in plain language, what your organisation is and which five or six pages someone would need in order to describe you accurately. That is a useful exercise whether or not a machine ever fetches the result. Organisations are frequently unable to agree on it, which is itself the finding.
The case against spending money on it is the evidence above. If a proposal includes llms.txt as a line item with a fee attached, or frames it as AI search optimisation, that tells you something about the proposal. The specification has one required line. There is no version of this that justifies a project rate.
How to write one
Create a file called llms.txt, write Markdown in it, and serve it at yourdomain.org/llms.txt. On most platforms that means dropping it in the public or static folder. The shape the specification asks for:
# Example Charity
> We fund and run early-years literacy programmes in the north west of England.
Registered charity in England and Wales, number 1234567. Founded 1998.
## Core
- [What we do](https://example.org/what-we-do): our three programmes and who they serve
- [Governance](https://example.org/governance): trustees, structure and decision-making
- [Annual report 2025/26](https://example.org/annual-report): audited accounts and impact data
- [Safeguarding policy](https://example.org/safeguarding): current version, approved March 2026
- [Contact](https://example.org/contact): press, partnerships and general enquiries
## Optional
- [Our history](https://example.org/history)
- [Past campaigns](https://example.org/campaigns/archive)The H1 is the only part the specification requires. The blockquote is a one-sentence description. The H2 sections hold lists of links, each a Markdown link with an optional colon and a short note explaining what the reader will find. The Optional section has a specific meaning in the specification: it marks secondary links a tool can skip when it needs less context.
Three rules that will keep it honest.
Link to the live page, not a copy. The point of the file is to send a reader to the real document. A summary that drifts out of step with the page it describes is worse than no file.
Only list pages you are prepared to be judged on. This is a public description of your organisation. If the annual report link points at last year's, that is now a published claim.
Put it on a review cadence. A file written once and forgotten becomes a confident, machine-readable account of an organisation that no longer exists in that form. It belongs in the same review that checks your policy pages are current.
What actually determines whether you can be cited
If the goal is for an AI answer to describe your charity correctly, the work sits somewhere other than this file.
Google ties eligibility for its AI features to the ordinary search index: a page has to be indexed and eligible to be shown in Google Search with a snippet before it can be used as a supporting link in an AI answer. So the question is not "do we have an llms.txt". It is whether the page carrying the information is indexed, reachable, and clear.
That means the familiar list. Governance documents published as HTML pages rather than buried in a PDF. Headings that describe what is under them. Stable URLs that do not change when the site is rebuilt. One page per question, answering that question near the top. Named people with named roles. This is the same work as answer engine optimisation and the same work as structured data, and none of it is specific to AI. It is what makes a site legible to a funder's researcher who has eight tabs open and four minutes.
The uncomfortable implication is that there is no file you can add to fix a site that is unclear. AI changed how quickly a wrong answer travels. It did not introduce a new control surface where you can declare what you want to be true.
The governance point
llms.txt is a small file and a reasonable idea whose main value, for now, is internal. Writing one forces an organisation to decide what it is in one paragraph and which pages represent it. Most cannot do that quickly, and the difficulty is the useful part.
What it is not is a substitute for the thing it describes. A curated list of five excellent pages is only as good as those five pages, and a machine-readable summary of an organisation that cannot say clearly what it does will be an accurate summary of that confusion.
If you are not sure what your website currently tells a funder, a regulator or an answer engine about your organisation, the Blueprint Audit assesses exactly that: what is published, what is findable, what is out of date, and what a reader concludes from it.
Two clicks. Three things.
Send the address of your website. Within two working days you get a short email naming the three things a funder would hold against it, including what AI assistants like ChatGPT and Google’s AI Overviews currently say about you. No report, no call, no follow-up unless you ask.
Received. Your three things arrive within two working days.
That did not send. Email hello@socialectric.com with your website address and I will reply from there.
Frequently asked questions
- What is the difference between llms.txt and sitemap.xml?
A sitemap is a machine-readable list of the URLs on your site, written in XML for search engine crawlers, and its job is discovery: it tells a crawler which pages exist and which you consider important. llms.txt is a Markdown file written for a language model rather than a crawler, and its job is orientation: it gives a short summary of what the organisation is and links to the handful of pages that matter most. The llms.txt specification makes the distinction itself, noting that sitemaps list every indexable page while llms.txt offers a curated overview, and that a full sitemap usually covers far more content than fits in a model's context window. They are complements, not substitutes, and neither replaces robots.txt, which governs access rather than meaning.
- Does AI search actually read llms.txt?
On the current evidence, mostly not. Google's documentation on its AI features says that you do not need to create new machine-readable files or AI text files to appear in AI Overviews or AI Mode, and that there are no additional requirements or special optimisations beyond ordinary search fundamentals. Ahrefs examined 137,000 domains in its own analytics product, found roughly 38,000 publishing a valid llms.txt, and reported that 97% of those files received no requests at all in May 2026. Where the file does get read is developer and agent tooling rather than consumer AI search. So publish one if you want to, but do not expect it to change your visibility, and do not let anyone sell it to you as an AI search strategy.
- Does our charity website need a sitemap?
Possibly not, which surprises most people. Google's own guidance says a sitemap matters most for large sites, brand new sites with few inbound links, and sites with a lot of video or image files, and that a small site of around 500 pages or fewer which is comprehensively linked internally may not need one, because a crawler can reach everything from the homepage. Most charity sites fall into that second group. The sitemap is still worth having, because it costs nothing and it tells Google when a page changed, but if your governance documents are not being found the sitemap is rarely the reason. An unlinked page that no menu points to is a much more common cause.
- How do we write an llms.txt file?
Create a plain text file called llms.txt, in Markdown, and serve it at yourdomain.org/llms.txt. The specification requires exactly one thing: an H1 with the name of the organisation or project. Everything else is optional, and the useful optional parts are a one-sentence blockquote summary of what you do, then H2 sections holding lists of links in the form of a Markdown link followed by an optional colon and a short note. Keep it to the pages someone would genuinely need in order to describe you accurately: what you do, who runs it, your annual report, your policies, your contact route. There is also an Optional section in the spec, for secondary links a model can skip when it needs less context.
- Who should own this file inside the organisation?
Whoever owns the website, and the point is that it has to be somebody. The file is a public, machine-read description of the organisation, so it carries the same accountability as any other published statement about who you are. If it lists an annual report that is two years out of date, or a policy page that has been moved, it is misdescribing you to anything that reads it. That makes it a governance item on a review cadence rather than a one-off technical task, and it belongs in the same review that checks your published policies are current.
- If llms.txt barely gets read, what actually makes our site usable by AI?
Being indexed, and being clear. Google ties eligibility for its AI features to the ordinary search index: a page has to be indexed and eligible to be shown with a snippet before an AI answer can use it as a supporting link. So the work that pays is unglamorous. Pages that exist at stable URLs, headings that describe what is under them, governance documents in HTML rather than locked inside a PDF, and content that answers the question it claims to answer. That is the same list as answer engine optimisation, and it is the same list as being readable by a trustee in a hurry.
- Is there any risk in publishing an llms.txt file?
Very little technically, and one thing worth thinking about. The file itself is inert Markdown and cannot break a page or affect your rankings. The risk is editorial: you are writing a short, authoritative description of your organisation and pointing at a small set of pages, which means you are choosing what represents you. Published badly, it reads as a stale summary of an organisation that has moved on. The other consideration is that llms.txt is not an access control. If you do not want your content used for model training, that decision belongs in robots.txt and in your terms, not here.