Free tool
The llms.txt generator that reads your real site
Paste your domain and get a ready-to-upload llms.txt: sitemap crawled, pages sorted into sections, real titles and descriptions where they matter most. No signup, no AI guesswork — done in about twenty seconds.
Reads your public sitemap · no AI calls · nothing stored
After you download
Three steps to live
The file only works from one place, and it only stays useful if it stays true. Here’s the whole install.
- 01
Upload it to your web root
The file must be served at https://yoursite.com/llms.txt — the root of your domain, right next to robots.txt, not inside a subfolder. On most hosts that’s one drag into the public directory; on WordPress, a file-manager or SEO plugin that writes root files does it without FTP.
- 02
Check it, then keep it current
Open yoursite.com/llms.txt in a browser — you should see plain text, not a styled page or a redirect. Then treat it like your sitemap: regenerate after you ship a new section, product line, or docs area. A stale map quietly misleads every assistant that reads it.
- 03
Optional: add llms-full.txt
The spec also allows a companion file that inlines full page content instead of linking out. If you run documentation an assistant should be able to read end-to-end, that’s the next step — but the curated map you just generated is the part every site should have.
The standard
What is llms.txt?
A proposed standard for telling AI systems what your site is and where its substance lives — in the one format they parse best.
llms.txt is a plain-markdown file served from your web root. It opens with your site’s name as a heading, a one-line summary as a blockquote, and then curated sections of links — each with a short description of what an assistant will find there.
Your existing files each do a different job. robots.txt says who may crawl. sitemap.xml lists everything you have, without judgment. llms.txt is the editorial layer neither of them provides: what actually matters here, and why — written in markdown, the format language models read with the least friction.
Why bother
A curated map beats a thousand-page crawl
AI assistants work under hard context limits. What they can hold about your site is small — so what you put in front of them matters.
When an assistant tries to understand your site at answer time, it can’t read all of it. It samples — a page here, a nav there, whatever the crawler happened to keep. A hundred-line llms.txt replaces that sampling with your own editorial judgment: the pages you’d show a smart visitor with five minutes, each labeled with what it’s for.
The honest caveat: llms.txt is a proposal, and adoption across AI platforms is still early. Nobody should promise you it changes your AI visibility tomorrow. But the cost of having one is a few minutes and a file upload, the downside is zero, and the sites that adopted sitemap.xml early would tell you how that asymmetry tends to pay off.
Under the hood
How this generator works
Deterministic from end to end — parsing and sorting, no LLM anywhere in the pipeline. The same site in gives the same file out.
Sitemap in
We fetch your homepage for the site name and summary, then read /sitemap.xml — following a sitemap index to its most page-like children, and falling back to /sitemap_index.xml, /wp-sitemap.xml, or your homepage’s own links. Up to 150 unique same-site pages, with assets, feeds, and WordPress internals filtered out.
Sections by path
Every URL is sorted by path heuristics: /docs and /help land in Docs, /blog and /news in Blog, /pricing and /features in Products, /about and /contact in Company, legal pages in Optional (last, as the spec asks), and everything else in Key pages.
Real titles, honest descriptions
The twelve most important pages — shortest paths first — get fetched for their actual title tag and meta description. Every other entry gets a clean name derived from its slug. Descriptions are never invented: if your page doesn’t declare one, the line simply goes without.
Quick answers
llms.txt, in four questions
Does llms.txt replace robots.txt?
No — they answer different questions. robots.txt tells crawlers what they may fetch; llms.txt tells assistants what’s worth reading once they’re in. You want both: check who can reach you with our AI Crawler Checker, then hand the ones you allow a proper map.
Do AI companies actually respect it?
Honestly: adoption is early and uneven. Some AI companies publish llms.txt files for their own docs and a growing set of tools fetch them, but no major assistant has committed to honoring the file everywhere. Treat it like sitemap.xml in its first year — cheap to adopt, useful to every system that looks, and a signal of a well-kept site.
Where exactly do I put the file?
At the root of your domain, so it resolves at https://yoursite.com/llms.txt — the same place robots.txt lives. Serve it as plain UTF-8 text (it’s markdown, but the .txt extension is part of the spec) and make sure it isn’t redirected to an HTML page.
What about llms-full.txt?
It’s the optional companion: where llms.txt is a curated map of links, llms-full.txt inlines the full content of your key pages so an assistant can read them without another fetch. Start with the map — it’s the part of the proposal with the clearest value — and add the full version if you maintain docs that deserve it.
Start tracking
A map is step one
OptimizeCamp measures whether AI answers actually cite you — and finds the content gaps that keep you out of them.