Info · Directives

No llms.txt

Why it matters

llms.txt is an emerging convention that gives AI systems a curated guide to the site's key content.

How to fix it

Consider publishing /llms.txt with links to your most important pages.

How Crawlune finds it

Crawlune flags No llms.txt automatically on every crawl and lists the affected URLs under theDirectives tab, where you can filter, inspect each page and export the list to CSV. The free version crawls up to 500 URLs with every check enabled.