No robots.txt

Opportunity severity, in the Directives tab

Why it matters

Without robots.txt every crawler must guess, and you cannot declare sitemaps or block crawl traps.

How to fix it

Publish a robots.txt, even a permissive one, and reference your sitemap in it.

How Crawlune finds it

Crawlune flags No robots.txt automatically on every crawl and lists the affected URLs under the Directives tab, where you can filter, inspect each page and export the list to CSV. The free version crawls up to 550 URLs with every check enabled.