Conflicting robots directives

Error severity, in the Directives tab

Why it matters

The meta robots tag and the X-Robots-Tag header disagree; search engines apply the most restrictive.

How to fix it

Use one source of truth for robots directives.

How Crawlune finds it

Crawlune flags Conflicting robots directives automatically on every crawl and lists the affected URLs under the Directives tab, where you can filter, inspect each page and export the list to CSV. The free version crawls up to 550 URLs with every check enabled.