Technical SEO
Robots.txt Diff & Deploy-Risk Checker
Paste your live robots.txt and the version you're about to deploy. This runs both through the real longest-match/wildcard/$ crawler engine across the crawlers and paths you choose, then flags exactly what changed — and whether it's safe to ship.
Runs entirely in your browser. Nothing you paste here is sent anywhere except an anonymized usage count.
Leave empty to just audit the new file on its own, with no before/after comparison.
The version you are about to deploy.
Each rule engine run is per crawler — real crawlers never merge a specific group with the wildcard group.
Path and optional query string only, no domain. Include your homepage, key templates, and anything you're worried about.
How this works, and why a text diff isn't enough
A plain line-by-line text diff can't tell you what actually changed for a crawler: reordering groups, adding an Allow that undoes a Disallow, or a longer, more specific rule overriding a shorter one all change behavior without necessarily looking dramatic in a diff. This tool re-runs the real crawler algorithm — most-specific User-agent group wins, then longest matching Allow/Disallow rule wins, Allow breaking ties — against both files and compares outcomes, not lines.
Severity is based on what changed: a newly blocked homepage or site-wide Disallow is Critical, a newly blocked CSS/JS file or a newly exposed path that looks sensitive (admin, checkout, account, .env) is a Warning, and any other newly blocked or allowed path is Info so you can sanity-check it.
It also checks structure independent of the paths you list: whether the new file introduces a blanket "Disallow: /", whether a Sitemap directive was quietly dropped, and which crawler-specific groups were added or removed.
Remember that Google and Bing ignore Crawl-delay entirely, and that once a crawler matches a specific group like "Googlebot", the rules under "User-agent: *" are never applied to it — they are not merged.