Crawl

Robots.txt & Sitemap Auditor

Name: Robots.txt & Sitemap Auditor
Address: Singapore
Telephone: 6281
Price range: 0
Rating: 4.8

Paste a domain to inspect robots.txt and crawl linked sitemaps (root/blog/blacklist/forum variants). Shows blocked URLs, orphaned sitemaps, and malformed entries. Client-side only—may be limited by CORS.

Audit

Site URL (e.g., https://example.com)

What we check

robots.txt availability and Disallow/Allow rules.
Sitemap discovery (robots.txt + common paths like /sitemap.xml, /blog/sitemap.xml).
Counts of URLs per sitemap and malformed entries.
Flags blocked URLs that also appear in sitemaps.