SITEMAPS & CRAWLING
Sitemap Finder
A sitemap lists the pages a site wants crawled. Search engines find it through robots.txt, Search Console or a guess at the usual address. This check does what a crawler does: reads robots.txt, tries the usual places, and opens each sitemap it finds.
- Checks the live page
- Public addresses only
- Free, no sign-up
- CheckSitemap lines in robots.txt, and whether they load
- CheckThe usual locations: /sitemap.xml, /sitemap_index.xml, /wp-sitemap.xml
- CheckURL set or sitemap index, gzip, and entry count
- CheckEntries without full URLs or on another host
Free
no sign-up
Live
fetched when you run the check
5 checks
what the tool looks at
30 a minute
checks per visitor, to keep it free
Loading the tool…
HOW IT WORKS
How to use the Sitemap Finder
A domain or any page URL on the site.
Enter a domain or any page on the site.
See which sitemaps exist and where they were found.
Add the main sitemap to robots.txt and to Search Console if it is missing from either.
WHAT IT CHECKS
What it looks at, and how to read it.
Every sitemap found with where it was found, its status, type, entry count and newest date, plus the child sitemaps of an index.
What it checks
- Sitemap lines in robots.txt, and whether they load
- The usual locations: /sitemap.xml, /sitemap_index.xml, /wp-sitemap.xml
- URL set or sitemap index, gzip, and entry count
- Entries without full URLs or on another host
- Duplicate URLs and invalid or future lastmod dates
How to read the results
- A sitemap that robots.txt names but that fails to load is a broken signal crawlers keep retrying.
- Several sitemaps are normal; an index ties them together.
- lastmod helps only when it is the real date the page changed.
The concepts behind it, in the SEO glossary: XML sitemap.
GOOD TO KNOW
Common mistakes, an example and the limits.
Common mistakes
- Setting every lastmod to today on each rebuild.
- Listing redirected, noindex or non-canonical URLs.
- Two plugins each publishing a sitemap with different contents.
A worked example
A store finds both /sitemap_index.xml from its SEO plugin and /wp-sitemap.xml from WordPress. It keeps one, disables the other, and lists the survivor in robots.txt.
Limitations
- Checks robots.txt and four common locations; a sitemap at an unusual address is found only if robots.txt names it.
- Reads up to 20 MB of each sitemap, and lists up to 500 entries of an index.
- Does not fetch the URLs inside a sitemap.
FAQ
Questions about the Sitemap Finder
Straight answers about what it does and where its limits are.
Do I need a sitemap?
Small, well-linked sites are found without one, but a sitemap helps search engines find new and deep pages sooner, and is easy to add.
Does a sitemap get pages indexed?
It helps pages get found and crawled. Whether they are indexed still depends on the pages themselves.
See which sitemap pages Google has indexed.
Free tools check one thing at a time. The NdovuSEOLabs platform connects your own Search Console, analytics and server logs, and keeps checking every page.