E-COMMERCE & LOCAL SEO
Faceted URL Crawl Risk Analyzer
Faceted navigation lets shoppers filter by brand, colour and price, and lets crawlers find millions of near-identical URLs. Google’s guidance is to keep the filters you don’t need indexed out of crawling. This analyzer reads a sample of your URLs and shows where the combinations come from.
- Runs in your browser
- Nothing you paste is uploaded
- Free, no sign-up
- CheckParameter kinds: filter, sort, pagination, search, session, tracking
- CheckPossible filter combinations
- CheckSame parameters in different orders
- CheckEmpty values and values in mixed letter case
Free
no sign-up, no usage limit
0 uploads
what you paste stays on your device
5 checks
what the tool looks at
Offline
works once the page has loaded
Loading the tool…
HOW IT WORKS
How to use the Faceted URL Crawl Risk Analyzer
A list of URLs, one per line, up to about 50,000.
Export URLs from a crawl, your server logs or Search Console’s pages report.
Paste them and read the parameter table.
Decide which filters deserve indexable pages and contain the rest.
WHAT IT CHECKS
What it looks at, and how to read it.
Each parameter with its kind and advice, an estimate of possible combinations, checks for inconsistencies, and robots.txt lines for parameters that never need crawling.
What it checks
- Parameter kinds: filter, sort, pagination, search, session, tracking
- Possible filter combinations
- Same parameters in different orders
- Empty values and values in mixed letter case
- Session and tracking parameters in URLs
How to read the results
- A filter people search for (brand, type) may deserve an indexable page; sort orders never do.
- The suggested robots.txt lines stop crawling, not indexing of URLs already known.
The concepts behind it, in the SEO glossary: Crawl budget, Faceted navigation.
GOOD TO KNOW
Common mistakes, an example and the limits.
Common mistakes
- Blocking filters in robots.txt and also noindexing them; a blocked page’s noindex is never seen.
- Linking tracking parameters internally.
A worked example
In the example, brand and colour appear in two orders, a session id leaks into links and four filters stack on one URL.
Limitations
- Estimates combinations from the values in your list; the real number on the site may be larger.
- Recognises parameters by common names; a filter called something unusual is treated as a filter.
FAQ
Questions about the Faceted URL Crawl Risk Analyzer
Straight answers about what it does and where its limits are.
Should I use robots.txt or canonicals for filters?
robots.txt to stop crawling filters you never want indexed; canonicals for near-duplicates that must stay crawlable.
Need this for a whole website, over time?
The NdovuSEOLabs platform connects your own Search Console, analytics and server logs, and audits every page on a schedule.