CRAWLER OPERATOR
Tech Warrior crawlers.
Tech Warrior runs one bot in this directory, ScooterZA. It can be allowed or blocked on its own.
- 1 bot
- 1 can be verified
- 1 follow robots.txt
TECH WARRIOR
Every Tech Warrior bot.
Open a bot for its robots.txt rule, verification and documentation.
What does blocking Tech Warrior’s crawlers cost?
None of Tech Warrior’s bots build a search index, so blocking them does not change your search rankings.
How do I check a request really comes from Tech Warrior?
A request is really from Tech Warrior when its IP is in the list Tech Warrior publishes (techwarrior.co.za). The user agent alone can be copied by anyone.
Where does this come from?
From Tech Warrior’s own documentation, checked for each bot and linked on its profile, through the registry Ndovu uses to classify crawler traffic in Crawl Analytics.
See which of these crawlers visit your site.
Crawl Analytics reads your own server and CDN logs and checks each claim against the operator’s published IP ranges and DNS.