Skip to main content

Try Base free for 14 days, no card or M-Pesa needed See pricing

CRAWLER DIRECTORY · MONITORING BOTS

Monitoring bots.

Uptime checkers, redirect services, compliance scanners and brand monitors that load the pages someone asked them to watch.

Browse the listAll categories

Monitoring bots by purpose16 named bots
16MonitoringAwarioBot, Better Stack, Checkly…
16named bots
13can be verified
5follow robots.txt
16operators

MONITORING BOTS

All 16 named bots.

Search by name or token, or filter by purpose, verification and operator.

16 bots

AwarioUnverified

AwarioBot

AwarioBot AwarioSmartBot AwarioRssBot

  • Monitoring

Follows robots.txt

Better StackVerifiable

Better Stack

No robots.txt token

  • Monitoring

User-triggered

ChecklyVerifiable

Checkly

No robots.txt token

  • Monitoring

User-triggered

FreshworksUnverified

FreshpingBot

No robots.txt token

  • Monitoring

User-triggered

HetrixToolsVerifiable

HetrixTools

No robots.txt token

  • Monitoring

User-triggered

SolarWindsVerifiable

Pingdom

No robots.txt token

  • Monitoring

User-triggered

QA.techVerifiable

QATechBot

QATechBot

  • Monitoring

Follows robots.txt

redirect.pizzaVerifiable

redirect.pizza

No robots.txt token

  • Monitoring

Not documented

Tech WarriorVerifiable

ScooterZA

ScooterZA

  • Monitoring

Follows robots.txt

ZohoVerifiable

Site24x7

No robots.txt token

  • Monitoring

User-triggered

StatusCakeVerifiable

StatusCake

No robots.txt token

  • Monitoring

User-triggered

TermlyVerifiable

TermlyBot

TermlyBot

  • Monitoring

Follows robots.txt

UptimeRobotVerifiable

UptimeRobot

No robots.txt token

  • Monitoring

User-triggered

WEBSPARKVerifiable

WARDBot

WARDBot

  • Monitoring

Follows robots.txt

FAQ

Monitoring bots: common questions.

Before you change your robots.txt.

All questions

Should I block monitoring bots?

Usually not. Most check pages that you, or a service you use, asked them to watch, and they do not read robots.txt. To stop one, remove the check in that service or block it at the server.

How do I block one?

Open its profile and copy the robots.txt rule. Bots that ignore robots.txt, or fetch only when a person asks, need a server, firewall or CDN rule instead.

See which of these crawlers visit your site.

Crawl Analytics reads your own server and CDN logs and checks each claim against the operator’s published IP ranges and DNS.

Explore Crawl Analytics