Skip to main content

Try Base free for 14 days, no card or M-Pesa needed See pricing

SOLUTIONS · NEWS & MEDIA

SEO for publishers who need to know who reads their articles.

See which search and AI crawlers fetch your stories and how much bandwidth they use, block the ones that only take, and hear about it when a Google update moves your traffic.

  • Verified crawlers from your logs
  • Bandwidth by bot
  • Google update alerts
Site health

Crawl Analytics

Which search and AI bots crawl your site, and what goes wrong.

Inspect URL
OverviewBot insightsCrawl issuesErrors & blocksSearch crawlingAI crawlingAll URLsSite sectionsCrawlersRequests
Bandwidth by crawleryournews.co.ke · last 7 days
Googlebot41 GB
Bytespider26 GB
GPTBot12 GB
Bingbot7 GB
Verifiedcrawlers checked against operator addresses
Bandwidthper bot, from your logs
400 daysof crawl history on Max
Every 5 minchecks for server-error surges
Inside the product

Your logs, your crawlers, your call.

Know who reads your journalism before you decide who may.

AI crawlers

See which AI bots read your stories

Crawl Analytics separates search crawlers from AI crawlers and verifies each one, so a fake “Googlebot” is never counted as Google.

  • Search vs AI crawlers
  • Verified, unverified and spoofed
  • Alert on new crawlers
Explore Crawl Analytics
Site health

Crawl Analytics

Search crawlingAI crawlingAll URLsSite sectionsCrawlersRequests
AI crawlersLast 7 days
GPTBot182K requestsVerified
OAI-SearchBot41K requestsVerified
ClaudeBot36K requestsVerified
“Googlebot” from an unknown network9K requestsSpoofed
Bandwidth

Know what each bot costs you

Bot insights show requests and bandwidth per crawler, so you can see which bots take the most and decide on robots.txt with numbers.

  • Bandwidth per bot
  • Volume spike alerts
  • robots.txt changes alerted
Look up any crawler
Site health

Crawl Analytics

OverviewBot insightsCrawl issuesErrors & blocksSearch crawlingAI crawlingAll URLsSite sectionsCrawlersRequests
Bot insightsLast 7 days
CrawlerRequestsBandwidth
Googlebot612K41 GB
Bytespider388K26 GB
GPTBot182K12 GB
Google updates

Know when an update hits you

Search Performance marks Google’s confirmed updates on your traffic, and an alert tells you when one moves your clicks.

  • Updates on your chart
  • Alert on impact
  • Site sections compared
Explore Search Console insights
Performance

Search Performance

Search engineAll availableGoogleBing
Clicks around a core updateDaily
Clicks48.2K−14%vs previous
Impressions2.1M−6%vs previous
Clicks1Core update rollout
1
28d ago15d agoToday
Sections

See which sections win and lose

Site sections are grouped automatically by path, so /politics/, /sports/ and /business/ each get their own clicks and trend.

  • Automatic site sections
  • Your own content groups
  • Branded vs non-branded
Explore Search Console insights
Performance

Search Performance

AI searchesSegmentsGoogle updates
Clicks by sectionLast 28 days
/news/412K
/sports/239K
/business/128K
/entertainment/111K
Article speed

Fix speed on the article template

Core Web Vitals and site audit issues are grouped by template, so one fix to the article layout reaches every story.

  • Article template
  • Real-user Core Web Vitals
  • Issues by template
Explore Core Web Vitals
Site health

Site Audit

ImagesTemplatesSitemapsInternal linksArchitecture
TemplatesLatest audit
TemplateURLsAvg response
Article9,4201.4 s
Tag archive3,1102.2 s
Author archive2401.9 s
Data & limits

What publishers should know.

Logs neededCrawler data comes from your server or CDN logs: the WordPress plugin, Cloudflare Logpush or an import.
Verified means checkedA bot is verified against its operator’s published addresses; others show as unverified or spoofed.
Not coveredGoogle Discover and Top Stories reports are not part of the product.
History180 days of crawl history on Pro, 400 days on Max.
How it works

From logs to decisions.

  1. 01

    Connect your logs

    Install the WordPress plugin or send Cloudflare logs.

  2. 02

    Read the crawlers

    See verified search and AI bots, bandwidth and spoofed traffic.

  3. 03

    Decide and watch

    Change robots.txt, then watch requests and alerts.

With your AI agent

Ask what an AI bot is worth to you.

Your agent reads bot activity and GA4 through MCP.

  • Bot activity
  • Request detail
  • GA4 acquisition
Claude · yournews.co.ke

YouWhich AI crawlers take the most bandwidth but send us no visits?

get_bot_activity
get_request_intelligence
get_google_analytics_traffic_acquisition

AgentBytespider used 26 GB this week and no ByteDance product sent a measurable visit. GPTBot used 12 GB; ChatGPT sent 1,940 sessions. Blocking Bytespider saves the most for the least.

Use cases

Built for publishers.

News sites

Fresh stories, fast

Check Googlebot reaches new articles and errors don’t pile up.

Magazines & blogs

Evergreen that lasts

See which sections hold traffic after each Google update.

Broadcasters

Big sites, big bills

Find the bots behind your bandwidth bill.

Bandwidth study

Keep exploring

FAQ

News & media: common questions.

All questions
Can I see which AI bots crawl my site?

Yes. Crawl Analytics reads your logs and verifies each search and AI crawler against its operator’s published addresses.

Can I see bandwidth per bot?

Yes. Bot insights show requests and bandwidth for each crawler.

Does it cover Google Discover or Top Stories?

No. Search data comes from Search Console’s web search reports.

Will I know if a Google update hits?

Yes. Updates are marked on your Search Performance chart and an alert fires when one moves your clicks.

How do I get started?

Create an account at app.ndovuseolabs.com, add your website and connect Search Console. Every new account starts with a 14-day free trial of Base, with no card or M-Pesa needed; then choose the plan that fits.

SEO for news & media

See who reads your stories.

Try Base free for 14 days. No card or M-Pesa needed.