AI SEARCH & BOTS
AI Crawler Log Analyzer
Anyone can put "GPTBot" in a user agent. Server logs show which AI crawlers came and how often, but only an IP check tells the real ones from imitators. This analyzer reads your log lines in the browser and checks each bot against the IP ranges its operator publishes, with the same engine as NdovuSEOLabs Bot Intelligence.
- Runs in your browser
- Nothing you paste is uploaded
- Free, no sign-up
- CheckUser agents of every documented AI and search crawler
- CheckIP checks against published ranges
- CheckResponse codes by bot
- CheckMost-requested paths
Free
no sign-up, no usage limit
0 uploads
what you paste stays on your device
4 checks
what the tool looks at
Offline
works once the page has loaded
Loading the tool…
HOW IT WORKS
How to use the AI Crawler Log Analyzer
Access log lines in Nginx or Apache combined format.
Download a day of access logs from your host (Nginx or Apache combined format).
Paste the lines; nothing is uploaded.
Look at verified and spoofed counts, then at the paths each bot requests.
WHAT IT CHECKS
What it looks at, and how to read it.
Each bot’s requests, how many were verified or spoofed, error responses and most-requested paths.
What it checks
- User agents of every documented AI and search crawler
- IP checks against published ranges
- Response codes by bot
- Most-requested paths
How to read the results
- Spoofed means the request claimed a bot name from an address the operator does not use.
- Unverified means there is no published range to check, or the address is not in one.
The concepts behind it, in the SEO glossary: Web crawler, Crawl logs, AI crawlers.
GOOD TO KNOW
Common mistakes, an example and the limits.
Common mistakes
- Counting user agents as real visits without verifying them.
- Blocking a real crawler by IP after reading spoofed traffic.
- Reading one hour of logs as a trend.
A worked example
A day of logs shows 1,200 "GPTBot" requests: 900 verified from OpenAI’s ranges, 300 spoofed from a hosting provider scraping the catalogue.
Limitations
- Verification uses the IP ranges operators publish; bots verified only by reverse DNS show as unverified here.
- Your log is read in your browser. Only the public IP range lists are downloaded from the Ndovu tools API.
FAQ
Questions about the AI Crawler Log Analyzer
Straight answers about what it does and where its limits are.
Is my log uploaded?
No. It is parsed in your browser. The tool downloads the public crawler IP lists to verify against; your data never leaves your device.
Which log formats work?
The combined format Nginx and Apache use by default, with an optional response time at the end.
Verify every bot visit continuously from your logs.
Free tools check one thing at a time. The NdovuSEOLabs platform connects your own Search Console, analytics and server logs, and keeps checking every page.