An independent record of who is crawling the web
Bot WHOIS exists so that anyone running a website can answer one question with confidence: who is this bot, and should I let it in? We track, verify, and rate every automated client we can find - and we take no money from the operators we list.
The web got a lot more crowded
A few years ago, the bots hitting your site were mostly search engines you recognised. Today the picture is very different: AI training crawlers, answer-engine indexers, autonomous assistants, and a long tail of scrapers and monitors - many of which arrived without announcement. Site owners were left guessing which user-agents were real, which honoured robots.txt, and which were quietly impersonating someone else.
We started Bot WHOIS to replace that guesswork with a verifiable record. Every bot is cross-referenced against its operator's own published signals and independent sources before it is listed, and every rating is grounded in those sources - operator records, Cloudflare's verified-bot programme, and public documentation - rather than marketing. The whole registry is queryable through our API, and the process behind it is documented on our Methodology page.
Principles
Independent
No operator pays to be listed, verified, or rated. The registry is not for sale.
Verifiable
Every listing is tied back to sources you can check yourself - operator records, Cloudflare's verified-bot programme, and public documentation.
Evidence-based
Ratings combine a bot's verified identity with any behaviour our sources have flagged - drawn from operator records, verified-bot programmes, and documentation - not the reputation of the company behind it.
Correctable
We get things wrong sometimes. Anyone can send us a correction, and we review it against our sources.
Seen a bot we are missing?
Tell us and we will run it through the same collection and verification as every other entry.