All categories
CategoryBotsFlagged botsDescription
MO
Monitoring & Analytics
226
0
Uptime, performance, and analytics bots that check that sites are up and measure how they perform.
›
AI
AI Assistant
142
0
Agents that browse pages and take actions on behalf of a user in real time, rather than crawling in bulk.
›
SE
Search Engine Crawler
120
0
Discovery-and-indexing crawlers that feed web, image, news, and video search results.
›
AC
Academic Research
113
0
Crawlers run by universities and research groups to study the web.
›
SE
SEO
111
0
Backlink, rank-tracking, and site-audit crawlers run by marketing-tools vendors.
›
FE
Feed Fetcher
73
0
Bots that poll RSS/Atom feeds and syndicated content on a schedule.
›
SE
Security
60
0
Scanners that probe the web for vulnerabilities, exposed services, and threats.
›
WE
Webhooks
57
0
Event-driven callbacks and notification fetchers from apps and platforms.
›
AD
Advertising & Marketing
52
0
Crawlers used for ad verification, brand safety, and contextual targeting.
›
PA
Page Preview
52
0
Link-preview fetchers that render a card when your URL is shared in chat or social apps.
›
AI
AI Crawler
42
0
Bots that gather public web pages to build training data for large language models.
›
AG
Aggregator
39
0
Bots that collect and republish content or metadata from across many sites.
›
OT
Other
27
0
Bots that do not fit the main categories.
›
AI
AI Search
17
2
Crawlers powering AI answer engines that index pages and cite them as sources in generated responses.
›
VE
Verified Bot
16
0
Bots confirmed against an authoritative source such as Cloudflare’s verified-bot list.
›
UN
Uncategorized
11
1
Bots that have not yet been categorised.
›
AC
Accessibility
10
0
Bots that audit pages for accessibility and compliance issues.
›
AR
Archiver
7
0
Bots that capture and preserve snapshots of web pages over time.
›
SO
Social Media Marketing
4
0
Bots that support social-media scheduling, monitoring, and marketing tools.
›
SE
Search Engine
2
0
Automated web clients grouped under this category.
›
20 categories · 1,181 bots tracked