CR
CrowBot
GoodBotVerifiedBOTCaptures public storefronts to study how online stores are built. Obeys robots.txt, rate limited, signed requests.
Identification
Bot name
CrowBot
Category
Academic Research
Kind
BOT
Verification
Verified bot
Cloudflare slug
crowbot
Block or allow CrowBot
Block it
Add a Disallow rule for CrowBot in your robots.txt file. You can also block at the server level using your web server configuration or CDN firewall rules to filter requests matching the user-agent string.
# Block CrowBot from crawling your entire site User-agent: CrowBot Disallow: / # Allow CrowBot full access User-agent: CrowBot Allow: /
Allow & verify
Ensure your robots.txt allows CrowBot. Verify requests by checking the user-agent string.
Traffic & trends
Cloudflare Radar traffic
No Cloudflare traffic data available.
Google Trends interest
Loading trend data…
Links & references
Data sources
This profile is compiled from the following sources.
CloudflareRadar
Last updated October 27, 2026
Something wrong with this entry? Suggest a correction