Claude
GoodBotVerifiedBOTClaudeBot helps enhance the utility and safety of generative AI models by collecting web content that could potentially contribute to their training. Operated by Anthropic.
Identification
ClaudeBotHow we rated this bot
Verified by Cloudflare, which confirms who runs it, with no widespread reports of abuse.
What we read
- Meta Has New Web-Crawling Bots That Sneakily Get Around Robots.txt - Business Insider
- The Major Companies and Media Outlets Blocking OpenAI's Crawler GPTBot - Business Insider
- Known Agents Review: Google Analytics for Bots | Agent Finder
- Aggressive
- HUMAN and TollBit Empower Websites with Robust Bot Management and Content Monetization Solutions for AI Agents
Block or allow Claude
Add a Disallow rule for ClaudeBot in your robots.txt file. You can also block at the server level using your web server configuration or CDN firewall rules to filter requests matching the user-agent string.
# Block Claude from crawling your entire site User-agent: ClaudeBot Disallow: / # Allow Claude full access User-agent: ClaudeBot Allow: /
Ensure your robots.txt allows ClaudeBot. Verify requests by checking the user-agent string.
Traffic & trends
Share of bot traffic observed by Cloudflare Radar.
Loading trend data…
From the community
Recent Reddit discussion mentioning this bot.
Related bots
Other crawlers operated by Anthropic.
Links & references
Data sources
This profile is compiled from the following sources.
Something wrong with this entry? Suggest a correction