CL
Cloudflare Web Scanner
GoodBotVerifiedBOTCloudflare Web Scanner is a well-behaved crawler that checks domain liveness, content type, robots.txt directives, and other web properties to map the shape of the web.
Identification
Bot name
Cloudflare Web Scanner
Category
Search Engine Crawler
Kind
BOT
Verification
Verified bot
Cloudflare slug
cloudflare-web-scanner
User-agent patterns
CloudflareWebScannerUser-agent string
CloudflareWebScanner/1.0
Block or allow Cloudflare Web Scanner
Block it
Add a Disallow rule for CloudflareWebScanner in your robots.txt file. You can also block at the server level using your web server configuration or CDN firewall rules to filter requests matching the user-agent string.
# Block Cloudflare Web Scanner from crawling your entire site User-agent: CloudflareWebScanner Disallow: / # Allow Cloudflare Web Scanner full access User-agent: CloudflareWebScanner Allow: /
Allow & verify
Ensure your robots.txt allows CloudflareWebScanner. Verify requests by checking the user-agent string.
Traffic & trends
Cloudflare Radar traffic
No Cloudflare traffic data available.
Google Trends interest
Loading trend data…
Links & references
Data sources
This profile is compiled from the following sources.
CloudflareRadar
Last updated September 8, 2026
Something wrong with this entry? Suggest a correction