FirmCredBot
GoodBotBotwhois.orgA specialized crawler used by the Firm Cred directory to verify the status, availability, and basic information of listed business websites.
Identification
How we rated this bot
Identifies itself honestly, and we found no credible reports of abuse.
What we read
- Web Robots: baiduspider bad bot ignores robots.txt
- Google just added a crawler that ignores robots.txt — Suganthan
- AI Crawl Control robots.txt injection ignores settings across 4 zones (Disallow per - Application Security / Bot Management - Cloudflare Community
- AI Crawl Control robots.txt injection ignores settings across 4 zones (Disallow per - Application Security / Bot Management - Cloudflare Community
- Meta Has New Web-Crawling Bots That Sneakily Get Around Robots.txt - Business Insider
Block or allow FirmCredBot
Add a Disallow rule for FirmCredBot/1.0 (+https://www.firmcred.com/bot; directory listing compiler; contact: [email protected]) in your robots.txt file. You can also block at the server level using your web server configuration or CDN firewall rules to filter requests matching the user-agent string.
# Block FirmCredBot from crawling your entire site User-agent: FirmCredBot/1.0 (+https://www.firmcred.com/bot; directory listing compiler; contact: [email protected]) Disallow: / # Allow FirmCredBot full access User-agent: FirmCredBot/1.0 (+https://www.firmcred.com/bot; directory listing compiler; contact: [email protected]) Allow: /
Ensure your robots.txt allows FirmCredBot/1.0 (+https://www.firmcred.com/bot; directory listing compiler; contact: [email protected]). Verify requests by checking the user-agent string.
Traffic & trends
No Cloudflare traffic data available.
Loading trend data…
Links & references
Data sources
This profile is compiled from the following sources.
Something wrong with this entry? Suggest a correction