Directory/Categories/Search Engine Crawler/CareerCornerBot

CareerCornerBot

GoodBotVerifiedBOT

Reads job listings on employers' careers sites for job seekers. Obeys robots.txt, 1 req/s per host, stops on 403/429.

Operator website ↗

GoodBot

Assessment

Search Engine Crawler

Identification

Bot name

CareerCornerBot

Block or allow CareerCornerBot

Block it

Add a Disallow rule for CareerCornerBot in your robots.txt file. You can also block at the server level using your web server configuration or CDN firewall rules to filter requests matching the user-agent string.

# Block CareerCornerBot from crawling your entire site
User-agent: CareerCornerBot
Disallow: /

# Allow CareerCornerBot full access
User-agent: CareerCornerBot
Allow: /