PR
Project Syndicate Crawler
GoodBotVerifiedBOTFinds reprints of Project Syndicate commentaries, to confirm licensed syndication and detect unlicensed republication.
Identification
Bot name
Project Syndicate Crawler
Category
Academic Research
Kind
BOT
Verification
Verified bot
Cloudflare slug
project-syndicate-crawler
User-agent patterns
PSCrawler/User-agent string
PSCrawler/1.0 (+https://www.project-syndicate.org/pages/crawler-info)
Rated Oct 9, 2026 from public evidence
How we rated this bot
Verified by Cloudflare, which confirms who runs it, with no widespread reports of abuse.
Names itself honestly
Yes
Verified by Cloudflare
Yes
Says it follows robots.txt and opt-outs
Not stated
Ignores site rules
None found
Disguises itself or evades blocks
None found
Overloads sites
None found
How widely people complain
No complaint about this bot in the evidence
Reddit discussion
There is no Reddit discussion of this bot in the evidence
What we read
- Googlebot ignores robots.txt - Google Search Central Community
- Crawler without limits: Perplexity ignores robots.txt | heise online
- Perplexity AI ignores no-crawling rules on websites, crawls them anyway | Malwarebytes
- A new web crawler launched by Meta last month is quietly scraping the internet for AI training data
- Meta Has New Web-Crawling Bots That Sneakily Get Around Robots.txt - Business Insider
Block or allow Project Syndicate Crawler
Block it
Add a Disallow rule for PSCrawler/ in your robots.txt file. You can also block at the server level using your web server configuration or CDN firewall rules to filter requests matching the user-agent string.
# Block Project Syndicate Crawler from crawling your entire site User-agent: PSCrawler/ Disallow: / # Allow Project Syndicate Crawler full access User-agent: PSCrawler/ Allow: /
Allow & verify
Ensure your robots.txt allows PSCrawler/. Verify requests by checking the user-agent string.
Traffic & trends
Cloudflare Radar traffic
No Cloudflare traffic data available.
Google Trends interest
Loading trend data…
Links & references
Data sources
This profile is compiled from the following sources.
CloudflareRadar
Last updated October 9, 2026
Something wrong with this entry? Suggest a correction