GraphInsightBot
GoodBotVerifiedBOTFetches articles from the RSS/Atom feeds GraphInsight readers subscribe to. No crawling, no AI training. Operated by GraphInsight.
What is GraphInsightBot?
GraphInsightBot is the dedicated crawler for GraphInsight, a news aggregation service that provides users with a unified reading experience. Unlike general-purpose search crawlers, this bot operates exclusively on URLs explicitly published in RSS or Atom feeds subscribed to by its users. It is designed to be lightweight and polite, ensuring that it does not overwhelm servers by spacing out requests and adhering strictly to site-specific crawl-delay settings.
Identification
How we rated this bot
Verified by Cloudflare, which confirms who runs it, with no widespread reports of abuse.
What we read
- google ignoring robots.txt ? - Google Search Central Community
- I wonder if it will actually respect the robots.txt this time. | Hacker News
- How to deal with AI crawlers that ignore robots.txt | Soups Ranjan posted on the topic | LinkedIn
- Meta Has New Web-Crawling Bots That Sneakily Get Around Robots.txt - Business Insider
- 20 Common Robots.txt Issues (and How to Avoid Them)
Why it crawls your site
The bot crawls to fetch the full content of articles linked within user-subscribed feeds so they can be displayed in the GraphInsight application. It prioritizes transparency by signing its requests using Ed25519 (RFC 9421), allowing site administrators to cryptographically verify that the traffic originates from the official GraphInsight infrastructure. It does not scrape sites indiscriminately, nor does it use the collected data for training AI models or republishing content.
Block or allow GraphInsightBot
Add a Disallow rule for GraphInsightBot in your robots.txt file. You can also block at the server level using your web server configuration or CDN firewall rules to filter requests matching the user-agent string.
# Block GraphInsightBot from crawling your entire site User-agent: GraphInsightBot Disallow: / # Allow GraphInsightBot full access User-agent: GraphInsightBot Allow: /
Ensure your robots.txt allows GraphInsightBot. Verify requests by checking the user-agent string.
Traffic & trends
No Cloudflare traffic data available.
Loading trend data…
Links & references
Data sources
This profile is compiled from the following sources.
Something wrong with this entry? Suggest a correction