BeanwiredBot
GoodBotVerifiedBOTReads public specialty-coffee catalogs to power release/restock/price alerts. Honors robots.txt. No logins or orders. Operated by Beanwired.
What is BeanwiredBot?
BeanwiredBot is a niche crawler operated by Beanwired, designed to track public-facing specialty coffee shop catalogs. It monitors product feeds and catalog pages to notify subscribers about restocks, new releases, and price changes. The bot is engineered to be lightweight, operating with configurable crawl intervals and built-in backoff mechanisms to ensure it does not negatively impact site performance.
Identification
How we rated this bot
Verified by Cloudflare, which confirms who runs it, with no widespread reports of abuse.
What we read
- google ignoring robots.txt ? - Google Search Central Community
- I wonder if it will actually respect the robots.txt this time. | Hacker News
- How to deal with AI crawlers that ignore robots.txt | Soups Ranjan posted on the topic | LinkedIn
- Meta Has New Web-Crawling Bots That Sneakily Get Around Robots.txt - Business Insider
- Michael Tsai - Blog - AI Companies Ignoring Robots.txt
Why it crawls your site
The bot crawls to maintain an up-to-date index of coffee product availability, allowing the Beanwired platform to send timely alerts to its users. It specifically targets public catalog pages and JSON product feeds. The operator provides clear documentation and direct communication channels for site owners to adjust crawl frequency or request removal, demonstrating a commitment to cooperative web scraping practices.
Block or allow BeanwiredBot
Add a Disallow rule for BeanwiredBot in your robots.txt file. You can also block at the server level using your web server configuration or CDN firewall rules to filter requests matching the user-agent string.
# Block BeanwiredBot from crawling your entire site User-agent: BeanwiredBot Disallow: / # Allow BeanwiredBot full access User-agent: BeanwiredBot Allow: /
Ensure your robots.txt allows BeanwiredBot. Verify requests by checking the user-agent string.
Traffic & trends
No Cloudflare traffic data available.
Loading trend data…
Links & references
Data sources
This profile is compiled from the following sources.
Something wrong with this entry? Suggest a correction