AD
AdRetriever Site Walk
GoodBotVerifiedBOTSite Walk lets AdRetriever keep your digital ads up to date by collecting product information from your website.
Identification
Bot name
AdRetriever Site Walk
Category
Academic Research
Kind
BOT
Verification
Verified bot
Cloudflare slug
adretriever-site-walk
User-agent patterns
AdRetrieverUser-agent string
AdRetriever (https://adretriever.com)
Rated Oct 30, 2026 from public evidence
How we rated this bot
Verified by Cloudflare, which confirms who runs it, with no widespread reports of abuse.
Names itself honestly
Yes
Verified by Cloudflare
Yes
Says it follows robots.txt and opt-outs
Not stated
Ignores site rules
None found
Disguises itself or evades blocks
None found
Overloads sites
None found
How widely people complain
No complaint about this bot in the evidence
Reddit discussion
There is no Reddit discussion of this bot in the evidence
What we read
- google ignoring robots.txt ? - Google Search Central Community
- Crawlers not respecting robots.txt, need to block access.
- How to deal with AI crawlers that ignore robots.txt | Soups Ranjan posted on the topic | LinkedIn
- Robots.txt vs. AI Blockers: Managing Who Scrapes Your Content
- Meta Has New Web-Crawling Bots That Sneakily Get Around Robots.txt - Business Insider
Block or allow AdRetriever Site Walk
Block it
Add a Disallow rule for AdRetriever in your robots.txt file. You can also block at the server level using your web server configuration or CDN firewall rules to filter requests matching the user-agent string.
# Block AdRetriever Site Walk from crawling your entire site User-agent: AdRetriever Disallow: / # Allow AdRetriever Site Walk full access User-agent: AdRetriever Allow: /
Allow & verify
Ensure your robots.txt allows AdRetriever. Verify requests by checking the user-agent string.
Traffic & trends
Cloudflare Radar traffic
No Cloudflare traffic data available.
Google Trends interest
Loading trend data…
Links & references
Data sources
This profile is compiled from the following sources.
CloudflareRadar
Last updated October 30, 2026
Something wrong with this entry? Suggest a correction