Airwallex Website Scanner
GoodBotVerifiedBOTAn automated tool that collects information from publicly accessible pages on websites submitted to Airwallex. Operated by Airwallex.
What is Airwallex Website Scanner?
The Airwallex Website Scanner is a specialized automated agent operated by the financial technology company Airwallex. It is designed to crawl publicly accessible pages of websites submitted by customers to verify business legitimacy and assess potential risks. By analyzing site content, the scanner helps Airwallex fulfill its regulatory obligations and maintain the integrity of its financial services platform.
Identification
AirwallexWebsiteScannerHow we rated this bot
Verified by Cloudflare, which confirms who runs it, with no widespread reports of abuse.
What we read
- Respecting robots.txt in Web Scraping - DZone
- Beyond Robots.txt: Implementing AI.txt and LLMs.txt for Purpose-Based Scraping Control
- What is robots.txt and does it affect web scraping? - Openindex
- YandexBot: User Agent, robots.txt, and Why It Is Not an AI Crawler
- Meta Has New Web-Crawling Bots That Sneakily Get Around Robots.txt - Business Insider
Why it crawls your site
The scanner operates to gather data necessary for mandatory financial compliance and risk management processes. It visits the homepage and a limited number of related pages to confirm that a business's online presence aligns with the information provided during the onboarding process. Because it does not access private or credential-protected areas, it serves as a non-intrusive verification tool that can be identified by its specific IP address range and documented user-agent string.
Block or allow Airwallex Website Scanner
Add a Disallow rule for AirwallexWebsiteScanner in your robots.txt file. You can also block at the server level using your web server configuration or CDN firewall rules to filter requests matching the user-agent string.
# Block Airwallex Website Scanner from crawling your entire site User-agent: AirwallexWebsiteScanner Disallow: / # Allow Airwallex Website Scanner full access User-agent: AirwallexWebsiteScanner Allow: /
Ensure your robots.txt allows AirwallexWebsiteScanner. Verify requests by checking the user-agent string.
Traffic & trends
No Cloudflare traffic data available.
Loading trend data…
Links & references
Data sources
This profile is compiled from the following sources.
Something wrong with this entry? Suggest a correction