Internet.nl
GoodBotVerifiedBOTInternet.nl is a test tool for modern interne Operated by Internet Standards Platform.
What is Internet.nl?
Internet.nl is a collaborative initiative by the Dutch Internet Standards Platform, involving government and industry stakeholders like SIDN, RIPE NCC, and the NCSC-NL. The bot performs automated technical audits of websites, email servers, and connection configurations to verify the implementation of modern security protocols such as DNSSEC, TLS, and IPv6. By providing transparent, actionable reports, it encourages the adoption of best practices to make the internet more resilient and secure.
Identification
internetnlHow we rated this bot
Verified by Cloudflare, which confirms who runs it, with no widespread reports of abuse.
What we read
- I wonder if it will actually respect the robots.txt this time. - Hacker News
- Crawler without limits: Perplexity ignores robots.txt | heise online
- Why are Bing and SEMRushBot ignoring crawl-delay in my robots.txt ...
- google ignoring robots.txt ? - Google Search Central Community
- Meta Has New Web-Crawling Bots That Sneakily Get Around Robots.txt - Business Insider
Why it crawls your site
The bot crawls specific domains at the request of users or as part of periodic platform-wide assessments to generate a 'scorecard' for internet standards. It analyzes the technical headers, DNS records, and encryption configurations of a target site to identify vulnerabilities or outdated protocols. Site owners can verify the operator by visiting the provided URL in the User-Agent string, which details the platform's mission and the participating organizations behind the project.
Block or allow Internet.nl
Add a Disallow rule for internetnl in your robots.txt file. You can also block at the server level using your web server configuration or CDN firewall rules to filter requests matching the user-agent string.
# Block Internet.nl from crawling your entire site User-agent: internetnl Disallow: / # Allow Internet.nl full access User-agent: internetnl Allow: /
Ensure your robots.txt allows internetnl. Verify requests by checking the user-agent string.
Traffic & trends
No Cloudflare traffic data available.
Loading trend data…
From the community
Recent Reddit discussion mentioning this bot.
Links & references
Data sources
This profile is compiled from the following sources.
Something wrong with this entry? Suggest a correction