all-ai-ips.json
GoodBotCommunity-sourcedA reference identifier for automated scripts or configuration files that aggregate and maintain lists of verified IP addresses used by AI and search engine crawlers.
What is all-ai-ips.json?
The 'all-ai-ips.json' identifier refers to automated processes or static files that consolidate official IP ranges published by major AI and search engine providers. These lists are maintained by developers and SEO professionals to ensure that legitimate crawlers from entities like OpenAI, Google, and Bing are correctly identified and permitted through firewall or WAF rules. By centralizing these endpoints, it allows for dynamic updates to server-side access control lists, preventing accidental blocking of essential search engine traffic.
Identification
Why it crawls your site
This entity does not crawl websites in the traditional sense; rather, it acts as a data source for infrastructure management. It retrieves official JSON endpoints provided by search engine and AI companies to keep local security configurations synchronized with the latest infrastructure changes. Verification is performed by cross-referencing the IP ranges contained within these files against the official documentation provided by the respective crawler operators.
IP ranges
Requests from all-ai-ips.json originate from these published ranges. For strict verification, match the source IP against this list in addition to checking the user-agent and reverse-DNS.
Block or allow all-ai-ips.json
Add a Disallow rule for all-ai-ips.json in your robots.txt file. You can also block at the server level using your web server configuration or CDN firewall rules to filter requests matching the user-agent string.
# Block all-ai-ips.json from crawling your entire site User-agent: all-ai-ips.json Disallow: / # Allow all-ai-ips.json full access User-agent: all-ai-ips.json Allow: /
Ensure your robots.txt allows all-ai-ips.json. Verify requests by checking the user-agent string.
Traffic & trends
No Cloudflare traffic data available.
Loading trend data…
Data sources
This profile is compiled from the following sources.