OAI-SearchBot/1.0

UndecidedCommunity-sourced

OAI-SearchBot/1.0 is a web crawler developed by OpenAI to index websites and enhance the visibility of their links within OpenAI's SearchGPT prototype. Unlike other crawlers that gather data for training AI models, OAI-SearchBot focuses solely on surfacing website links in search results. ( shahnoorblogger.com ) The user-agent string for OAI-SearchBot is: To control OAI-SearchBot's access to your website, you can configure your robots.txt file accordingly. For example, to allow OAI-SearchBot to crawl your entire site, you would include: Conversely, to disallow it, you would use: By managing OAI-SearchBot's access, you can influence how your website appears in OpenAI's search results. ( shahnoorblogger.com )

Operator website ↗

Undecided

Assessment

Uncategorized

Identification

Bot name

OAI-SearchBot/1.0

IP ranges

Requests from OAI-SearchBot/1.0 originate from these published ranges. For strict verification, match the source IP against this list in addition to checking the user-agent and reverse-DNS.

Showing 24 of 461 ranges

Block or allow OAI-SearchBot/1.0

Block it

Add a Disallow rule for Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot in your robots.txt file. You can also block at the server level using your web server configuration or CDN firewall rules to filter requests matching the user-agent string.

# Block OAI-SearchBot/1.0 from crawling your entire site
User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot
Disallow: /

# Allow OAI-SearchBot/1.0 full access
User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot
Allow: /

Allow & verify

Ensure your robots.txt allows Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot. Verify requests by checking the user-agent string.