Applebot/0.1
UndecidedCommunity-sourcedApplebot is Apple's web crawler, designed to index and analyze web content to enhance various features within Apple's ecosystem, including Spotlight, Siri, and Safari. User-Agent Identification: Applebot identifies itself through specific user-agent strings, which include "Applebot" and other details. For example: Desktop: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 (Applebot/0.1; +http://www.apple.com/go/applebot) Mobile: Mozilla/5.0 (iPhone; CPU iPhone OS 17_4_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4.1 Mobile/15E148 Safari/604.1 (Applebot/0.1; +http://www.apple.com/go/applebot) Crawling Behavior: Applebot adheres to standard robots.txt directives, allowing website owners to control its access to their content. For instance, to restrict Applebot from crawling certain directories, a website's robots.txt file can include: Additionally, Applebot supports robots meta tags in HTML documents, enabling further control over indexing and snippet generation. For example, to prevent indexing of a page, the following meta tag can be used: By understanding and configuring these settings, website owners can effectively manage how Applebot interacts with their sites, ensuring optimal visibility and performance within Apple's search-related features. Operated by Apple Inc..
What is Applebot/0.1?
Applebot is the official web crawler operated by Apple to power search-related features across its ecosystem, including Safari, Spotlight, and Siri. Beyond traditional indexing, the bot is increasingly utilized to gather data for training Apple's generative AI models, such as those powering Apple Intelligence. Apple provides granular control for site owners, allowing them to distinguish between standard search indexing and AI training data collection via specific robots.txt directives.
Identification
Why it crawls your site
Applebot crawls the web to ensure that Apple's search and AI-driven services have access to up-to-date, relevant information. It functions by fetching web pages to build an index for discovery and to extract data for model training. Operators can verify the bot's authenticity by performing a reverse DNS lookup on the originating IP address to confirm it resolves to the applebot.apple.com domain or by checking the IP against Apple's published CIDR list.
Block or allow Applebot/0.1
Add a Disallow rule for Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1.1 Safari/605.1.15 (Applebot/0.1; +http://www.apple.com/go/applebot) in your robots.txt file. You can also block at the server level using your web server configuration or CDN firewall rules to filter requests matching the user-agent string.
# Block Applebot/0.1 from crawling your entire site User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1.1 Safari/605.1.15 (Applebot/0.1; +http://www.apple.com/go/applebot) Disallow: / # Allow Applebot/0.1 full access User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1.1 Safari/605.1.15 (Applebot/0.1; +http://www.apple.com/go/applebot) Allow: /
Ensure your robots.txt allows Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1.1 Safari/605.1.15 (Applebot/0.1; +http://www.apple.com/go/applebot). Verify requests by checking the user-agent string.
Traffic & trends
No Cloudflare traffic data available.
Loading trend data…
From the community
Recent Reddit discussion mentioning this bot.
Links & references
Data sources
This profile is compiled from the following sources.
Something wrong with this entry? Suggest a correction