yandexbot
GoodBotCommunity-sourcedThe primary web crawler for the Yandex search engine, responsible for indexing content for the Russian-based search platform. Operated by Yandex.
What is yandexbot?
YandexBot is the official automated crawler operated by Yandex, the largest search engine in Russia. It systematically traverses the web to discover, download, and index pages to ensure they appear in Yandex search results. The bot is highly configurable via the Yandex Webmaster tools, allowing site owners to control crawl frequency and indexation preferences.
Identification
Why it crawls your site
The bot crawls the internet to maintain an up-to-date index of web content, enabling Yandex to provide relevant search results to its users. It processes HTML, images, and various media formats to understand page structure and relevance. Site administrators can verify the bot's authenticity by performing a reverse DNS lookup on the requesting IP address to ensure it resolves to a yandex.ru or yandex.net domain.
IP ranges
Requests from yandexbot originate from these published ranges. For strict verification, match the source IP against this list in addition to checking the user-agent and reverse-DNS.
Block or allow yandexbot
Add a Disallow rule for yandexbot in your robots.txt file. You can also block at the server level using your web server configuration or CDN firewall rules to filter requests matching the user-agent string.
# Block yandexbot from crawling your entire site User-agent: yandexbot Disallow: / # Allow yandexbot full access User-agent: yandexbot Allow: /
Ensure your robots.txt allows yandexbot. Verify requests by checking the user-agent string.
Traffic & trends
No Cloudflare traffic data available.
Loading trend data…
From the community
Recent Reddit discussion mentioning this bot.
Data sources
This profile is compiled from the following sources.