duckduckbot
GoodBotCommunity-sourcedThe primary web crawler used by DuckDuckGo to index content for its privacy-focused search engine. Operated by DuckDuckGo.
What is duckduckbot?
DuckDuckBot is the proprietary web crawler operated by DuckDuckGo, designed to discover and index web pages to populate the search engine's results. Unlike many other crawlers, it is built with a focus on privacy, ensuring that the crawling process does not track individual user behavior. It follows standard web protocols, including adherence to robots.txt files, to ensure site owners maintain control over how their content is indexed.
Identification
Why it crawls your site
The bot crawls the web to maintain an up-to-date index of content, allowing DuckDuckGo to provide relevant search results to its users. By visiting sites, it parses HTML and metadata to understand page context and ranking signals. Site owners can verify the bot by performing a reverse DNS lookup on the requesting IP address, which should resolve to a domain ending in duckduckgo.com.
IP ranges
Requests from duckduckbot originate from these published ranges. For strict verification, match the source IP against this list in addition to checking the user-agent and reverse-DNS.
Block or allow duckduckbot
Add a Disallow rule for duckduckbot in your robots.txt file. You can also block at the server level using your web server configuration or CDN firewall rules to filter requests matching the user-agent string.
# Block duckduckbot from crawling your entire site User-agent: duckduckbot Disallow: / # Allow duckduckbot full access User-agent: duckduckbot Allow: /
Ensure your robots.txt allows duckduckbot. Verify requests by checking the user-agent string.
Traffic & trends
No Cloudflare traffic data available.
Loading trend data…
From the community
Recent Reddit discussion mentioning this bot.
Related bots
Other crawlers operated by DuckDuckGo.
Data sources
This profile is compiled from the following sources.