Tumblr
GoodBotVerifiedBOTOn Tumblr, post authors can paste a URL in their post, and we'll "unfurl" that URL into a pretty Link "Block" for their post by making a request to the URL and parsing the response. Operated by Automattic.
What is Tumblr?
Tumblr is a web crawler operated by Automattic. On Tumblr, post authors can paste a URL in their post, and we'll "unfurl" that URL into a pretty Link "Block" for their post by making a request to the URL and parsing the response.
Identification
Tumblr/How we rated this bot
Verified by Cloudflare, which confirms who runs it, with no widespread reports of abuse.
What we read
- Is it possible to override the robots.txt file on a Tumblr blog? - Quora
- Crawlers not respecting robots.txt, need to block access. - Reddit
- I wonder if it will actually respect the robots.txt this time. - Hacker News
- google ignoring robots.txt ? - Google Search Central Community
- What Is Robots.txt? | How a Robots.txt File Works - Cloudflare
Why it crawls your site
Tumblr crawls websites to generate link previews and thumbnails for shared URLs. It is operated by Automattic as part of their link preview infrastructure. If you see this bot in your server logs, it is a verified crawler and generally safe.
Block or allow Tumblr
Add a Disallow rule for Tumblr/ in your robots.txt file. You can also block at the server level using your web server configuration or CDN firewall rules to filter requests matching the user-agent string.
# Block Tumblr from crawling your entire site User-agent: Tumblr/ Disallow: / # Allow Tumblr full access User-agent: Tumblr/ Allow: /
Ensure your robots.txt allows Tumblr/. Verify requests are genuine by checking the user-agent string and referring to Automattic's documentation.
Traffic & trends
No Cloudflare traffic data available.
Loading trend data…
From the community
Recent Reddit discussion mentioning this bot.
Related bots
Other crawlers operated by Automattic.
Links & references
Data sources
This profile is compiled from the following sources.
Something wrong with this entry? Suggest a correction