Directory/Categories/SEO/OnCrawl

OnCrawl

Name: OnCrawl
Author: Oncrawl

GoodBotVerifiedBOT

Enterprise SEO platform powered by the industry-leading SEO Crawler and Log Analyzer Operated by Oncrawl.

Operator website ↗

GoodBot

Assessment

SEO

What is OnCrawl?

OnCrawl is a web crawler operated by Oncrawl. Enterprise SEO platform powered by the industry-leading SEO Crawler and Log Analyzer

Helpful — Verified, safe crawler. Respects robots.txt and provides operator documentation.

Identification

Bot name

OnCrawl

Operator

Oncrawl

Why it crawls your site

OnCrawl crawls websites to analyze search engine optimization factors and backlinks. It is operated by Oncrawl as part of their SEO analysis infrastructure. If you see this bot in your server logs, it is a verified crawler and generally safe.

Block or allow OnCrawl

Block it

Add a Disallow rule for www.oncrawl.com in your robots.txt file. You can also block at the server level using your web server configuration or CDN firewall rules to filter requests matching the user-agent string.

# Block OnCrawl from crawling your entire site
User-agent: www.oncrawl.com
Disallow: /

# Allow OnCrawl full access
User-agent: www.oncrawl.com
Allow: /

Allow & verify

Ensure your robots.txt allows www.oncrawl.com. Verify requests are genuine by checking the user-agent string and referring to Oncrawl's documentation.