Directory/Categories/AI Assistant/Known Agent

Known Agent

Name: Known Agent
Author: Known Agents

GoodBotVerifiedAGENT

Known Agents helps website owners track, control, and optimize traffic from AI agents and bots. Operated by Known Agents.

Operator website ↗

GoodBot

Assessment

AI Assistant

What is Known Agent?

Known Agent is a web crawler operated by Known Agents. Known Agents helps website owners track, control, and optimize traffic from AI agents and bots.

Helpful — Verified, safe crawler. Respects robots.txt and provides operator documentation.

Identification

Bot name

Known Agent

Operator

Known Agents

Why it crawls your site

Known Agent crawls websites to collect and process web content. It is operated by Known Agents as part of their web crawling infrastructure. If you see this bot in your server logs, it is a verified crawler and generally safe.

Block or allow Known Agent

Block it

Add a Disallow rule for Known Agent in your robots.txt file. You can also block at the server level using your web server configuration or CDN firewall rules to filter requests matching the user-agent string.

# Block Known Agent from crawling your entire site
User-agent: Known Agent
Disallow: /

# Allow Known Agent full access
User-agent: Known Agent
Allow: /

Allow & verify

Ensure your robots.txt allows Known Agent. Verify requests are genuine by checking the user-agent string and referring to Known Agents's documentation.