payu

The 'payu' bot is a component of Cloudflare's Pay-per-Crawl initiative, designed to provide website owners with a technical framework to charge AI companies for accessing their content. Rather than acting as a traditional scraper, it serves as a signaling agent within the Cloudflare ecosystem to verify that a crawler has agreed to the site owner's terms or payment requirements. This system shifts the dynamic from a binary 'allow or block' model to a transactional one, allowing publishers to set specific costs for the data ingested by AI models.

This bot does not crawl for content indexing or model training purposes; instead, it functions as a verification and enforcement layer. It crawls to validate that incoming automated traffic complies with the financial or access policies defined by the site administrator via Cloudflare's dashboard. Operators can verify this bot by checking for Cloudflare-signed headers and ensuring the traffic originates from Cloudflare's verified infrastructure, which confirms the bot is acting on behalf of the site owner's configured security policies.