Patreon stops asking AI bots not to scrape, starts blocking them with Cloudflare

Patreon has stopped asking AI training bots to obey its robots.txt file and is now actively blocking them through a partnership with Cloudflare. The shift addresses growing sophistication in AI scraping and protects creator content from unauthorized training.

What happened

Patreon uses Cloudflare's AI Crawl Control technology to enforce its policies, moving beyond the voluntary compliance model of robots.txt. The company notes that AI scraping had become more sophisticated since it first put measures in place in 2023. A paywall previously limited crawler access, but newly introduced discovery tools like a redesigned Home Feed and tweet-like Quips could expose more content. During testing, weekly AI-training crawler attempts dropped from thousands to zero, indicating that scrapers were ignoring robots.txt directives. Patreon will allow bots that index pages and organize information if they send users back to Patreon.

Why AI builders should care

This is a signal that enforceable AI training data protections are becoming practical for platforms with paywalled or creator-owned content. Cloudflare's broader tools, including a Pay Per Crawl marketplace and default blocking of mixed-use crawlers on ad-hosted pages, offer models for other publishers. For AI builders, this means that scraping high-quality, gated content will face infrastructure-level barriers, not just polite requests. Product teams building AI training pipelines should anticipate that platforms will adopt similar enforcement, shrinking the pool of easily scraped creator data.

Practical implications

Developers and product teams should consider how API-level access controls or Cloudflare-style enforcement can protect training data in their own platforms. If you are building AI crawlers, distinguish clearly between indexing for search (which Patreon allows) and training (which it blocks). Publishers can combine AI-specific controls with existing access restrictions to reduce unwanted ingestion while keeping legitimate search traffic. Patreon's product chief Drew Rowny stated that "consent shouldn't depend on whether a scraper chooses to behave", highlighting a hardening stance across the creator economy.

Caveats

The reported enforcement results come from Patreon's own testing and media coverage. Actual effectiveness may vary as crawlers adapt. The distinction between allowed indexing bots and blocked training bots depends on Patreon's ongoing policy enforcement. Additionally, not all of Cloudflare's Pay Per Crawl or mixed-use crawler policies are detailed in the source context; their broader application may evolve.