Cloudflare is introducing pay-per-crawl for websites that wish to monetize AI crawlers scraping their content. The idea of charging for website access isn't new - there was a lot of buzz in this area back when micro-transactions were tried during the crypto craze. But now we have a reputable company in Cloudflare offering to serve as the middle-man between publishers and large tech companies with budgets in the hundreds of billions. Though pay-per-crawl is still in closed beta, it's not hard to imagine the fundamental structure of the internet could change in the future, if websites migrate from being largely supported by ads to being supported by crawler fees. And who would stand to benefit the most from such a dramatic shift? The one operating the turnstiles, Cloudflare. https://blog.cloudflare.com/introducing-pay...