INSUBCONTINENT EXCLUSIVE:
For Cloudflare's plan to work, AI companies must sign up, too
However, while some AI companies may not see the incentive, Cloudflare has confirmed that it has partnered with AI companies on the
initiative, which may benefit from having a simple interface to negotiate with content creators.Cloudflare suggested its AI partners benefit
from "long-term collaboration" with creators whose updated content will help AI products stay relevant
They also can stop wasting money scraping poor quality data sources, a Cloudflare blog said."Without ongoing contributions from content
"Cloudflare is working with AI companies to give them more signals, and ultimately improve the quality and relevance of content they can
A healthy, sustainable ecosystem of original content is critical for AI innovation and relevance."However, Cloudflare's gamble seems to
depend on AI companies agreeing to pay the prices set by publishers, and that could potentially scramble the experiment if bidding wars
reduce rates to the point that they alienate publishers
It also hinges on Cloudflare detecting the AI bots, which, for now, relies on user reports and Cloudflare's analysis of mass traffic
emerge that reflects the true value of original content," Cloudflare said.Looking to the future, Cloudflare suggested that its pay-per-crawl
system would "evolve significantly." Perhaps one day publishers could use it to "charge different rates for different paths or content
types," potentially even introducing dynamic pricing in the AI scraping environment
In that future, Cloudflare predicted that AI companies would possibly be incentivized to create agents that would crawl the web, seeking the
best content deals to support specific AI products.
