Cloudflare CEO Matthew Prince said the company has blocked more than 416 billion AI bot requests since making it the default option in July of this year when it announced its Content Independence Day initiative. In an interview with Wired, Prince said the feature would allow website owners to block AI crawlers by default unless the AI company pays for access to their content.
“The business model of the Internet has always been to generate content that drives traffic and then sell either goods, subscriptions, or advertising. But what people don’t realize is that AI is a platform change,” Prince told Wired. The Internet business model is about to change dramatically. I don’t know what it will turn into, but I spend almost every waking moment thinking about it. ”
Although Cloudflare blocks almost all AI crawlers, there is one particular bot that cannot be blocked without impacting a customer’s online presence. That’s Google. The search giant has combined its search and AI crawlers into one, so users who opt out of Google’s AI crawler will no longer be indexed in Google’s search results. “You can’t opt out of one without opting out of both. That’s really hard. It’s crazy,” Prince continued. “Yesterday’s monopoly position cannot be used to obtain a monopoly position in tomorrow’s market.”
you may like
Human-generated content is critical for AI companies to train their models, as studies have shown that training on AI-generated data tilts AI models. While AI Overview has been proven to reduce website traffic, particularly impacting websites that rely heavily on visibility and views for advertising revenue, licensing agreements can help offset this and help online publications remain a viable source of income for creators and publishers.
Cloudflare will also benefit from a diverse internet hosting content from real humans. The CEO told Wired that the company is aiming for a future where creators and businesses thrive on a level playing field. That means more websites needing protection, which means more potential customers for Cloudflare. This makes the company one of the world’s largest content distribution networks, accounting for 79.9% of the market as of 2022. But sometimes the Internet is vulnerable, as in November, when a single misconfigured file destroyed huge swathes of the web.
This highlights the current problem with the global web infrastructure, which relies on just a few large companies such as AWS, Azure, Cloudflare, CrowdStrike, and Google to serve the entire planet. These agencies have made life easier and more streamlined for businesses that rely on the Internet, but at the same time, an outage of just one of these services means billions of dollars in losses and severe disruption around the world.
Follow Tom’s Hardware on Google News or add us as your preferred source to get the latest news, analysis, and reviews in your feed.
Get the best Tom’s Hardware news and in-depth reviews straight to your inbox.
