speaker-0: Yeah.
speaker-1: What do you say to critics who argue that Cloudflare is moving from internet infrastructure into private rulemaking for
speaker-0: We as a company have a mission which is to help build a better internet and we take this very seriously.
speaker-1: If a bot cannot clearly say I'm here for search or I'm here for training or AI answers, Cloudflare may block it more strictly. Does that make life harder for smaller AI starters?
speaker-0: Yeah, we have lots of incentive and actually lots of focus on helping this long tail ecosystem emerge. So actually it's exactly the opposite.
speaker-1: Can a one person creator really make money from this or does it mostly work for large media?
speaker-0: We are super excited actually about long tail creators. And we have a bunch of local news that sits behind Cloudflare that are now making more money from licensing than they are from ads.
speaker-1: Giving publishers more data about AI traffic and referrals, what uncomfortable truth do you expect the data to show?
speaker-0: So information is power, and this is one of the main things that we were focused on.
speaker-1: Hi, I'm Kim Eisenberg, Super Intelligence Editor-in-Chief, and today I'm joined by Stephanie Cohen, Chief Strategy Officer at Cloudflare. Stephanie joined Cloudflare in 2024 after more than two decades at Goldman Sachs, where she also worked on strategy and led major global business areas. At Cloudflare, she focuses on the future of the network, AI, the developer platform. International growth and how the business model of the internet is actually changing. Stephanie, it is great to have you here today.
speaker-0: Thanks, Kim. It is wonderful to be here. Good to see you.
speaker-1: Great. The main question today is simple. ⁓ AI systems use content from the open web, but creators and publishers often do not see the traffic or payment in return, right? Cloudflare wants to give website owners more control over AI bots and create new ways for content owners to get paid. So let's start with the basic problem. If I publish an article or newsletter, an AI company can crawl it, learn from it, and answer people using my word. without sending me traffic or paying me, right? Is that the problem Cloveflair wants to solve?
speaker-0: So, about a year ago, we announced Content Independence Day. And the philosophy was quite simple: that you should have transparency and control over what bots and agents are accessing your site. We just had the one-year anniversary of Content Independence Day, and our philosophy for that was your content, your rules. We are now in a place where the traffic on the internet is more than 50%. It's actually almost 60% non-human. So the internet itself is fundamentally changing. And what that means. Is that every site, no matter who you are, what you do, you need to understand what is happening on your site. For the vast majority of companies, they want to optimize for AI. They want AI to ingest all of their information. Cloudflare is a good example. We want AI to take all of our developer docs, all of our blogs, everything that is on our website, because when you're deciding how to build, we want you to use Cloudflare. But for those companies that monetize via ads or subscriptions or affiliate revenue, they need a way to be discoverable without giving their content. content away for free. And that is what we're trying to help. We're trying to help all companies optimize and also those companies that need incremental control so they can figure out what their business model is in this new era.
speaker-1: Yeah, wow, sixty percent and crazy and crazy number, right? So but Cloudflare sits between many websites and the rest of the internet. What can Cloudflare do that a single creator or publisher cannot do alone?
speaker-0: Let's take a step back and what is Cloudflare? Cloudflare is a large programmable network. We're actually in more than 330 cities and more than 125 countries. And we have millions and millions of customers on the Cloudflare network. So there are a lot of things we can do, but I'll focus on two main things. One is speed and reliability. So this idea of performance. The idea that you want traffic to get through to you fast, reliably, and securely, and you want bad traffic to get deflected. Well, because we're in all of those locations, it doesn't matter where your audience is. It will show up and it will show up quickly and reliably. The second thing is around security. So because we have this breadth and depth of network, we see what's going on on the network and we see what's going on on the internet. So what we can do is make sure that we know which traffic you want to get through, and which traffic you would view as malicious. And the only way you can do that is if you have the breadth and depth of information that Cloudflare has.
speaker-1: You started this push around content independence day, correct? One year later, is this already a real turning point or is it still very early?
speaker-0: We see an ecosystem emerging. So the first thing that we tried to do about a year ago was make sure that people had transparency and that they had the tools to control. So this is just the idea of what who is trying to access my site as it relates to bots and agents, and then I can decide whether I want to block, allow, charge, optimize. And we see millions and millions of site owners, thousands and thousands of creators using those tools. And they're using those tools in many ways to create scarcity because There's no market without scarcity. And so we see companies, big and small, that are using these tools in order to create scarcity. And the signs that we have that this is working is we have a lot of big companies. People Inc. is a really good example of companies that are creating scarcity with their content and they're using that to do deals with AI companies. Neil Vogel is the CEO of People Inc. On his earnings call, it tells you how important it is. On his earnings call, he talked about the fact that he was using the Cloudflare tools and it was allowing him to negotiate better deals. With the AI companies. But it's not just the large publishers. We're also working with companies like Beehive that have independent creators on their platform so that each one of those creators can make decisions about what they want to block, allow, charge, optimize. And the last thing is we've started to partner with a bunch of AI search companies. Ceramic and U.com are another good example of an ecosystem that's emerging. This idea that if you're going to create search that is being used for AI, that sending money back to creators makes sense. And so ceramic, The deal we have with ceramic is they're paying per use for content behind the Cloudflare network. And the U partnership is about high quality, specific content, the type of content that you would normally see behind a paywall. So there's a lot of work we need to do, but we do see an ecosystem that's emerging.
speaker-1: Yeah, I love to hear because we're also, for example, using BI for our newsletter, so this is good to hear, right? So but last year the idea was paper crawl and AI bot pays when it requests a page. Now we're talking more about paper use. Paying when the content is actually used in an AI answer. Why is that actually better?
speaker-0: Our idea is we need to let a thousand flowers bloom. We need to figure out a bunch of different ways to get this to work because this is a new ecosystem. But it really is the difference between input and output. And I think you actually see this across the AI ecosystem, which is we're more and more output oriented. And paper use is more of an output. So what you're paying for is whether or not the information was used in the answer. The second thing is there'll likely be a convergence between crawling and usage. One of the things that we've been working On is how can we make crawling more efficient? So, because more than 20% of the internet sits behind Cloudflare, we have amazing signals about what's going on on the internet. And our data shows that more than half the time when AI is crawling the internet, they're crawling this something they've already crawled and it hasn't changed at all. And so we can use signals, we can use the data that we have inside the Cloudflare network to basically alert the crawler that something has changed, otherwise, no need to crawl again. And we just announced a partnership with OpenAI where we're doing research on this because more efficient crawling is great for the AI companies, it's great for the site owner, and it's also great for the world.
speaker-1: Fantastic. But who decides what a piece of content is worth? The publisher, the AI company, Cloudflare or the market?
speaker-0: Yeah, this is a market. This is about supply and demand. And so I'll talk to you a little bit about how something specifically works on the network. So In network protocols, there's something called a 402 response. A 402 response means payment required. And so it's not a block, it's a request for a payment. And because 402 is embedded basically inside of the internet, machines can use it to pay other machines. So the easiest way to think about how a market is going to develop, how prices will develop, is if you're a crawler, you can send out your bot and you could say this is the most I'm willing to pay, and then the site. That is going to respond with that 402 can have the minimum they're willing to accept. And then over time, those two markets will come together. Right now, we're in a place where the market is just is nascent. And so we're really only seeing negotiated deals happen rather than programmatic deals happen. But we think as there is more scarcity, as there is more volume, that there will be more programmatic deals. And if you want a sense for what the volume of demand is today on the Cloudflare Network, there's more than 2 billion 402 responses. Responses a day. The vast majority of those don't end in payments, but we can see that there's an ecosystem that wants to emerge. And by doing all of these protocols, things like 402, which we then created, the X402 Foundation with Coinbase and Stripe, but with many others, and now contribute it to the Linux Foundation, with all those things in place, I think you'll see more of a programmatic marketplace where the market decides the price.
speaker-1: Interesting. And Cloudflare has said that AI bots often crawl the same unchanged pages again and again and again. Can the new system reduce that waste or does it only charge for it?
speaker-0: Yeah, we have our mission is to help build a better internet. And so this inefficient crawling is just not good for anyone. And so the signals, this ability to provide insight to a crawler on whether or not the information has changed, is something that makes sense for all types of sites and all types of crawlers. And so we worked actually with a company, their name is Clay. They help you do research, they help you do go to market research. And you can imagine that they're constantly crawling websites to look at the management team. And see if the people are still in the same roles. Well, they don't need to keep crawling those same sites over and over and over again. So they can use Cloudflare and our signals product in order to decide whether or not to do that crawl, which saves them money, it's much more efficient, less load on the site. And so that's for content that it's probably not content that people are paying for today, but it still makes sense for it to be more efficient.
speaker-1: Yeah. ⁓ efficiency is always good. But from September fifteenth, twenty twenty six, new domains on cloud flareable block training and agent bots by default on pages with ads while search will stay allowed. Why make that the default?
speaker-0: Our goal, and what we've heard by the way, after having thousands and thousands of conversations, is that the sites behind Cloudflare they want to be discoverable without having to give their content away for free. And the reason we're changing the defaults, but not today, in about two months, two and a half months from when we made the announcement, is because we want to engage with the ecosystem, because the whole goal is that an ecosystem emerges. And so, as you said, the defaults will be set, and so this is for new sites for existing customers or for new customers. The defaults will be set in the signup flow. They can easily change them. We will also change the defaults for the free customers behind the Cloudflare network if they have not gone in and made any changes between the time of the announcement and when the defaults change. And as you noted, this is for pages with ads. Because what we learned from having all those conversations is that the vast majority of sites behind the Cloudflare network want to optimize for AI. They want their information inside of AI engines. It's the businesses that monetize with ads and subscriptions. And these are super important businesses because this is how we have diversity of content. This is how we have information in the world. And humans need high-quality, original information, but also AI needs it. So they need very specific defaults and tools.
speaker-1: Yeah. Let's step back a little bit. For people who are not deep in this topic because I wanna reach all the people that are curious and interested, but maybe not so deep in this topic. What is the difference between search changing and agent traffic?
speaker-0: Yeah, so the three things that we have, as you said, are search, training, and agent. And so think about search as indexing. Think about that as the ability to be discoverable. Think about that as AI knowing that you exist. And so when someone asks a question, you're an option because AI, the engine, actually knows that you exist. Training is how the models gain their intelligence. And of course, the information they get during training is important as it relates to the answer, but often the AI or the Agent will go out to the internet in order to get the latest information. And then the third thing that you need is agent. And this is where a human is generally asking an agent to do something. It could also be an agent asking another agent to do something, but they're taking an action. So you can understand why if you're a company that has ads or subscriptions, you'd want to be discoverable and therefore in search. But you'd want to make sure when a deeper answer is given, when an action is taken, that the human Actually, it goes to your site. By the way, there are businesses and companies that want their audience completely engaged in their environment. And so if they want their audience to engage in an AI experience, they want that to happen inside of their own environment. So they may tweak. By the way, there are companies that monetize with ads that may say, you know what, in addition to ads, I want to have licensing revenue. So I'm happy to allow these other things, but I want to get paid for it. And then there are a whole host of people, and we think they're being created today, who want to write. For the superintelligence for a for a whole host of reasons. And so while the defaults are important, the ability to change them is also important.
speaker-1: If a bot cannot clearly say I'm here for search or I'm here for training or AI answers, Cloudflare may block it more strictly. Does that make life harder for smaller AI startups? Because as you know, since the AI revolution, just immersion happened, startups they're growing on a on a huge scale, right?
speaker-0: Yeah, we have lots of incentive and actually lots of focus on helping this long-tail ecosystem emerge. So actually, it's exactly the opposite. And so if you look at AI companies that have been born recently, they actually tend to have different crawlers because they were created in the last couple of years. And so the idea of something between search and training and agent, it actually existed. And so their entire business model was actually built around separating those things. It's the companies that have been Around a lot longer that had different businesses that are now in the AI businesses, which tend to merge their crawlers. And the problem with that is that it makes it hard for companies and sites with ads to be discoverable without giving their content away for free. And so those sites, without having separate crawlers, don't have the same level of transparency. So they can't tell why the bot is accessing their site and they don't have the same level of control.
speaker-1: Obviously Google is the hard case here, right? Publishers need Google search traffic, but Google also has AI answers even more frequently nowadays. How do you handle that tension?
speaker-0: Yeah, we're really excited to use this two and a half month period to engage with the ecosystem. Because the goal around this, and by the way, the goal around all of this actually has never been blocking. Sometimes you have to use defense to create a good offense, but the goal has been to create a sustainable ecosystem. So the goal is to figure out what is the technical solution so that sites can decide to be discoverable. They can be discoverable inside of search, but they also don't have to give their content away for free.
speaker-1: Mm-hmm. Okay, now now a very probably difficult question, but I'm so so curious. Are the big AI companies actually willing to pay? Are companies like OpenAI, Google Anthropic Perplexity coming to the table? I mean you already mentioned them, right? But I mean it revolves around this question actually, right? Yeah.
speaker-0: Over the last year, we've seen more than 50 deals in the content space. And the reason for this is because you need high-quality original content for AI to flourish. If you think, if you want to do an analogy, think a little bit about a streaming businesses right now. Right now, the difference between streaming businesses is what content they have access to. And if we can imagine a world, I know it's not the world today, but if we can imagine a world where compute is not as scarce, where maybe some of the research talent is not as scarce, the The thing that will separate different AI companies is access to content. And so we see lots of engagement with the AI companies.
speaker-1: Interesting. What leverage do publishers really have? If an AI company says no, can it just get content somewhere else?
speaker-0: Of course, it depends on the type of content that you have. And so what we see is that if you have high quality original content, there is real demand for it. And once you create scarcity, then they will figure out a way to get access to it by paying for it. And so, example, there is a healthcare site that sits behind the Cloudflare network. We saw one bot try to go after information on that site more almost 800 million times in a 30 day period. And so if the question is whether or not there is demand For high-quality original content, of course, there is. ⁓ the content that is more me too content is going to be more difficult. For example, there have been articles that have been written that literally the only point of the article is to tell you when a big sporting event starts. Because people will often search for that. And then, of course, that article is plastered with ads all over the place. I think we can all agree that that's not adding to human intelligence. And so I think those business models are more challenged. But for content creators that are creating high-quality original Content for speak a specific voice, even if it's on content that's generally available, we think there are exciting business models for those content creators.
speaker-1: You're giving publishers more data about AI traffic and referrals. What uncomfortable truth do you expect the data to show?
speaker-0: Yeah, so information is power, and this is one of the main things that we were focused on. And so I know that the transformation can be difficult, but it's really important to actually have access to the information. And so what publishers are able to see is they're able to see who is accessing their site. They're certainly able to see who's giving them referrals, but it gives them really interesting and good ideas about what they should produce. And amazing conversations have come by just giving people information. So an example would be a large publisher. They condenas. Is a good example. There they have Vogue. They so if you think about if AI is accessing their site directly, it's getting access to the articles in the pictures, but it's not getting access to data behind that. But how interesting would it be to see all of the pictures that were taken for the photo shoot? Or knowing which type of customers tend to click on a specific photo or a specific article. And so what we've seen by giving content creators access to the information, there's a bunch of creativity that happens, which is how How do we take a business that was optimized for humans and now create a business that works for humans when humans come to my site, but also create a business that'll be optimized for machines?
speaker-1: Okay, exciting. let's change the focus a little bit because a lot of my audience are solo creators, newsletter writers, and small publishers. Can a one-person creator really make money from this or does it mostly work for large media companies?
speaker-0: We are super excited actually about long tail creators. And we have a bunch of local news that sits behind Cloudflare that are now making more money from licensing than they are from ads. And so for content creators that are out there, we think there are a ton of business models that work for this. And we mentioned the Beehive partnership earlier. We also are working with GoDaddy on something very similar. But I think by getting tools like this in the hands of individual creators, they're gonna come up with really cool, really interesting business models. I also think that. The human internet is not going away. I know I talked about the fact that more than you know almost 60% of the traffic on the Cloudflare network is now non-human, but that's not because humans are going away from the internet. It's because if you search for something as a human, you may go to five or ten sites, but if you're an agent, you're gonna go to thousands of sites. So, of course, the traffic from agents is gonna be more than the traffic from humans, but humans still want to read things from other humans, they want to participate in communities, and so we're quite bullish. about the individual creator community and what they're going to create with this new environment.
speaker-1: If someone runs runs a small website or newsletter, what is one thing they should do this week to stop giving their work away for free?
speaker-0: Sign up for Cloudflare. The ability to do that is not only available on Cloudflare, but it is available in our f on our free tier. And so there's a whole host of tools that you can also add to that. But you can AI crawl control available on Cloudflare, super easy to sign up, and you can start seeing what's happening on your site and then take control over what you want to block, allow ultimately if you decide to charge for that. So I would love for you to do that, and I would love to hear feedback on it.
speaker-1: Great. Let's ⁓ change the picture once again a little bit. So Cloudflare can now help decide which bots get access and which ones need to pay. Why should Cloudflare play that role?
speaker-0: The customers behind the Cloudflow network make the decision. They decide which bots they're blocking, allowing, and charging. The only thing that we automatically deflect is malicious traffic on the Cloudflare network and the more of the cybersecurity suite of our products. And so the idea is to put these type of tools in the hands of creators of all shapes and sizes.
speaker-1: Mm. And what do you say to critics who argue that Cloudflare is moving from internet infrastructure into private rule marking sorry, private rule making for the web?
speaker-0: We As a company, we have a mission, which is to help build a better internet. And we take this very seriously. And what that means is we have to engage with the broad ecosystem on how we think about how the internet should function. And at the end of the day, you need to choose defaults, but we do that by engaging with the broader community. And I think the example on the first of July is a really good example of us doing that. While we announced our intent to change the defaults, we didn't change the defaults immediately, we're changing them in a couple of months. And the idea is to continue the conversations that we've been having for over a year now to figure out what the right defaults are. But we also make it very easy for customers behind the Cloudflare network to change those defaults.
speaker-1: If AI answers replace clicks, what does a healthy open web look like five years from now? And I know five years is very far away, especially in the current time, but ⁓ maybe ⁓ one to three, even five years. What do you think?
speaker-0: The I was on a panel earlier today. ⁓ I think I mentioned that I meant that we are developers conference, and so The one of the things someone asked was about a moonshot. So I want to talk about how the internet functions, how payments function, and then like where we can go from here. So Cloudflare can handle more than 500 million transactions per second. The world's largest payment company can handle less than a hundred thousand transactions per second. In less than two seconds, we process more than they process in an entire day. But if you think about what's happening on the agentic internet, an internet that is predominantly agents, you can imagine Imagine a world where every interaction, every transaction, every request is a chance for commerce. But you need a payment system that matches that. So, in addition to all the things that we're talking about, we're working with a broader community. We were part recently of ⁓ Open Standard, which is a which is a stable coin that is coming out this idea of creating payment rails that will work at internet scale. And so I do think we're gonna get to a place which is some people view that when the original internet was created, that this is the One thing that we didn't get right. It's actually why that 402 protocol exists that I talked about earlier. It exists because the original idea of the internet was around micropayments. Micropayments would be super annoying for humans on the internet, but they're really, really easy for machines. And so I think we're gonna get to a place where a lot of business models emerge because this idea of just an interaction, a machine-to-machine interaction on the internet, could actually be a chance for payments, which opens the door to many, many new business models that didn't exist. Yeah.
speaker-1: Do you think the open web survives this change or are we moving toward more paywalls, license licensing seals and low quality AI content? What is your take on this?
speaker-0: The open internet is thriving and In the AI era, if you think about it, think about yourself. Think about if you went away, like locked yourself in a closet for a week, and then you came out and someone started asking you questions, you would be trying to do things without full context. And think about if it was much longer than that. And so, what we've seen in a world where agents are doing things, they need context. And that context happens from the open web. It also, distribution is now definitely down to basically zero. So if you're a contest Creator, the ability for you to create content and be out on the open web, high quality original content, that that is now a place where it's easier than ever. And so we're more excited than ever. There was a time when people talked about this with the that the web was in decline. We see exactly the opposite on the Cloudflare network. We see traffic increasing, we see the diversity of the internet increasing. Like we are very optimistic about what's happening on the open web.
speaker-1: Okay, this is very, very interesting. in the end I always ask the question if there is something less we haven't covered in the discussion, something you wanna let the community know about, something we haven't talked about. Is there anything left from your side?
speaker-0: The world, the internet has become agentic. And the internet has always had a business model. And so this agentic internet needs a business model. We're at the very beginnings of figuring out what that business model is, and we're super excited to partner and to test and iterate and discover what that new agentic internet is gonna look like.
speaker-1: Stephanie, this was so interesting. Fantastic talking to you. Thank you so, so much. Obviously, I highly recommend checking out all the informations. I'll post it down below on YouTube and all our social media. Stephanie, thank you very much. It was such a pleasure.
speaker-0: Such a pleasure. Thanks for having me. You too.
speaker-1: Have a great day.
speaker-1: What do you say to critics who argue that Cloudflare is moving from internet infrastructure into private rulemaking for
speaker-0: We as a company have a mission which is to help build a better internet and we take this very seriously.
speaker-1: If a bot cannot clearly say I'm here for search or I'm here for training or AI answers, Cloudflare may block it more strictly. Does that make life harder for smaller AI starters?
speaker-0: Yeah, we have lots of incentive and actually lots of focus on helping this long tail ecosystem emerge. So actually it's exactly the opposite.
speaker-1: Can a one person creator really make money from this or does it mostly work for large media?
speaker-0: We are super excited actually about long tail creators. And we have a bunch of local news that sits behind Cloudflare that are now making more money from licensing than they are from ads.
speaker-1: Giving publishers more data about AI traffic and referrals, what uncomfortable truth do you expect the data to show?
speaker-0: So information is power, and this is one of the main things that we were focused on.
speaker-1: Hi, I'm Kim Eisenberg, Super Intelligence Editor-in-Chief, and today I'm joined by Stephanie Cohen, Chief Strategy Officer at Cloudflare. Stephanie joined Cloudflare in 2024 after more than two decades at Goldman Sachs, where she also worked on strategy and led major global business areas. At Cloudflare, she focuses on the future of the network, AI, the developer platform. International growth and how the business model of the internet is actually changing. Stephanie, it is great to have you here today.
speaker-0: Thanks, Kim. It is wonderful to be here. Good to see you.
speaker-1: Great. The main question today is simple. ⁓ AI systems use content from the open web, but creators and publishers often do not see the traffic or payment in return, right? Cloudflare wants to give website owners more control over AI bots and create new ways for content owners to get paid. So let's start with the basic problem. If I publish an article or newsletter, an AI company can crawl it, learn from it, and answer people using my word. without sending me traffic or paying me, right? Is that the problem Cloveflair wants to solve?
speaker-0: So, about a year ago, we announced Content Independence Day. And the philosophy was quite simple: that you should have transparency and control over what bots and agents are accessing your site. We just had the one-year anniversary of Content Independence Day, and our philosophy for that was your content, your rules. We are now in a place where the traffic on the internet is more than 50%. It's actually almost 60% non-human. So the internet itself is fundamentally changing. And what that means. Is that every site, no matter who you are, what you do, you need to understand what is happening on your site. For the vast majority of companies, they want to optimize for AI. They want AI to ingest all of their information. Cloudflare is a good example. We want AI to take all of our developer docs, all of our blogs, everything that is on our website, because when you're deciding how to build, we want you to use Cloudflare. But for those companies that monetize via ads or subscriptions or affiliate revenue, they need a way to be discoverable without giving their content. content away for free. And that is what we're trying to help. We're trying to help all companies optimize and also those companies that need incremental control so they can figure out what their business model is in this new era.
speaker-1: Yeah, wow, sixty percent and crazy and crazy number, right? So but Cloudflare sits between many websites and the rest of the internet. What can Cloudflare do that a single creator or publisher cannot do alone?
speaker-0: Let's take a step back and what is Cloudflare? Cloudflare is a large programmable network. We're actually in more than 330 cities and more than 125 countries. And we have millions and millions of customers on the Cloudflare network. So there are a lot of things we can do, but I'll focus on two main things. One is speed and reliability. So this idea of performance. The idea that you want traffic to get through to you fast, reliably, and securely, and you want bad traffic to get deflected. Well, because we're in all of those locations, it doesn't matter where your audience is. It will show up and it will show up quickly and reliably. The second thing is around security. So because we have this breadth and depth of network, we see what's going on on the network and we see what's going on on the internet. So what we can do is make sure that we know which traffic you want to get through, and which traffic you would view as malicious. And the only way you can do that is if you have the breadth and depth of information that Cloudflare has.
speaker-1: You started this push around content independence day, correct? One year later, is this already a real turning point or is it still very early?
speaker-0: We see an ecosystem emerging. So the first thing that we tried to do about a year ago was make sure that people had transparency and that they had the tools to control. So this is just the idea of what who is trying to access my site as it relates to bots and agents, and then I can decide whether I want to block, allow, charge, optimize. And we see millions and millions of site owners, thousands and thousands of creators using those tools. And they're using those tools in many ways to create scarcity because There's no market without scarcity. And so we see companies, big and small, that are using these tools in order to create scarcity. And the signs that we have that this is working is we have a lot of big companies. People Inc. is a really good example of companies that are creating scarcity with their content and they're using that to do deals with AI companies. Neil Vogel is the CEO of People Inc. On his earnings call, it tells you how important it is. On his earnings call, he talked about the fact that he was using the Cloudflare tools and it was allowing him to negotiate better deals. With the AI companies. But it's not just the large publishers. We're also working with companies like Beehive that have independent creators on their platform so that each one of those creators can make decisions about what they want to block, allow, charge, optimize. And the last thing is we've started to partner with a bunch of AI search companies. Ceramic and U.com are another good example of an ecosystem that's emerging. This idea that if you're going to create search that is being used for AI, that sending money back to creators makes sense. And so ceramic, The deal we have with ceramic is they're paying per use for content behind the Cloudflare network. And the U partnership is about high quality, specific content, the type of content that you would normally see behind a paywall. So there's a lot of work we need to do, but we do see an ecosystem that's emerging.
speaker-1: Yeah, I love to hear because we're also, for example, using BI for our newsletter, so this is good to hear, right? So but last year the idea was paper crawl and AI bot pays when it requests a page. Now we're talking more about paper use. Paying when the content is actually used in an AI answer. Why is that actually better?
speaker-0: Our idea is we need to let a thousand flowers bloom. We need to figure out a bunch of different ways to get this to work because this is a new ecosystem. But it really is the difference between input and output. And I think you actually see this across the AI ecosystem, which is we're more and more output oriented. And paper use is more of an output. So what you're paying for is whether or not the information was used in the answer. The second thing is there'll likely be a convergence between crawling and usage. One of the things that we've been working On is how can we make crawling more efficient? So, because more than 20% of the internet sits behind Cloudflare, we have amazing signals about what's going on on the internet. And our data shows that more than half the time when AI is crawling the internet, they're crawling this something they've already crawled and it hasn't changed at all. And so we can use signals, we can use the data that we have inside the Cloudflare network to basically alert the crawler that something has changed, otherwise, no need to crawl again. And we just announced a partnership with OpenAI where we're doing research on this because more efficient crawling is great for the AI companies, it's great for the site owner, and it's also great for the world.
speaker-1: Fantastic. But who decides what a piece of content is worth? The publisher, the AI company, Cloudflare or the market?
speaker-0: Yeah, this is a market. This is about supply and demand. And so I'll talk to you a little bit about how something specifically works on the network. So In network protocols, there's something called a 402 response. A 402 response means payment required. And so it's not a block, it's a request for a payment. And because 402 is embedded basically inside of the internet, machines can use it to pay other machines. So the easiest way to think about how a market is going to develop, how prices will develop, is if you're a crawler, you can send out your bot and you could say this is the most I'm willing to pay, and then the site. That is going to respond with that 402 can have the minimum they're willing to accept. And then over time, those two markets will come together. Right now, we're in a place where the market is just is nascent. And so we're really only seeing negotiated deals happen rather than programmatic deals happen. But we think as there is more scarcity, as there is more volume, that there will be more programmatic deals. And if you want a sense for what the volume of demand is today on the Cloudflare Network, there's more than 2 billion 402 responses. Responses a day. The vast majority of those don't end in payments, but we can see that there's an ecosystem that wants to emerge. And by doing all of these protocols, things like 402, which we then created, the X402 Foundation with Coinbase and Stripe, but with many others, and now contribute it to the Linux Foundation, with all those things in place, I think you'll see more of a programmatic marketplace where the market decides the price.
speaker-1: Interesting. And Cloudflare has said that AI bots often crawl the same unchanged pages again and again and again. Can the new system reduce that waste or does it only charge for it?
speaker-0: Yeah, we have our mission is to help build a better internet. And so this inefficient crawling is just not good for anyone. And so the signals, this ability to provide insight to a crawler on whether or not the information has changed, is something that makes sense for all types of sites and all types of crawlers. And so we worked actually with a company, their name is Clay. They help you do research, they help you do go to market research. And you can imagine that they're constantly crawling websites to look at the management team. And see if the people are still in the same roles. Well, they don't need to keep crawling those same sites over and over and over again. So they can use Cloudflare and our signals product in order to decide whether or not to do that crawl, which saves them money, it's much more efficient, less load on the site. And so that's for content that it's probably not content that people are paying for today, but it still makes sense for it to be more efficient.
speaker-1: Yeah. ⁓ efficiency is always good. But from September fifteenth, twenty twenty six, new domains on cloud flareable block training and agent bots by default on pages with ads while search will stay allowed. Why make that the default?
speaker-0: Our goal, and what we've heard by the way, after having thousands and thousands of conversations, is that the sites behind Cloudflare they want to be discoverable without having to give their content away for free. And the reason we're changing the defaults, but not today, in about two months, two and a half months from when we made the announcement, is because we want to engage with the ecosystem, because the whole goal is that an ecosystem emerges. And so, as you said, the defaults will be set, and so this is for new sites for existing customers or for new customers. The defaults will be set in the signup flow. They can easily change them. We will also change the defaults for the free customers behind the Cloudflare network if they have not gone in and made any changes between the time of the announcement and when the defaults change. And as you noted, this is for pages with ads. Because what we learned from having all those conversations is that the vast majority of sites behind the Cloudflare network want to optimize for AI. They want their information inside of AI engines. It's the businesses that monetize with ads and subscriptions. And these are super important businesses because this is how we have diversity of content. This is how we have information in the world. And humans need high-quality, original information, but also AI needs it. So they need very specific defaults and tools.
speaker-1: Yeah. Let's step back a little bit. For people who are not deep in this topic because I wanna reach all the people that are curious and interested, but maybe not so deep in this topic. What is the difference between search changing and agent traffic?
speaker-0: Yeah, so the three things that we have, as you said, are search, training, and agent. And so think about search as indexing. Think about that as the ability to be discoverable. Think about that as AI knowing that you exist. And so when someone asks a question, you're an option because AI, the engine, actually knows that you exist. Training is how the models gain their intelligence. And of course, the information they get during training is important as it relates to the answer, but often the AI or the Agent will go out to the internet in order to get the latest information. And then the third thing that you need is agent. And this is where a human is generally asking an agent to do something. It could also be an agent asking another agent to do something, but they're taking an action. So you can understand why if you're a company that has ads or subscriptions, you'd want to be discoverable and therefore in search. But you'd want to make sure when a deeper answer is given, when an action is taken, that the human Actually, it goes to your site. By the way, there are businesses and companies that want their audience completely engaged in their environment. And so if they want their audience to engage in an AI experience, they want that to happen inside of their own environment. So they may tweak. By the way, there are companies that monetize with ads that may say, you know what, in addition to ads, I want to have licensing revenue. So I'm happy to allow these other things, but I want to get paid for it. And then there are a whole host of people, and we think they're being created today, who want to write. For the superintelligence for a for a whole host of reasons. And so while the defaults are important, the ability to change them is also important.
speaker-1: If a bot cannot clearly say I'm here for search or I'm here for training or AI answers, Cloudflare may block it more strictly. Does that make life harder for smaller AI startups? Because as you know, since the AI revolution, just immersion happened, startups they're growing on a on a huge scale, right?
speaker-0: Yeah, we have lots of incentive and actually lots of focus on helping this long-tail ecosystem emerge. So actually, it's exactly the opposite. And so if you look at AI companies that have been born recently, they actually tend to have different crawlers because they were created in the last couple of years. And so the idea of something between search and training and agent, it actually existed. And so their entire business model was actually built around separating those things. It's the companies that have been Around a lot longer that had different businesses that are now in the AI businesses, which tend to merge their crawlers. And the problem with that is that it makes it hard for companies and sites with ads to be discoverable without giving their content away for free. And so those sites, without having separate crawlers, don't have the same level of transparency. So they can't tell why the bot is accessing their site and they don't have the same level of control.
speaker-1: Obviously Google is the hard case here, right? Publishers need Google search traffic, but Google also has AI answers even more frequently nowadays. How do you handle that tension?
speaker-0: Yeah, we're really excited to use this two and a half month period to engage with the ecosystem. Because the goal around this, and by the way, the goal around all of this actually has never been blocking. Sometimes you have to use defense to create a good offense, but the goal has been to create a sustainable ecosystem. So the goal is to figure out what is the technical solution so that sites can decide to be discoverable. They can be discoverable inside of search, but they also don't have to give their content away for free.
speaker-1: Mm-hmm. Okay, now now a very probably difficult question, but I'm so so curious. Are the big AI companies actually willing to pay? Are companies like OpenAI, Google Anthropic Perplexity coming to the table? I mean you already mentioned them, right? But I mean it revolves around this question actually, right? Yeah.
speaker-0: Over the last year, we've seen more than 50 deals in the content space. And the reason for this is because you need high-quality original content for AI to flourish. If you think, if you want to do an analogy, think a little bit about a streaming businesses right now. Right now, the difference between streaming businesses is what content they have access to. And if we can imagine a world, I know it's not the world today, but if we can imagine a world where compute is not as scarce, where maybe some of the research talent is not as scarce, the The thing that will separate different AI companies is access to content. And so we see lots of engagement with the AI companies.
speaker-1: Interesting. What leverage do publishers really have? If an AI company says no, can it just get content somewhere else?
speaker-0: Of course, it depends on the type of content that you have. And so what we see is that if you have high quality original content, there is real demand for it. And once you create scarcity, then they will figure out a way to get access to it by paying for it. And so, example, there is a healthcare site that sits behind the Cloudflare network. We saw one bot try to go after information on that site more almost 800 million times in a 30 day period. And so if the question is whether or not there is demand For high-quality original content, of course, there is. ⁓ the content that is more me too content is going to be more difficult. For example, there have been articles that have been written that literally the only point of the article is to tell you when a big sporting event starts. Because people will often search for that. And then, of course, that article is plastered with ads all over the place. I think we can all agree that that's not adding to human intelligence. And so I think those business models are more challenged. But for content creators that are creating high-quality original Content for speak a specific voice, even if it's on content that's generally available, we think there are exciting business models for those content creators.
speaker-1: You're giving publishers more data about AI traffic and referrals. What uncomfortable truth do you expect the data to show?
speaker-0: Yeah, so information is power, and this is one of the main things that we were focused on. And so I know that the transformation can be difficult, but it's really important to actually have access to the information. And so what publishers are able to see is they're able to see who is accessing their site. They're certainly able to see who's giving them referrals, but it gives them really interesting and good ideas about what they should produce. And amazing conversations have come by just giving people information. So an example would be a large publisher. They condenas. Is a good example. There they have Vogue. They so if you think about if AI is accessing their site directly, it's getting access to the articles in the pictures, but it's not getting access to data behind that. But how interesting would it be to see all of the pictures that were taken for the photo shoot? Or knowing which type of customers tend to click on a specific photo or a specific article. And so what we've seen by giving content creators access to the information, there's a bunch of creativity that happens, which is how How do we take a business that was optimized for humans and now create a business that works for humans when humans come to my site, but also create a business that'll be optimized for machines?
speaker-1: Okay, exciting. let's change the focus a little bit because a lot of my audience are solo creators, newsletter writers, and small publishers. Can a one-person creator really make money from this or does it mostly work for large media companies?
speaker-0: We are super excited actually about long tail creators. And we have a bunch of local news that sits behind Cloudflare that are now making more money from licensing than they are from ads. And so for content creators that are out there, we think there are a ton of business models that work for this. And we mentioned the Beehive partnership earlier. We also are working with GoDaddy on something very similar. But I think by getting tools like this in the hands of individual creators, they're gonna come up with really cool, really interesting business models. I also think that. The human internet is not going away. I know I talked about the fact that more than you know almost 60% of the traffic on the Cloudflare network is now non-human, but that's not because humans are going away from the internet. It's because if you search for something as a human, you may go to five or ten sites, but if you're an agent, you're gonna go to thousands of sites. So, of course, the traffic from agents is gonna be more than the traffic from humans, but humans still want to read things from other humans, they want to participate in communities, and so we're quite bullish. about the individual creator community and what they're going to create with this new environment.
speaker-1: If someone runs runs a small website or newsletter, what is one thing they should do this week to stop giving their work away for free?
speaker-0: Sign up for Cloudflare. The ability to do that is not only available on Cloudflare, but it is available in our f on our free tier. And so there's a whole host of tools that you can also add to that. But you can AI crawl control available on Cloudflare, super easy to sign up, and you can start seeing what's happening on your site and then take control over what you want to block, allow ultimately if you decide to charge for that. So I would love for you to do that, and I would love to hear feedback on it.
speaker-1: Great. Let's ⁓ change the picture once again a little bit. So Cloudflare can now help decide which bots get access and which ones need to pay. Why should Cloudflare play that role?
speaker-0: The customers behind the Cloudflow network make the decision. They decide which bots they're blocking, allowing, and charging. The only thing that we automatically deflect is malicious traffic on the Cloudflare network and the more of the cybersecurity suite of our products. And so the idea is to put these type of tools in the hands of creators of all shapes and sizes.
speaker-1: Mm. And what do you say to critics who argue that Cloudflare is moving from internet infrastructure into private rule marking sorry, private rule making for the web?
speaker-0: We As a company, we have a mission, which is to help build a better internet. And we take this very seriously. And what that means is we have to engage with the broad ecosystem on how we think about how the internet should function. And at the end of the day, you need to choose defaults, but we do that by engaging with the broader community. And I think the example on the first of July is a really good example of us doing that. While we announced our intent to change the defaults, we didn't change the defaults immediately, we're changing them in a couple of months. And the idea is to continue the conversations that we've been having for over a year now to figure out what the right defaults are. But we also make it very easy for customers behind the Cloudflare network to change those defaults.
speaker-1: If AI answers replace clicks, what does a healthy open web look like five years from now? And I know five years is very far away, especially in the current time, but ⁓ maybe ⁓ one to three, even five years. What do you think?
speaker-0: The I was on a panel earlier today. ⁓ I think I mentioned that I meant that we are developers conference, and so The one of the things someone asked was about a moonshot. So I want to talk about how the internet functions, how payments function, and then like where we can go from here. So Cloudflare can handle more than 500 million transactions per second. The world's largest payment company can handle less than a hundred thousand transactions per second. In less than two seconds, we process more than they process in an entire day. But if you think about what's happening on the agentic internet, an internet that is predominantly agents, you can imagine Imagine a world where every interaction, every transaction, every request is a chance for commerce. But you need a payment system that matches that. So, in addition to all the things that we're talking about, we're working with a broader community. We were part recently of ⁓ Open Standard, which is a which is a stable coin that is coming out this idea of creating payment rails that will work at internet scale. And so I do think we're gonna get to a place which is some people view that when the original internet was created, that this is the One thing that we didn't get right. It's actually why that 402 protocol exists that I talked about earlier. It exists because the original idea of the internet was around micropayments. Micropayments would be super annoying for humans on the internet, but they're really, really easy for machines. And so I think we're gonna get to a place where a lot of business models emerge because this idea of just an interaction, a machine-to-machine interaction on the internet, could actually be a chance for payments, which opens the door to many, many new business models that didn't exist. Yeah.
speaker-1: Do you think the open web survives this change or are we moving toward more paywalls, license licensing seals and low quality AI content? What is your take on this?
speaker-0: The open internet is thriving and In the AI era, if you think about it, think about yourself. Think about if you went away, like locked yourself in a closet for a week, and then you came out and someone started asking you questions, you would be trying to do things without full context. And think about if it was much longer than that. And so, what we've seen in a world where agents are doing things, they need context. And that context happens from the open web. It also, distribution is now definitely down to basically zero. So if you're a contest Creator, the ability for you to create content and be out on the open web, high quality original content, that that is now a place where it's easier than ever. And so we're more excited than ever. There was a time when people talked about this with the that the web was in decline. We see exactly the opposite on the Cloudflare network. We see traffic increasing, we see the diversity of the internet increasing. Like we are very optimistic about what's happening on the open web.
speaker-1: Okay, this is very, very interesting. in the end I always ask the question if there is something less we haven't covered in the discussion, something you wanna let the community know about, something we haven't talked about. Is there anything left from your side?
speaker-0: The world, the internet has become agentic. And the internet has always had a business model. And so this agentic internet needs a business model. We're at the very beginnings of figuring out what that business model is, and we're super excited to partner and to test and iterate and discover what that new agentic internet is gonna look like.
speaker-1: Stephanie, this was so interesting. Fantastic talking to you. Thank you so, so much. Obviously, I highly recommend checking out all the informations. I'll post it down below on YouTube and all our social media. Stephanie, thank you very much. It was such a pleasure.
speaker-0: Such a pleasure. Thanks for having me. You too.
speaker-1: Have a great day.
