Latest / Best of Business / Stephanie Cohen: Cloudflare chief strategy officer on the company's new strategy for AI scrapers

Transcript

Stephanie Cohen: Cloudflare chief strategy officer on the company's new strategy for AI scrapers · Best of Business

• 0:09
  Speaker 1: You're listening to a podcast from News Talks, it'd be
• 0:12
  Speaker 1: follow this and our wide range of podcasts now on iHeartRadio.
• 0:16
  Speaker 2: There are increasing moves to stop AI companies from scraping
• 0:19
  Speaker 2: websites for free, which is what's been going on. Cloud
• 0:22
  Speaker 2: Fair is a website infrastructure company that looks after about
• 0:25
  Speaker 2: sixteen percent of the global Internet traffic. It has decided
• 0:29
  Speaker 2: to block the AI crawlers as a default. Steffanie Cohen
• 0:32
  Speaker 2: is cloud Fair's chief strategy officer and with US high Stephanie.
• 0:36
  Speaker 3: Hi lovely to be here.
• 0:37
  Speaker 2: It's great to chat to you. What proportion of your
• 0:40
  Speaker 2: sites that you guys are in charge of do you
• 0:42
  Speaker 2: think want to keep AI crawlers off?
• 0:46
  Speaker 3: Yeah? I think what if you look at cloud Flair's
• 0:50
  Speaker 3: Internet presence a cloud flare, more than twenty percent of
• 0:52
  Speaker 3: the Internet sits behind cloud Flair, And what we know
• 0:55
  Speaker 3: about the people that sit behind cloud flair is they
• 0:56
  Speaker 3: want control over who is scraping their site. They want
• 0:59
  Speaker 3: to know who's on their site, they want to know
• 1:01
  Speaker 3: what they're planning to do on that site, and then
• 1:03
  Speaker 3: they want to have the option to either block, allow,
• 1:06
  Speaker 3: or charge AI crawlers for being on their site.
• 1:09
  Speaker 2: If they change, how much do they change?
• 1:13
  Speaker 3: So all of the content creators on cloud flair site
• 1:16
  Speaker 3: get to determine whether or not they allow the crawlers
• 1:19
  Speaker 3: on their site and what they're going to charge. And
• 1:20
  Speaker 3: the way that it works is that that communication happens
• 1:23
  Speaker 3: privately between the site owner and the AI crawler. So
• 1:28
  Speaker 3: each site will determine their own price and at the
• 1:30
  Speaker 3: same time, each crawler will also determine their own price.
• 1:33
  Speaker 3: And when the market meets, that's where we'll begin to
• 1:35
  Speaker 3: see a sustainable ecosystem.
• 1:37
  Speaker 2: Flourish I see and so what roughly on average what's being.
• 1:41
  Speaker 3: Changed, So we're really at the beginning of creating this
• 1:46
  Speaker 3: sustainable ecosystem. The first thing that we needed to do
• 1:48
  Speaker 3: is make sure that content creators had control and so
• 1:51
  Speaker 3: that's why we change the defaults on the cloud Flair network.
• 1:54
  Speaker 3: We've now changed it so that for new domains on
• 1:56
  Speaker 3: the cloud Flare network, AI crawlers are blocked by default,
• 1:59
  Speaker 3: and that will allow this market to come together. But
• 2:02
  Speaker 3: as you know, there's been a lot of deals done
• 2:04
  Speaker 3: with very large AI companies with very large publisher and
• 2:08
  Speaker 3: what we're going to be able to do on our
• 2:10
  Speaker 3: marketplace is bring the long tail of both AI crawlers
• 2:13
  Speaker 3: and content creators together and we'll see what that produces.
• 2:16
  Speaker 2: Okay, so if okay, so we're obviously not going to
• 2:20
  Speaker 2: find out how much is being charged, and I can
• 2:21
  Speaker 2: understand it might be commercially sensitive, But if everybody in
• 2:24
  Speaker 2: the Internet was to charge the AI crawlers for the
• 2:27
  Speaker 2: AI crawlers using it, would this actually lead to the
• 2:31
  Speaker 2: likes of media companies being able to fund to make
• 2:35
  Speaker 2: a decent amount of money off it to fund the
• 2:37
  Speaker 2: gap that they're seeing in declining advertising rates at the moment.
• 2:41
  Speaker 3: So, as you know, what's going on on the Internet
• 2:44
  Speaker 3: is that people are reading the derivative rather than reading
• 2:46
  Speaker 3: the ration. And when that happens, it's very difficult to
• 2:49
  Speaker 3: monetize through ads, subscriptions, or just through the satisfaction of
• 2:53
  Speaker 3: knowing that someone is reading your content. And so we
• 2:55
  Speaker 3: think this is the first step, but we think there'll
• 2:58
  Speaker 3: be a sustainable business model that includes being paid for crawling.
• 3:03
  Speaker 3: But there are probably other business models that we can't
• 3:06
  Speaker 3: even see are going to emerge based on what's going
• 3:09
  Speaker 3: on with AI. But what we know is that in
• 3:12
  Speaker 3: order for AI to thrive, content has to thrive. So
• 3:15
  Speaker 3: This is not just about protecting content creators, but this
• 3:18
  Speaker 3: is also about creating a world where AI innovation can
• 3:21
  Speaker 3: thrive as well.
• 3:22
  Speaker 2: Do you think everybody on the Internet has to do
• 3:25
  Speaker 2: this in order for it to work, because otherwise the
• 3:27
  Speaker 2: II is just going to scripe the freeway science, isn't it.
• 3:31
  Speaker 3: I think that what has to happen is if you look,
• 3:33
  Speaker 3: there have been very large AI companies and very large
• 3:36
  Speaker 3: content creators that have deals. But the only way that
• 3:40
  Speaker 3: those deals work is if others don't get access to
• 3:43
  Speaker 3: that content for free. So what everyone has to have
• 3:45
  Speaker 3: is the ability to have control over who's on their site.
• 3:48
  Speaker 3: But they may decide that it makes sense to let
• 3:50
  Speaker 3: someone through because they already have signed a deal with them,
• 3:53
  Speaker 3: or because they are sending traffic back to them. So
• 3:56
  Speaker 3: I think what matters is that you have the control
• 3:59
  Speaker 3: that you need so that you can set the deals
• 4:01
  Speaker 3: that you want. Maybe some of them will be exclusive,
• 4:03
  Speaker 3: maybe some of them you'll charge many people, some maybe
• 4:06
  Speaker 3: you won't charge some at all because you're going a
• 4:08
  Speaker 3: bunch of traffic from those crawlers.
• 4:10
  Speaker 2: So it may it may end up in a situation
• 4:13
  Speaker 2: where one of the crawlers is the best one of
• 4:16
  Speaker 2: the AI providers is the best AI provider because it
• 4:19
  Speaker 2: has access to the best websites. Yeah, whereas other others
• 4:22
  Speaker 2: may be crappy because they haven't got the good access.
• 4:25
  Speaker 2: Is that a potential future.
• 4:27
  Speaker 3: I think there's a real optimistic view of the future
• 4:30
  Speaker 3: where this is where original, high quality content thrives because
• 4:35
  Speaker 3: to your point, AI crawlers want access to that content,
• 4:38
  Speaker 3: and so we do think that there is a real
• 4:40
  Speaker 3: opportunity here for us to create an ecosystem where there's
• 4:44
  Speaker 3: demand for high quality, original content, which we think is
• 4:46
  Speaker 3: good for AI but also good for human beings.
• 4:49
  Speaker 2: Ah, this is fascinating. Hey, thank you, Stiffane. You appreciated.
• 4:51
  Speaker 2: Stiffaney Cohen cloud Fair, chief strategy Officer.
• 4:55
  Speaker 1: For more from us talks at b listen live on
• 4:57
  Speaker 1: air or online, and keep our shows with you wherever
• 5:00
  Speaker 1: you go with our podcasts on iart Radio.