Latest / Giant Ideas / World Leading Tech Analyst, Benedict Evans: Making Sense of AI

Transcript

World Leading Tech Analyst, Benedict Evans: Making Sense of AI · Giant Ideas

• 0:01
  SPEAKER_00: We've had this huge amount of technology push, but it's not clear what the product differences would be in the products or what a product strategy would look like, other than put a better UI and better branding on the model.
• 0:11
  SPEAKER_00: And in a sense, the only product strategy you can see at OpenAI is like hire Kevin Wheel.
• 0:15
  SPEAKER_02: Hello, and welcome to Giant Ideas with me, Cameron McLean, and me, Tommy Stadlin.
• 0:20
  SPEAKER_01: We're co-founders of Giant Ventures, which builds and backs purpose-driven companies.
• 0:24
  SPEAKER_02: At Giant, we're lucky to meet extraordinary people with giant ideas that are changing the world.
• 0:29
  SPEAKER_02: This podcast brings you behind-the-scenes access to those ideas and the inspiring stories of the people behind them.
• 0:35
  SPEAKER_01: We explore how one giant idea can kickstart a billion-dollar company, shape culture, and transform life as we know it.
• 0:42
  SPEAKER_01: Today we're joined by one of the most insightful voices in technology, Benedict Evans.
• 0:47
  SPEAKER_01: He's the founder of Benedict's Newsletter, one of the most widely read newsletters in the industry, where he asked the questions what matters in tech, what might it mean, and what will happen next.
• 0:58
  SPEAKER_02: If you followed the evolution of technology, it's likely you've read Benedict's work.
• 1:02
  SPEAKER_02: He has spent 20 years analysing media and technology and worked across equity research, strategy, consulting, and venture capital.
• 1:10
  SPEAKER_02: He was a partner at Andreas and Horowitz and is now an independent analyst.
• 1:14
  SPEAKER_01: Benedict's newsletter now attracts around 200,000 subscribers.
• 1:18
  SPEAKER_01: He writes essays, presents annual presentations that are much read across venture capital and startup land.
• 1:25
  SPEAKER_01: And he writes a weekly column analysing the new questions and ideas shaping society and technology.
• 1:31
  SPEAKER_01: So welcome to giant ideas.
• 1:33
  SPEAKER_01: We'd love to start just by asking you maybe to compare and contrast the previous big technology era of mobile and web 2.0 with today's era of AI.
• 1:43
  SPEAKER_01: You were a sort of major analyst of the previous era, you're one of the big analysts of this era.
• 1:47
  SPEAKER_01: How do you see the two and how do you compare them?
• 1:51
  SPEAKER_00: Um well, so maybe there's sort of two answers to this.
• 1:55
  SPEAKER_00: One of them is we are sort of at the stage now of like the internet in the mid-90s or mobile in the mid-2000s, when it was very clear that this was going to be an enormous thing, but it wasn't clear at all what the market structure was going to be, where the value capture is, what the building blocks are, how it all fits together, what it's going to look like.
• 2:16
  SPEAKER_00: Um, so you look at the web in the mid-in, like I've got this great diagram of from 1995 from a research house of something called cyberspace, within which it includes the matrix, within which is obviously this is very 1995, within which there is something called the web.
• 2:31
  SPEAKER_00: And it because it kind of wasn't clear, it was clear everyone was going to have some kind of device that was connected to some kind of a network.
• 2:38
  SPEAKER_00: It wasn't clear it was going to be a computer and not a terminal.
• 2:40
  SPEAKER_00: It wasn't clear it would be the internet and not all these other networks.
• 2:44
  SPEAKER_00: It wasn't clear the internet was going to be the web.
• 2:46
  SPEAKER_00: It wasn't clear how the web was going to work.
• 2:47
  SPEAKER_00: Like Tim Berners-Lee's original web browser had an editor because he thought this was kind of like a network drive.
• 2:52
  SPEAKER_00: It was a document sharing system, not a publishing system.
• 2:55
  SPEAKER_00: And it wasn't clear that all the value was going to be in search advertising and social.
• 2:59
  SPEAKER_00: I mean, search advertising didn't appear for another five years, and social doesn't really work for another 10 years.
• 3:05
  SPEAKER_00: Same thing with smartphones.
• 3:06
  SPEAKER_00: And like, you know, I had my bought my first smartphone in I don't know, 2001, maybe.
• 3:23
  SPEAKER_00: And we're sort of at the same stage now, I sent in in as I think in that like you've got this thing that's amazingly cool, but it's not clear how it works.
• 3:31
  SPEAKER_00: It's not clear where the value capture is or the market structure.
• 3:34
  SPEAKER_00: It's not entirely clear what you do with it.
• 3:36
  SPEAKER_00: There's a bunch of people kind of running around with their hair on fire saying, oh my god, it's amazing, you don't get it, this is everything.
• 3:40
  SPEAKER_00: And then you go and look at the survey data and you see that basically like 75% of people in the developed world have looked at this stuff, and 50% of those people have said, Well, I don't know what to do with this and never used it again.
• 3:50
  SPEAKER_00: So there's this sort of sense that like, you know, the curve is ticking up, but we don't quite know what this is gonna look like.
• 3:55
  SPEAKER_00: I think there's a then there's a sort of a fundamental difference, which is that, you know, in 2007, 2010, you knew what the next iPhone could and couldn't be.
• 4:04
  SPEAKER_00: Like you didn't know exactly what Apple was going to do, but like you knew it couldn't like have 10 times the speed and it couldn't fly.
• 4:09
  SPEAKER_00: You know, you know, it wasn't gonna unfull, you know it wasn't gonna roll up, you know it wasn't gonna project onto the wall.
• 4:14
  SPEAKER_00: In 1995, you kind of knew that the whole world wasn't gonna have a PC with broadband in a year.
• 4:18
  SPEAKER_00: Like you knew what the roadmap for deploying broadband was, you knew what how fast computers could get, and so on.
• 4:22
  SPEAKER_00: So you kind of knew like at a kind of a basic physics level what was going to be possible in three to five years' time.
• 4:27
  SPEAKER_00: And we kind of don't know that with generative AI.
• 4:30
  SPEAKER_00: So it may be that what we've got now is kind of gonna flatten out in the next year or so, and what we've got now is kind of it.
• 4:37
  SPEAKER_00: And it'll improve incrementally and it'll get faster and cheaper, but this is basically the level of capability we're gonna have, or maybe a bit more.
• 4:43
  SPEAKER_00: Or it may be then in five years' time, like you know, we'll all be queuing up to get our ration of gruel from our robot overlords.
• 4:49
  SPEAKER_00: But the point is we don't I mean, kind of exaggerating to make the point, but we don't have that same kind of basic theoretical understanding of what is it that this could be, which in a sense is the whole debate.
• 4:59
  SPEAKER_00: You know, is this stuff going to be like more spreadsheets or more web or more internet?
• 5:03
  SPEAKER_00: Is it just going to be another building block for another platform and then there'll be another platform after that, or is this more like computing or electricity?
• 5:09
  SPEAKER_00: And it's a fundamental change in the nature of what we can do.
• 5:12
  SPEAKER_00: And we don't know.
• 5:13
  SPEAKER_00: Do you have an idea?
• 5:14
  SPEAKER_00: I don't like um I don't like trying to analyse stuff on the basis of zero knowledge.
• 5:21
  SPEAKER_00: And this is why I think the doom fa intellectual kind of fallacy of the whole Doomer thesis is you're constantly trying to reason out the nature of something that you don't know anything about.
• 5:29
  SPEAKER_00: And, you know, your argument can be logically flawless, but that doesn't mean it's correct.
• 5:33
  SPEAKER_00: So, you know, it seems to me a sort of kind of a very pointless intellectual exercise.
• 5:37
  SPEAKER_00: It's like trying to reason out the nature of God.
• 5:39
  SPEAKER_00: I mean you can come out with you know something that's mathematically correct and says therefore God must be like this, but the problem is it's built on an absence of knowledge, you don't know.
• 5:46
  SPEAKER_00: And this is the Descartes fallacy.
• 5:48
  SPEAKER_01: How do we know when that big discontinuity moment happened?
• 5:52
  SPEAKER_01: So the equivalent of the iPhone moment.
• 5:54
  SPEAKER_01: I guess some people knew when the iPhone came out this was, you know, the the moment, most people didn't.
• 5:59
  SPEAKER_01: How do we know that it's not, as you said, like maybe maybe this is it, maybe this is the big moment and there's nothing more to come.
• 6:05
  SPEAKER_01: As a as an observer and an analyst, what's your process to try to make those calls of this is the big moment or actually we're nowhere near the big moment?
• 6:13
  SPEAKER_00: I think you know, I think if you knew what was going to happen all the time, then we'd live like in a different universe.
• 6:18
  SPEAKER_00: Um, there wouldn't be any VCs, like everything would there'd be no end everything would be risk-free.
• 6:22
  SPEAKER_00: Um that's like, you know, please invest only in the companies that don't work.
• 6:26
  SPEAKER_00: Um I think and and and if you go back a couple of years, like everyone kind of knew that mobile had happened five years ago at least.
• 6:36
  SPEAKER_00: And so then the question there was a long time where the question was, well, what's the next thing?
• 6:40
  SPEAKER_00: And the two big candidates, you know, people spend a lot of time talking about crypto and web 3, and like, is this the next platform that you would use to build software?
• 6:48
  SPEAKER_00: Not Web3 in like a generic sense of blockchains, but in the sense that like you could actually build distributed software on a blockchain.
• 6:55
  SPEAKER_00: You could build Instagram on a blockchain, and that would work differently in important ways.
• 6:59
  SPEAKER_00: So that theory is still sort of around.
• 7:01
  SPEAKER_00: There's a joke that like people who are still working on crypto are like those Japanese soldiers who are like lost on an island in the in the Pacific and didn't know the war was over, um, which is sort of funny, but not quite true.
• 7:10
  SPEAKER_00: Um and there's a bunch of people working on crypto.
• 7:12
  SPEAKER_00: It may be that crypto ends up just being like financial infrastructure, and maybe that crypto turns into like an ex another platform.
• 7:18
  SPEAKER_00: Um the other big candidate, of course, is some combination of VR and AR.
• 7:22
  SPEAKER_00: Um it's very it's always been kind of tough to see VR as being mainstream, um, or certainly as being any bigger than games consoles, which you know, a couple of hundred million people, maybe.
• 7:32
  SPEAKER_00: Easier to see glasses, something like this, maybe as being the next universal device after smartphones.
• 7:37
  SPEAKER_00: Um problem is we don't have the optics to do that.
• 7:39
  SPEAKER_00: We don't know when we will.
• 7:40
  SPEAKER_00: Like Apple and Google and Meta show stuff, Apple's got stuff, nothing's it's not clear that we're close to having something that would look like this.
• 7:48
  SPEAKER_02: The survey you mentioned, where I think it was uh what percentage?
• 7:51
  SPEAKER_02: 20% found continual usage in the in AI and the other agents.
• 7:55
  SPEAKER_00: That's not my survey, there's a whole bunch of people have done it's like half a dozen surveys.
• 7:58
  SPEAKER_02: And was that was that predominantly in in consumer?
• 8:00
  SPEAKER_02: Or was it more sort of consumer in consumer usage or enterprise usage?
• 8:05
  SPEAKER_00: So consumer, you know, so you where there are surveys that single out uh used at home, used for work, the used at home numbers tend to be lower.
• 8:15
  SPEAKER_00: The used for work numbers tend to be slightly higher.
• 8:18
  SPEAKER_00: And then they vary.
• 8:19
  SPEAKER_00: So, you know, people with STEM degrees use it 20%.
• 8:22
  SPEAKER_00: 20% of people with STEM degrees versus like 5% of people with no degree.
• 8:26
  SPEAKER_00: But even then, you're like, you've got a STEM master's degree and you work for a tech company, 20% of you are using this every day.
• 8:33
  SPEAKER_00: Now, on the one hand, it's a very grass, half full, half-full, glass, half-empty kind of observation.
• 8:37
  SPEAKER_00: Like on the one hand, you said, oh my god, this is amazing.
• 8:39
  SPEAKER_00: It's only been out two and a half years.
• 8:40
  SPEAKER_00: Already 20% of people are using this.
• 8:42
  SPEAKER_00: And that's way more than smartphones or PCs or the internet.
• 8:45
  SPEAKER_00: Much faster.
• 8:46
  SPEAKER_00: That's a really dumb analysis because to get a smartphone, you had to pay$1,000.
• 8:49
  SPEAKER_00: To get a PC, you had to pay two or three thousand dollars and wait for your telco to deploy broadband before it was useful.
• 8:54
  SPEAKER_00: So of course it took longer.
• 8:55
  SPEAKER_00: You know, the original, um I was just looking at this the other day.
• 8:58
  SPEAKER_00: If you wanted to run VisaCalc, which is the first successful software spreadsheet, which we can probably talk about in the late 70s, to get the Apple II with enough memory and a monitor and a flop and a disk drive and a printer was something between$10,000 and$15,000 adjusted for inflation.
• 9:16
  SPEAKER_00: So yeah, chat GPT happened faster than that.
• 9:20
  SPEAKER_00: So why is it that 50%, 60, 70% of people have looked at this and said, don't know what to do with this?
• 9:27
  SPEAKER_02: I'd be very curious if there's a I'm sure there's a magical number within OpenAI where you sort of use ChatGPT a number of times and then you become a super user.
• 9:34
  SPEAKER_02: So I suspect in time that will that will emerge because I definitely had a tipping point where I wasn't using it that often and then it became my default for everything.
• 9:42
  SPEAKER_00: Yeah, and I think there's there's a lot of uncertainty here.
• 9:44
  SPEAKER_00: So some of this is is this just because it's early and it will grow?
• 9:48
  SPEAKER_00: Some of it is, is this about breaking patterns and forming new habits?
• 9:51
  SPEAKER_00: Like you had to realize that you could use Google for that.
• 9:54
  SPEAKER_00: Some of it is about what kind of job do you have and what kind of work do you do.
• 9:59
  SPEAKER_00: Some of it is you need and and then there's a thesis that says no.
• 10:03
  SPEAKER_00: This is like, you know, go back to the early 80s.
• 10:06
  SPEAKER_00: The idea was you would buy a PC and then you would buy a database program and you would make your own inventory management software, or you would make your own tax calculating software.
• 10:14
  SPEAKER_00: And the answer is no.
• 10:15
  SPEAKER_00: You buy an inventory management software, you buy QuickBooks or whatever it was at the time.
• 10:21
  SPEAKER_00: And we are, and you can, you know, this is always the thing, you can believe two things at once.
• 10:25
  SPEAKER_00: You can believe this is a future and also believe that it has to get wrapped in software.
• 10:29
  SPEAKER_00: And so the devil's ad the devil's advocate position here would be to say, well, these people who are using it every day now are the same people who use Notion and Slack.
• 10:36
  SPEAKER_00: They're the early adopters who are always looking for a cool new tool, and this is a cool new tool, and they found it and they forced ways to find how to do how to use it.
• 10:43
  SPEAKER_00: And that overlaps with people who work in a certain kind of way.
• 10:49
  SPEAKER_00: Like, if you think about like which kind of teams can adopt Notion and which ones can't, well, small tech forward, flexible thing companies where people are doing lots of different things and people are free to do their own processes, then people are likely to use Notion or AirTable or one of these no-code things.
• 11:04
  SPEAKER_00: If you're doing accounts payable inside Exxon.
• 11:09
  SPEAKER_00: No, you don't get to get rid of Oracle and use Notion.
• 11:12
  SPEAKER_00: That's not how it works.
• 11:14
  SPEAKER_00: Um, and so there's there are these sort of, you know, it's kind of like the point about you know meetings.
• 11:18
  SPEAKER_00: Like most people have like two meetings a week.
• 11:21
  SPEAKER_00: So you get these startups saying we're gonna pitch something as A, I think it's gonna make it easier to manage email, manage your meetings.
• 11:26
  SPEAKER_00: Like basically the only people who have that problem are VCs.
• 11:28
  SPEAKER_00: No one else has that problem.
• 11:31
  SPEAKER_00: VCs and consultants have that problem.
• 11:33
  SPEAKER_00: Nobody else has a problem.
• 11:34
  SPEAKER_00: I've got 45 meetings this week.
• 11:37
  SPEAKER_00: So that's kind of the there's a sort of devil's advocate here, which is said it's clear it's clearly got product market fit in software development and marketing.
• 11:43
  SPEAKER_00: It's got software development, so it's got product market fit for a certain kind of knowledge worker.
• 11:48
  SPEAKER_00: Is it just going to grow out from those?
• 11:51
  SPEAKER_00: Does it need something else to grow out from those?
• 11:54
  SPEAKER_00: Or are those just the right people, the only people who can use it a lot as it is now?
• 12:01
  SPEAKER_00: Does it need other does it need to get wrapped in product?
• 12:04
  SPEAKER_00: I think it needs to get wrapped in product, but you know, I don't know.
• 12:07
  SPEAKER_00: We'll find out.
• 12:07
  SPEAKER_02: I do too.
• 12:08
  SPEAKER_02: Um and I think that's an interesting segue into sort of describing the value chain for our for our listeners.
• 12:13
  SPEAKER_02: So in terms of how if you took at the AR landscape today and what's been built and what could be built, how would you break it down?
• 12:21
  SPEAKER_00: So it seems to me that we we're sort of two and a half years into the model wars.
• 12:26
  SPEAKER_00: Um anybody with half a billion dollars can have a frontier model.
• 12:28
  SPEAKER_00: This is what Deep Seat demonstrated.
• 12:30
  SPEAKER_00: It wasn't five million dollars, it was like half a billion dollars, maybe double that.
• 12:34
  SPEAKER_00: Um but lots of people have got half a billion dollars.
• 12:37
  SPEAKER_00: And model building today, it's sort of like the feeds and speeds era of the PC industry, like in the late 80s, early 90s, like all these computer magazines, like group test, 20 new graphics cards, which one should you buy?
• 12:48
  SPEAKER_00: And they're like, well, how many megahertz, how much RAM, what video bandwidth?
• 12:51
  SPEAKER_00: Do you want CGA or VGA?
• 12:52
  SPEAKER_00: Well, it depends.
• 12:54
  SPEAKER_00: Um, do you want 16-bit colour?
• 12:56
  SPEAKER_00: Do you want 32-bit colour?
• 12:57
  SPEAKER_00: Well, this one, this is 32-bit colour.
• 12:59
  SPEAKER_00: Okay, and you're supposed to know what that means and care what that means.
• 13:02
  SPEAKER_00: And a lot of it is like that.
• 13:03
  SPEAKER_00: What what you know, what context window do you want?
• 13:05
  SPEAKER_00: Like, I don't fucking know.
• 13:06
  SPEAKER_00: I don't know what that means.
• 13:08
  SPEAKER_00: And so some of it is like that.
• 13:09
  SPEAKER_00: But but but in generality, like you've got this these sort of three things here.
• 13:13
  SPEAKER_00: On the one hand, you've got these half a dozen, and how you count it, half a dozen to a dozen organizations, trying to build a frontier model, and they're kind of leapfrogging each other, and every week there's in more models and more acronyms, and instead of like USB and PCI, it's like MCP and you know RAG and whatever the new acronym is.
• 13:32
  SPEAKER_00: You can't, you know, no one can keep track of all of these papers.
• 13:33
  SPEAKER_00: And there's hundreds of papers a week, no one can keep track.
• 13:36
  SPEAKER_00: It's like Moore's Law.
• 13:38
  SPEAKER_00: Um, if there were 10 Intels.
• 13:40
  SPEAKER_00: The other side of it is there's hundreds and hundreds of SaaS companies who are picking this up and using it to solve some point solution inside some industry.
• 13:48
  SPEAKER_00: So a friend of mine is using LLMs, and the problem he's solving is you do machine translation of mainframe cobalt code into Java, but the machine translation produces terrible code that's impossible to read.
• 13:58
  SPEAKER_00: And so he's using LLM to clean up the code.
• 14:01
  SPEAKER_00: Is that an AI company?
• 14:02
  SPEAKER_00: Or is that like a digital transformation company?
• 14:04
  SPEAKER_00: Or is that a mainframe process optimization company?
• 14:07
  SPEAKER_00: I don't even know what the category is called, but it's not like an AI company.
• 14:10
  SPEAKER_00: And there are hundreds and hundreds and hundreds of people picking those up and using that to solve some problem.
• 14:15
  SPEAKER_00: And this might be a$50 million opportunity, that might be a$500 million opportunity, that's what SaaS was.
• 14:20
  SPEAKER_00: Ince it with SaaS, it was we're basically using databases and cloud.
• 14:23
  SPEAKER_00: And now it's we're basically using LMs, and then we're gonna go out and solve some problem for the billing departments in airlines or whatever it is.
• 14:30
  SPEAKER_00: So you've got those kind of two fields where there's lots of stuff like specific creative engineering energy going on.
• 14:35
  SPEAKER_00: And then in the middle, there's this kind of big fuzzy mess of like you're supposed to use AI.
• 14:39
  SPEAKER_00: But what does that mean?
• 14:41
  SPEAKER_00: And it reminds me just a little bit and rather painfully of Metaverse.
• 14:45
  SPEAKER_00: You know, where people will say, What's your metaverse strategy?
• 14:48
  SPEAKER_00: And you would think, and I would I mean my reaction to that was, well, I can talk about VR, I can talk about NFTs.
• 14:54
  SPEAKER_00: I have an opinion, I mean you know, say what you like about NFTs.
• 14:57
  SPEAKER_00: At least there was like an actual specific tangible technology there and an idea.
• 15:01
  SPEAKER_00: I mean, most of it was nonsense, but it was an actual, there was something you could put your hands around and analyze and have an opinion about.
• 15:07
  SPEAKER_00: Um, you know, should games be interoperable?
• 15:09
  SPEAKER_00: You know, should social media be interoperable?
• 15:10
  SPEAKER_00: Like maybe I can have an opinion about that.
• 15:12
  SPEAKER_00: At least I know what we're talking about, but I don't know what you're talking about when you say metaverse.
• 15:16
  SPEAKER_00: And there's a sort of a similar point now when when like Toby, what's his name at uh at at Shopify says, Everyone in the company, you need to be using AI every day.
• 15:25
  SPEAKER_00: Okay, what?
• 15:26
  SPEAKER_00: What do you mean?
• 15:26
  SPEAKER_00: So I'm running the accounts payable team.
• 15:28
  SPEAKER_00: So, you know, we've just migrated from Oracle to Salesforce.
• 15:32
  SPEAKER_00: You want me to use AI?
• 15:34
  SPEAKER_00: What do you mean?
• 15:35
  SPEAKER_00: Toby, what am I supposed to do?
• 15:36
  SPEAKER_00: What do you want me to do?
• 15:38
  SPEAKER_00: Um and so there's a lot of this kind of fuzzy area in the middle where people are kind of pushing for product market fit.
• 15:45
  SPEAKER_00: And I think the interesting thing, which which is I'm sort of trying to write something about, is that if you look at as a consumer, as a normal person, you look at OpenAI, Chat GPT, or Google, or Gemini, or Meta, or Anthropic, or DeepSeek, or Mistral, or Quem, or whatever the others are, how do you tell the difference?
• 16:06
  SPEAKER_00: I mean, there's a different the colour, the logo is a different colour.
• 16:09
  SPEAKER_00: And you have this pull-down list that says, do you want to use GPT-4.0 or for a different result, do you want to use GPT-04?
• 16:17
  SPEAKER_00: And you know, the the there's a sort of there's a there's a moment here where we've had this huge amount of technology push, and but it's not clear what the product differences would be in the products or what a product strategy would look like, other than like put a better UI and better branding on the model.
• 16:33
  SPEAKER_00: There's a strategy of building better models.
• 16:36
  SPEAKER_00: There is like a corporate strategy where Meta says, you know, we want to make this commodity open source infrastructure that's sold at marginal cost, and um Google says this, you know, and Amazon kind of says the same thing.
• 16:49
  SPEAKER_00: And Google and Microsoft have kind of got a different approach because they've got a cloud business, but they've also got a legacy software business that they don't want to keep disrupted by this.
• 16:56
  SPEAKER_00: And so you can kind of it's got and you can talk about what Apple's trying to do and so on.
• 16:59
  SPEAKER_00: So there's like corporate strategy, but it's very hard to say, well, what is Mike Krieger going to do fund that's going to be absolutely fundamentally different from what Kevin Wheel's doing?
• 17:10
  SPEAKER_00: And in a sense, the only product strategy you can see at OpenAI is like hire Kevin Wheel.
• 17:14
  SPEAKER_00: I mean, he's I saw this sort of interview with him and Mike on stage, and and Kevin is like, okay, I wake up in the morning on Tuesday, I get an email that says we've got a new voice member, we've got a new voice model.
• 17:22
  SPEAKER_00: It's like, okay, I guess I'm adding a microphone button to the app.
• 17:25
  SPEAKER_00: Fascinating.
• 17:26
  SPEAKER_00: This is not product strategy.
• 17:28
  SPEAKER_01: And when you look at all of that, do you think that we could be in a similar situation to when we had the big search engine wars and you know then almost all of them went to zero, right?
• 17:37
  SPEAKER_01: So could OpenAI, for example, be the next RG?
• 17:41
  SPEAKER_01: Or do you think that they've now got to the scale of the brand that seems to be their main motive?
• 17:47
  SPEAKER_01: And uh, you know, I guess no network effects, but you would argue with Google's offerings maybe their their their motive somehow with data, and maybe with Meta and Grachle X it's it's some kind of network effect.
• 17:58
  SPEAKER_01: Do you feel like those are roughly the parameters?
• 18:01
  SPEAKER_01: And and yeah, do you think some of these companies or some of the LLMs uh companies could go to zero?
• 18:08
  SPEAKER_00: I think as at a primary level, there aren't any network effects.
• 18:12
  SPEAKER_00: This this thing does not get better if more people use it.
• 18:16
  SPEAKER_00: And that's partly because you're not retraining it continuously, it's partly because the data that's getting put it that would come from the queries is so tiny as a percentage of training data.
• 18:27
  SPEAKER_00: So, you know, what happened with you know, and the comparison with with with with Google is interesting because what actually happened was there were lots of people doing search one way, and they were all basically commodities, and they thought the answer was build a content portal, you know, Yahoo builds a media business around it.
• 18:43
  SPEAKER_00: And they thought they thought search was a commodity will outsource it to Google.
• 18:47
  SPEAKER_00: And Google had this fundamentally different approach to search that swept away all of the previous pre-existing search engines because it was so much better.
• 18:53
  SPEAKER_00: And it then had a network effect, which was, as we all kind of understand, you know, Google knows what you search for next.
• 18:59
  SPEAKER_00: When you get search results, you pick one and Google gets to see which search result was right, and so Google gets better because more people use it.
• 19:06
  SPEAKER_00: Um there's not is not yet apparent that there's any equivalent network effect in LLMs.
• 19:10
  SPEAKER_00: It's not yet apparent that there's any equivalent when it takes all effect.
• 19:13
  SPEAKER_00: There are things people can try and do are trying to do to build stickiness.
• 19:17
  SPEAKER_00: So the memory features that they're all starting to add where they can kind of go back and remember the stuff that you've asked in the past.
• 19:22
  SPEAKER_00: But that's not really a network effect, that's stickiness.
• 19:26
  SPEAKER_00: And I mean it might evolve into a network effect.
• 19:29
  SPEAKER_00: You can sort of see that it might start to learn there are other people that are kind of like you, but you know, there are a lot of stuff that would have to happen first for that to really work.
• 19:38
  SPEAKER_00: Um meanwhile, this sort of brand and habit.
• 19:43
  SPEAKER_00: And you know, there were right, there's this very live conversation about you know, why does people why do people use Google as they go through the antitrust remedy stage of the competition case that's going on in the US right now, or like every day these are the hearings that are happening?
• 19:56
  SPEAKER_00: Um and clearly some of Google's advantages have it, that it became a verb, and people go to Google because they go to Google, but they also go to Google because the results are better.
• 20:02
  SPEAKER_00: And if you go to go to Bing, like the first result seems fine, you use it for a day, you're like, no, this isn't as good as Google, and you go back.
• 20:08
  SPEAKER_00: Um, and yeah, Microsoft acknowledged this, and this is why they talk about the network effect.
• 20:12
  SPEAKER_00: Of course, when Microsoft says this, it's kind of self-serving.
• 20:14
  SPEAKER_00: Now, or Google, part of Google story here is they will say, no, it's about accumulated expertise.
• 20:20
  SPEAKER_00: It's something accumulated in from in in institutional knowledge, that we understand that it's no longer just page rank and which results you click.
• 20:26
  SPEAKER_00: There's hundreds of other things going on, and we know all this other stuff about how to build good search and what all the problems are and what breaks in the 20 different, 50, 100 different ways that things that you have to know about in order to build a good search engine.
• 20:39
  SPEAKER_00: Um, and so it might be that you get kind of, you know, that that that that OpenAI kind of pulls ahead because they have that accumulated knowledge.
• 20:48
  SPEAKER_00: But that would be something that would have to happen next.
• 20:52
  SPEAKER_00: It's not something that's happening now.
• 20:54
  SPEAKER_00: It's not apparent that that's what how it will work.
• 20:57
  SPEAKER_00: And in parallel to that, of course, there's a question we were just talking about, which is is it actually going to the UI, the chatbot, and typing something in the right UI, or is the end state of this that it gets abstracted away for most things?
• 21:09
  SPEAKER_00: And the answer is probably both.
• 21:11
  SPEAKER_00: That, you know, there'll be thousands of FAS outs where this is an API call.
• 21:15
  SPEAKER_00: But you will also go, there will also be things where you go to the LLM for sp for for specific things.
• 21:20
  SPEAKER_00: But there is this real challenge of how would you know what you would ask?
• 21:25
  SPEAKER_02: It seems perhaps it's a bit flippant to say, but it seems somewhat obvious to me that the value will accrue the application there, because even in the case of OpenAI, most of the excitement um has been driven by chat GPT.
• 21:39
  SPEAKER_02: And that's been the enabler.
• 21:40
  SPEAKER_00: Yeah, most of the money comes from the API.
• 21:43
  SPEAKER_02: But to your point about around it getting commoditized over time.
• 21:47
  SPEAKER_00: But this is what we said at the beginning.
• 21:49
  SPEAKER_00: This is a sort of elemental question is do these things keep getting better?
• 21:51
  SPEAKER_00: Because if these things get like 100X better, whatever that means, then you really can just go to the model and say, hey, like I need to buy a house.
• 21:58
  SPEAKER_02: Right.
• 22:00
  SPEAKER_00: It will go and log into your bank.
• 22:01
  SPEAKER_00: I mean, I'm going through this at the moment, but like it will go and log into your bank.
• 22:04
  SPEAKER_00: It will put together the pack for the mortgage broker and it will put together this and it will put together that.
• 22:08
  SPEAKER_00: And it will just, you know, it will give you five choices and say which one would you like to buy?
• 22:11
  SPEAKER_00: And I'll say this one.
• 22:12
  SPEAKER_00: And then you go and have a viewing, and then the next day it says, Right, this is your new mortgage payment.
• 22:16
  SPEAKER_00: It's been set up.
• 22:17
  SPEAKER_00: Don't worry about it.
• 22:17
  SPEAKER_00: It's taken care of.
• 22:18
  SPEAKER_00: It's done.
• 22:19
  SPEAKER_00: Poof.
• 22:20
  SPEAKER_00: And that's not really science fiction at the moment.
• 22:23
  SPEAKER_00: You can kind of squint and imagine that's possible.
• 22:26
  SPEAKER_00: But it certainly doesn't work now.
• 22:28
  SPEAKER_00: And you could build something that would you could certainly build something today that would try and do that.
• 22:32
  SPEAKER_00: It's the problem is, you know, it would have a 5% error rate at each stage, and there's 45 steps.
• 22:36
  SPEAKER_00: And what does 45 times the 5% error rate mean?
• 22:39
  SPEAKER_00: Yeah, it seems like it means it hasn't worked.
• 22:42
  SPEAKER_02: Mansa's getting quite close to that, no?
• 22:44
  SPEAKER_02: I mean, maybe not the mortgage element of it, but uh Yeah, I don't know.
• 22:47
  SPEAKER_00: It's like, you know, there's a story about somebody who who was going to Canline and um their their corporate travel agency um booked them a hotel and a train and a flight to go to Cat Ca.
• 22:58
  SPEAKER_00: Amazing.
• 22:59
  SPEAKER_00: Find themselves in normal day.
• 23:02
  SPEAKER_01: One of the evolutions that we've looked at in venture is, you know, venture capitalists were very excited about the LLMs and you know, you'd see these huge hundred million dollar seed rounds, and even now we're seeing with people who've left OpenAI ridiculous uh seed round evaluations because that's what's where people, some people think the value will accrue.
• 23:21
  SPEAKER_01: Um and then suddenly deep seat came and everyone panicked that it was all commoditized.
• 23:25
  SPEAKER_01: And having been very dismissive of what people called you know chat GPT rappers, suddenly people thought actually, yeah, it's these individual new startups who are because of the value of the UI being more valuable potentially than this commoditizable LLM.
• 23:39
  SPEAKER_01: That's where we should invest.
• 23:41
  SPEAKER_01: It feels like that's just going to kind of continue to go back and forth over the next few years while we figure out what's happening.
• 23:46
  SPEAKER_01: But do you if you if you were still a venture investor, where would you be putting your money?
• 23:52
  SPEAKER_00: Well, it's funny, like if you use Snapchat, we don't say, Oh, it's just a GCP wrapper.
• 23:59
  SPEAKER_00: So there's always a question of you know, how much are you really just reselling an underlying commodity with a very limited amount of differentiation on top, and the platform will clearly do that?
• 24:13
  SPEAKER_00: And how much is the platform just providing basic Plymouth primitives and you're building the product go to market, the understanding of the problem, um, the cost customer support, you're building the whole company on top.
• 24:25
  SPEAKER_00: And, you know, there's you know, God knows, you know, we don't look at the entire SaaS industry and say, well, they're just AWS wrappers.
• 24:31
  SPEAKER_00: Um on the other hand, clearly there are a bunch of people that really were just thin GPT wrappers.
• 24:36
  SPEAKER_00: So the word the thin bit is more much more important than the wrapper bit because everything is a wrapper of something.
• 24:41
  SPEAKER_00: You know, you know, Apple isn't if Apple, Apple is a TSMC wrapper, you know, this is you know, certainly you can you can reduce this to absurdity.
• 24:49
  SPEAKER_00: So the thin, the word, the important word there is thin.
• 24:51
  SPEAKER_00: Are you doing something that is naturally and inevitably going to end up as part of the platform?
• 24:55
  SPEAKER_00: I mean, you know, it's interesting to compare ChatGPT to spell checking.
• 24:59
  SPEAKER_00: And you go back to the 80s, like you go to computer magazine and they would have a group test of 10 spell checkers, and they're each like$200.
• 25:06
  SPEAKER_00: And they've got the whole product matrix of like, well, this spell checker is$150, and it's got a library of 25,000 words, and it supports Word and Word star and WordPerfect.
• 25:14
  SPEAKER_00: It's used by this one.
• 25:15
  SPEAKER_00: And then of course, A, that gets integrated into Word, and then it gets into a good, and then it just becomes a wavy red line, and then it just gets integrated in the operating system.
• 25:22
  SPEAKER_00: And like the idea that that was like a$200 product now seems ridiculous.
• 25:25
  SPEAKER_00: But there was a time when like that was it.
• 25:27
  SPEAKER_00: Like you wanted to do charts in a spreadsheet, you had to buy a chart program.
• 25:30
  SPEAKER_00: So there's always this question of how far up the stack does the platform go.
• 25:34
  SPEAKER_00: And again, this comes back to the question of how good does the model get.
• 25:36
  SPEAKER_00: Because if the model gets you know a thousand times better, then of course it'll go all the way up the stack.
• 25:40
  SPEAKER_00: You can just go to it and say buy me a house and it can do it.
• 25:42
  SPEAKER_00: But then we don't, but we don't know that.
• 25:45
  SPEAKER_00: And as long as that's not what's gonna happen, then you're betting that Amazon and Google and Microsoft and OpenAI and everything else are going to go out and one by one build the thing that automates tracking the maintenance schedules for aircraft engines at airline companies.
• 26:02
  SPEAKER_00: And of course they're not gonna do that.
• 26:04
  SPEAKER_00: They will sell that software to people who go and build that and sell that to airlines, they'll sell it to Accenture, they'll sell it to the startup that does that, just as they always have done.
• 26:13
  SPEAKER_01: I want to ask you more about defensibility.
• 26:15
  SPEAKER_01: So we we've seen a couple of companies who use one of the models, the LLMs, and then you know, help people do things like edit code or make websites.
• 26:23
  SPEAKER_01: So companies like Lovable, Cursor, just extraordinary revenue growth that we've never really seen before.
• 26:29
  SPEAKER_01: We certainly didn't see that in the web 2.0 era or the mobile era.
• 26:32
  SPEAKER_01: Um it's all very impressive, people get very excited.
• 26:35
  SPEAKER_01: Over the weekend, someone leaked the prompts that were used on things like Lovable and Cursor.
• 26:42
  SPEAKER_01: And you know, you could argue that that one leak just sort of gives away basically all of the value of some of those companies.
• 26:48
  SPEAKER_01: Maybe that's true, maybe that's not.
• 26:50
  SPEAKER_01: But if you take that as a sort of starting point and assume it's true for a moment, how does that make you feel about some of the very, very rapidly growing companies that have started to emerge from these very early parts of the new AI era?
• 27:03
  SPEAKER_00: Yeah, so again, to me, that's like saying, you know, here is a SQL that Salesforce uses to generate your lead gen page.
• 27:13
  SPEAKER_00: So SQL is so Salesforce is out of business.
• 27:15
  SPEAKER_00: Well, that's not what Salesforce is.
• 27:18
  SPEAKER_00: Um that's not what cursor is.
• 27:19
  SPEAKER_00: Cursor isn't like a carefully crafted crafted prompt.
• 27:23
  SPEAKER_00: Cursor is a product.
• 27:25
  SPEAKER_00: And um the same thing for all of these.
• 27:29
  SPEAKER_00: How do you build product?
• 27:31
  SPEAKER_00: Um, how do you build go-to-market?
• 27:32
  SPEAKER_00: How do you build defensibility?
• 27:34
  SPEAKER_00: How do you under how deeply do you understand what the market needs?
• 27:37
  SPEAKER_00: And if the answer is like not much, um, and it's basically just email, then sure, that will get, you know, that's not defensible.
• 27:44
  SPEAKER_00: If the answer is you've got to have a huge sales force and you've got to have really, really deep understanding of how the market works, then yes, that will definitely get absorbed.
• 27:51
  SPEAKER_02: And in terms of um, in terms of building product in the AI era, products like Cursor, do you think it's just age old wisdom, deeply understand your customer, you know, build a product for them, have a smart distribution model, or are there some qualities to these AI products that are different from the previous era?
• 28:07
  SPEAKER_02: In terms of how you build them?
• 28:08
  SPEAKER_00: Um Well, well, in a sense, it's always different.
• 28:11
  SPEAKER_00: Um but and but and and you know, each of these new things has some set of characteristics that make it different.
• 28:18
  SPEAKER_00: Um clearly, one of the sort of foundational questions or sort of challenges in building anything with an LLM is that it's a probabilistic system, um, not a deterministic system, and it's useful because it's doing things that you can't do with deterministic systems, but that also means that it's not going to produce exactly the same result in exactly the same way every single time.
• 28:36
  SPEAKER_00: And so what most people building quote unquote GPT wrappers are doing, half part of what they're doing is the tooling, part of what they're doing is the go-to-market, part of what they're doing is how do you think about that?
• 28:49
  SPEAKER_00: How do you manage that?
• 28:51
  SPEAKER_00: How do you filter and process and pre-process and post-process the input and the output so that the user knows what this is, or that you manage the problem, um, and so you can kind of control what will and won't happen.
• 29:02
  SPEAKER_02: That feels quite distinctive from what's come before.
• 29:07
  SPEAKER_00: It is.
• 29:08
  SPEAKER_00: Um, and you know, there's a lot of, you know, there's this is this old sort of joke that, you know, the English joke about the Frenchman who says that's all very well in practice, but does it work in theory?
• 29:18
  SPEAKER_00: And you can kind of spend too much time sort of f philosophizing about what this stuff is.
• 29:21
  SPEAKER_00: But you know, it's you know, there's a kind of important kind of conceptual shift in that, you know, traditional software is deterministic, it does exactly what you ask it to do every single time.
• 29:30
  SPEAKER_00: What machine learning and and then both machine learning and now this are probabilistic systems, and they solve things that you we weren't what for practical reasons, maybe maybe theoretical reasons, but certainly practical reasons, things that you couldn't get deterministic software to do.
• 29:43
  SPEAKER_00: But that also comes with their own their own sets of challenges.
• 29:46
  SPEAKER_00: And um it may be that as the models get better, this is kind of, you know, that the part of the models get better thesis.
• 29:52
  SPEAKER_00: If the models may get better, they're right all the time, even if they are still probabilistic, then they're right 99.99999% of the time.
• 29:58
  SPEAKER_00: So who cares if they don't actually if they're not actually deterministic?
• 30:01
  SPEAKER_00: But another way you can kind of think about this is that um what deterministic software does is it does things that are easy to explain to a computer, but maybe hard for people to do, like calculate a million mortgages in your head.
• 30:10
  SPEAKER_00: It's very easy to tell a computer how you would do that, it's very hard to do it.
• 30:13
  SPEAKER_00: What machine learning did was it was things that are hard to explain to a computer.
• 30:16
  SPEAKER_00: Like, how would you know that credit card transaction is weird?
• 30:19
  SPEAKER_00: Like, is that thing behind you really a Christmas tree?
• 30:21
  SPEAKER_00: Like, what is it?
• 30:23
  SPEAKER_00: Then maybe the one way you could think about LLMs is that they are good at things that would be easy to explain to an intern.
• 30:30
  SPEAKER_00: As soon as you get to something that would be hard to explain to an intern, then you're in trouble.
• 30:35
  SPEAKER_00: You realize I'm going to ask it how to do this thing, and I gave its three lines of instructions, and of course what I got back wasn't what I expected.
• 30:41
  SPEAKER_00: But then you think, well, if I'd given that to a 16-year-old, they probably wouldn't have given me what I wanted either, because they wouldn't have any of the kind of context or experience or understanding of what it was that I wanted.
• 30:50
  SPEAKER_00: And for actually get an intern to give that to me, I'd need to spend like a week, you know, showing them stuff and training them and building things, and then they maybe then I could give them the one-line prompt and they do it.
• 31:00
  SPEAKER_00: And so LLMs now, and it's kind of a useful way of thinking about it, is they do what's what would be easy to explain to a person, or easy to explain to a person who has no context or very little context.
• 31:11
  SPEAKER_01: Benedict, we ask everybody who comes on giant ideas, what is it about themselves that has allowed them to be so successful?
• 31:18
  SPEAKER_01: And in your case, I'd love to ask you not just that question, so what is about you do you think that has allowed you to become such a successful analyst of technology and trends and see the future and assess the past so well.
• 31:29
  SPEAKER_01: But also with AI, what are you doing in your in your work and your everyday life to make the most of these tools?
• 31:36
  SPEAKER_00: Um well, obviously it's because I'm both very clever and very modest.
• 31:40
  SPEAKER_00: Um I mean, I'm not sure how can I how else I can answer that question.
• 31:44
  SPEAKER_00: Um, I think a slightly more sort of neutral way of answering the question is like, I think, you know, an old boss of mine said that my career kind of looked like Brownian motion and I sort of ended up in the right place eventually.
• 31:56
  SPEAKER_00: But I think everyone has a sort of event diagram of stuff that you're good at and stuff that you enjoy and stuff that people will pay for.
• 32:03
  SPEAKER_00: And then you kind of pick what your priorities are.
• 32:05
  SPEAKER_00: And you know, there are some people who work in finance doing something that they hate that they're very good at that gets paid a lot of money.
• 32:10
  SPEAKER_00: And there are also people who do something that they love and they're very good at and pays almost nothing.
• 32:14
  SPEAKER_00: Um and so you kind of you know choose at least, hopefully at least two of those, and preferably all three, but you know, it depends.
• 32:19
  SPEAKER_00: But then different people have very different kinds of skills.
• 32:22
  SPEAKER_00: You know, there are people who are very good at, you know, you make a list of 10 or 15 skills, and you know no person probably has more than two or three of those.
• 32:30
  SPEAKER_00: Um and so I sort of you know slowly iterated into something that aligns with certainly two of those and and and preferably three.
• 32:38
  SPEAKER_01: How old were you when you figured that out?
• 32:42
  SPEAKER_00: I'll let you know.
• 32:44
  SPEAKER_01: And what about your use of AI tools?
• 32:46
  SPEAKER_01: How are you how are you using AI to stay on top of it?
• 32:49
  SPEAKER_00: Well, this is going goes back to AI.
• 32:51
  SPEAKER_00: Like I take a photograph, it's perfectly exposed.
• 32:53
  SPEAKER_00: Is that AI or is that software?
• 32:55
  SPEAKER_00: Well, ten years ago that was AI, now it's software.
• 32:57
  SPEAKER_00: What LLMs am I using?
• 32:59
  SPEAKER_00: I don't I struggle to find use cases.
• 33:02
  SPEAKER_01: Interesting.
• 33:03
  SPEAKER_00: So I don't write code, I don't brainstorm, I don't need a draft of an email.
• 33:09
  SPEAKER_00: I use um I do proofreading, um, which is basically with better spell check.
• 33:15
  SPEAKER_02: Do you use it for any software elements, like you know, um asking it for advice?
• 33:19
  SPEAKER_02: That's a a growing usage I'm seeing among quite a few people.
• 33:24
  SPEAKER_00: I don't know.
• 33:25
  SPEAKER_00: Um, and that may just be being on the spectrum, but I know I struggle enough asking for advice for people.
• 33:29
  SPEAKER_00: The idea of asking for advice for advice for computers, to me, I mean, I've ex I've played with with with with ChatGPT's voice mode, and to me, this is right at the bottom of the uncanny valley.
• 33:38
  SPEAKER_00: So you start asking a question and it says, hey, that sounds really exciting.
• 33:41
  SPEAKER_00: Tell me more.
• 33:43
  SPEAKER_00: Fuck off.
• 33:45
  SPEAKER_00: You know, it it's literally the uncanny valley that people talk about in computer animation.
• 33:49
  SPEAKER_00: I find it very un, you know, I have the similar, the same kind of visceral reaction that people talk about there.
• 33:55
  SPEAKER_00: There are other people who think this is great, you know, I'll just kind of bounce ideas back and forth and and and you know get something back.
• 34:01
  SPEAKER_00: Um, I suppose part of the, you know, the challenge here is always like what these things do is they tell you what most people would probably say.
• 34:10
  SPEAKER_00: Do you have lots of use cases where you just need what most people would probably say quickly?
• 34:15
  SPEAKER_00: And there are some people who do.
• 34:16
  SPEAKER_00: Like if you work in marketing, give me 50 ideas for a marketing slogan for this brand.
• 34:20
  SPEAKER_00: Like, again, it's like having an intern.
• 34:23
  SPEAKER_00: Like it gives you 50 ideas, 25 of them are crap, but five of them are things you wouldn't have thought of.
• 34:27
  SPEAKER_00: And it's still better than you spending an hour with a blank piece of paper trying to work it out.
• 34:31
  SPEAKER_00: I don't have that use case.
• 34:33
  SPEAKER_00: Other people do, so I kind of struggle to find ways in which this is useful, but you know, I have a very, very specific and very narrow set of things that I do.
• 34:41
  SPEAKER_00: Um, and for those things, this isn't particularly helpful.
• 34:45
  SPEAKER_00: Um, as I said, I don't write code.
• 34:47
  SPEAKER_00: I'm the lawyer looking at chat, looking at it, looking at the spreadsheet and thinking, well, that's great.
• 34:51
  SPEAKER_00: But I don't use I don't do that.
• 34:52
  SPEAKER_00: What do I do with this?
• 34:54
  SPEAKER_02: Well, thank you, Benedict.
• 34:55
  SPEAKER_02: That was a hugely informative uh discussion.
• 34:58
  SPEAKER_02: Um and uh we so appreciate you joining us on Giant Ideas.
• 35:02
  SPEAKER_02: Thanks for having me.
• 35:03
  SPEAKER_02: Thanks so much, Benedict.
• 35:04
  SPEAKER_01: If you like the episode, don't forget to like, subscribe, and follow the show, and you can find many more giant ideas wherever you get your podcasts.