Latest / General Magic & The Soul of Tech / Ep. 11 | Regurgitation or Intelligence? Can AI Generate Novel Thoughts? | John Giannandrea

Transcript

Ep. 11 | Regurgitation or Intelligence? Can AI Generate Novel Thoughts? | John Giannandrea · General Magic & The Soul of Tech

• 0:00
  SPEAKER_0: Hi, this is Mike Stern.
• 0:03
  SPEAKER_1: And this is Tom "Hersh" Hershenson.
• 0:05
  SPEAKER_0: Welcome to General Magic & The Soul of Tech.
• 0:10
  SPEAKER_2: We believed we were building something that would change the world, and in some ways we did.
• 0:15
  SPEAKER_2: In others, we didn't understand what we were setting in motion.
• 0:18
  SPEAKER_2: The story of General Magic isn't over.
• 0:21
  SPEAKER_2: We're still living inside what it started.
• 0:29
  SPEAKER_0: Hi, welcome to today's special episode of General Magic and the Soul of Tech.
• 0:33
  SPEAKER_0: I'm Mike Stern, one of our co-hosts.
• 0:36
  And in addition to our other co-host,
• 0:38
  we have a special guest and interlocutor,
• 0:41
  Steve Jarrett, who's going to be talking with John Giannandrea.
• 0:45
  SPEAKER_0: They are two of the giants of the American technology and global technology scene.
• 0:49
  SPEAKER_0: They have been at the center of innovation since they were young professionals at General Magic.
• 0:55
  And they have been at the center of the advent of AI and we look forward to having
• 1:00
  a really interesting conversation between the two of them,
• 1:03
  as well as Tom and me lobbing in a question from time to time.
• 1:07
  You know,
• 1:08
  later in his career,
• 1:09
  the singer-songwriter Smokey Robinson had a hit called Double Good Everything.
• 1:14
  SPEAKER_1: And that I think is the appropriate theme song for this episode.
• 1:18
  SPEAKER_1: And without further ado, Steve Jarrett and John Giannandrea.
• 1:23
  SPEAKER_0: Steve has run AI at Orange, John has run AI both at Google and at Apple.
• 1:28
  SPEAKER_0: What are the things that you'd want to talk to each other about?
• 1:31
  SPEAKER_3: Yeah.
• 1:32
  SPEAKER_3: Hey JG, great to see you.
• 1:33
  SPEAKER_3: Thanks for the time.
• 1:34
  SPEAKER_3: And Tom and Mike, thanks for having me on the show.
• 1:37
  SPEAKER_3: I'd love to hear first what really surprised you over the last year or two?
• 1:41
  SPEAKER_3: I've never seen development and innovation pace like this.
• 1:44
  Having witnessed the birth of the smartphone and the birth of the internet,
• 1:47
  which you had a front row seat for,
• 1:48
  it'd be great to hear you compare the lessons learned from those eras and how
• 1:53
  the AI era has been different and in the time that you spent at Apple working on AI,
• 1:58
  what were the things that really came unexpectedly,
• 2:01
  both in terms of pace and the challenges?
• 2:03
  SPEAKER_4: Yeah, great question.
• 2:04
  SPEAKER_4: Thanks for inviting me on the show.
• 2:06
  I mean, it is an extraordinary time,
• 2:08
  and it is interesting and you're comparing it to the 1990s and the sort of birth of
• 2:12
  the web, and then the mobile revolution that came after that.
• 2:16
  I think it is similar in the sense that resources are being poured into an area just
• 2:21
  because of the momentum of the technology.
• 2:23
  SPEAKER_4: It's not clear that all of those investments are going to be justified, honestly.
• 2:27
  But nonetheless,
• 2:29
  a lot of very smart people and a lot of resources are being poured into this area
• 2:32
  of AI.
• 2:34
  SPEAKER_4: For me this really started around about 2012 or so.
• 2:36
  SPEAKER_4: That's really when we started seeing deep learning do things that other techniques couldn't do.
• 2:41
  And initially in the ImageNet competition,
• 2:45
  so this idea of classification using large-scale label data.
• 2:48
  SPEAKER_4: But then it moved into the language space.
• 2:51
  I was at Google at the time and we were working on using deep learning
• 2:54
  to have better language understanding to make the search engine better was
• 2:57
  the basic headline idea.
• 2:59
  SPEAKER_4: And that led directly to the work around embeddings and so on and so forth.
• 3:03
  SPEAKER_4: But since that time it's been like a snowball gathering pace, right?
• 3:06
  SPEAKER_4: Because the results are so spectacular that people are pouring more resources into it.
• 3:10
  SPEAKER_4: I think like any significant technology development, it is a two-sided sword, right?
• 3:15
  SPEAKER_4: So I'm very worried about the risks that we now have around, say, cyber defense.
• 3:20
  SPEAKER_4: Because these models are now much better than humans at writing code.
• 3:23
  Therefore,
• 3:24
  it turns out the humans make lots of mistakes when they write code,
• 3:26
  and that's why we have bugs in code and security vulnerabilities.
• 3:28
  SPEAKER_4: And so, you know, the discussion and debate around Anthropic's model Mythos was quite real.
• 3:34
  SPEAKER_4: So you know, I do worry that most technologies are a two-edged sword.
• 3:38
  SPEAKER_4: But it's an exciting time.
• 3:39
  I'm a big fan of the ideas of a researcher in the 1970s called Roy Amara,
• 3:44
  which basically he said that things are overhyped in the short term
• 3:48
  and underappreciated in the long term.
• 3:49
  SPEAKER_4: And I felt that way about the web in the 1990s.
• 3:51
  In fact,
• 3:52
  famously John Doerr came to the Netscape offices and said to us,
• 3:55
  that this is massively overhyped in the short term,
• 3:57
  but it's actually going to be much more consequential in the long term.
• 4:00
  SPEAKER_4: And that is true.
• 4:00
  The web,
• 4:01
  the evolution to the web is still playing out,
• 4:03
  20 years after Netscape,
• 4:05
  I couldn't make an appointment with my dentist online, right?
• 4:07
  I'm sure I will be able to eventually,
• 4:09
  but IT upgrades just take a very, very long time to happen, right?
• 4:12
  SPEAKER_4: And so the AI upgrade for most of our economy will take decades to happen.
• 4:18
  Despite the current frenetic pace here in Silicon Valley,
• 4:20
  it feels like it's all happening overnight, but really it isn't, right?
• 4:23
  If you're,
• 4:23
  I don't know,
• 4:25
  a car rental company or something,
• 4:27
  it will take you many,
• 4:27
  many, many years to figure out how to apply this technology in the appropriate way.
• 4:30
  SPEAKER_3: Yeah, absolutely.
• 4:31
  SPEAKER_3: One of the interesting things about Apple was the early stance on privacy.
• 4:36
  SPEAKER_3: And I think that speaks to these concerns that you mentioned.
• 4:39
  How even though it's hard to see where things are going to go,
• 4:42
  it is clear that there are some initial concerns that we need to mitigate.
• 4:46
  And it'd be interesting to hear you talk about how did that position evolve
• 4:51
  and what are your perspectives now looking forward in terms of what you think
• 4:55
  the major risks we need to mitigate for and how can we mitigate given that the pace
• 5:00
  of change is so fast.
• 5:01
  Yeah, I mean Apple's obviously all always had a very strong stance on privacy
• 5:05
  and that's not just marketing.
• 5:07
  SPEAKER_4: It's like it comes from users' expectations, right?
• 5:10
  So people who buy Apple products actually really appreciate the fact that
• 5:14
  the company is going above and beyond to try and preserve their privacy
• 5:18
  and not have their data when it's not necessary.
• 5:21
  And so yeah,
• 5:22
  I think that because of the sort of frenetic pace of developing AI,
• 5:25
  there's this sense that like all of your personal data needs to be used
• 5:28
  to train these models.
• 5:29
  What we said two years ago was we would never use your personal data to train
• 5:32
  the core models because it's completely unnecessary.
• 5:35
  And so people who are copying and pasting their medical records into some chatbot,
• 5:39
  it's just not a good long-term solution.
• 5:42
  But it's more deeper than that,
• 5:43
  which is that the data that you probably want AI to operate
• 5:47
  over is actually your most personal data.
• 5:49
  So it's like,
• 5:49
  you know,
• 5:49
  your calendar entry or who you're seeing for dinner tonight or a message
• 5:53
  that your significant other just sent you.
• 5:55
  These are things that you shouldn't have to upload into the cloud in order
• 5:58
  to have AI operate over that data.
• 6:00
  And that's the sort of use cases that Apple highlighted
• 6:03
  at its developer conference this year are those cases where you're using AI,
• 6:07
  but you're using it to operate over your personal data,
• 6:09
  which is not shared with anybody else.
• 6:11
  Yeah,
• 6:12
  you were talking about the balance of investment and how all this money's pouring
• 6:15
  into massive data centers.
• 6:17
  But as you were saying,
• 6:17
  I think there's a significant balance to strike between what would run
• 6:21
  on your local device and what would run in the cloud.
• 6:23
  SPEAKER_4: Yeah.
• 6:25
  It's even more nuanced than that,
• 6:27
  which is that the largest models that we have as an industry,
• 6:30
  like call it like a trillion parameters.
• 6:32
  SPEAKER_4: It doesn't make economic sense to run those models at scale in the cloud, even, right?
• 6:37
  And so that's why you see all the frontier labs producing smaller models,
• 6:39
  flash models or quick models,
• 6:41
  whatever,
• 6:42
  because realistically you can't really afford to run more than
• 6:44
  a few hundred billion parameters at high QPSs.
• 6:48
  So it's really like that trade-off between model size,
• 6:51
  where it runs, and how much intelligence you need for a particular use case.
• 6:55
  One of my favorite use cases on the iPhone,
• 6:58
  which most people probably aren't aware of,
• 7:00
  is that there's a feature in the iPhone called focus modes where you can say,
• 7:04
  I'm in a meeting,
• 7:05
  so don't interrupt me and don't send me notifications for the next hour
• 7:08
  or something like that.
• 7:09
  But if your friend sends you a note saying that the basement is flooding,
• 7:13
  the LLM on your iPhone is smart enough to understand that the content of
• 7:16
  that message means it probably should break through the focus modes, right?
• 7:20
  SPEAKER_4: And actually tell you that, hey, that your basement's flooding.
• 7:22
  And that's just a really wonderful like quiet,
• 7:27
  ambient level of intelligence that's actually baked into the device in your pocket.
• 7:31
  And I get excited about use cases like that
• 7:33
  where things just generally become smarter in a very quiet way,
• 7:36
  as opposed to everything's now chat.
• 7:38
  It's funny,
• 7:39
  we talked to Marc Porat and he was saying that by 2030 or so,
• 7:43
  you'd have a superintelligence in your pocket,
• 7:45
  that would be capable of many different use cases: coach, doctor,
• 7:49
  companion,
• 7:50
  teacher, tutor,
• 7:52
  and that it could be very powerful but run locally,
• 7:54
  and given how good he was at seeing the future back in the 90s,
• 7:58
  I take his prognostications, seriously.
• 8:01
  SPEAKER_0: Is that something that the two of you see coming as well?
• 8:04
  SPEAKER_4: Yeah, I do think the tutor use case is really interesting one.
• 8:07
  SPEAKER_4: So personalized education.
• 8:08
  Many years ago,
• 8:09
  a co-founder of my last startup, Danny Hillis, wrote an essay called "Aristotle."
• 8:13
  And the premise of the essay was that Alexander the Great had the best teacher ever
• 8:16
  because Aristotle knew everything that there was to be known
• 8:19
  and he knew what Alexander knew.
• 8:20
  SPEAKER_4: And so they could sit down of an afternoon and have a personal tuition.
• 8:24
  SPEAKER_4: And so yes, I think the technology is leading us to a place where we will have that.
• 8:27
  And I was just talking to somebody the other day about the implications
• 8:30
  for education are really quite profound.
• 8:32
  SPEAKER_4: But again, it will probably take longer to happen than people think.
• 8:35
  SPEAKER_4: It's not going to happen overnight.
• 8:36
  Schools,
• 8:37
  educators,
• 8:38
  universities,
• 8:39
  professors are all struggling with what does AI even mean in
• 8:42
  the educational context, right?
• 8:44
  SPEAKER_4: So but yes, these things will change, but they will change, slowly.
• 8:48
  SPEAKER_4: People like to talk at in AI circles about these models are jagged.
• 8:52
  SPEAKER_4: And what they mean by that is that they work really, really well until they don't.
• 8:55
  I mean, all of the consumer products that are powered
• 8:57
  by AI today all have disclaimers saying check the results.
• 8:59
  Like when was the last time you saw a consumer product that has that disclaimer
• 9:03
  that said this might be complete nonsense?
• 9:04
  SPEAKER_4: Right?
• 9:05
  SPEAKER_4: But that's just the nature of the technology today is that it's not very well grounded.
• 9:09
  In the legal field,
• 9:10
  which I'm familiar with,
• 9:12
  there's an issue that's come up with more import maybe than in other areas,
• 9:17
  the question of hallucinations.
• 9:18
  SPEAKER_1: I'm wondering do you see certain professions as particularly threatened by AI in the near term?
• 9:24
  Or is your prediction that across the board these things are going
• 9:29
  to take much longer than the most pessimistic predictions.
• 9:32
  Well, I do think things take longer than people imagine to fundamentally change,
• 9:37
  but that doesn't mean that they don't fundamentally change.
• 9:39
  SPEAKER_4: I would rather use the analogy of like, power tools don't make carpenters go away.
• 9:43
  SPEAKER_4: Just the way you do carpentry is different.
• 9:45
  SPEAKER_4: There's still a craft to be done there.
• 9:47
  SPEAKER_4: And if humans are completely removed from the loop, then you will probably get very bad results.
• 9:52
  SPEAKER_4: You mentioned law, right?
• 9:53
  There's obviously been lawsuits and models hallucinating and producing citations
• 9:57
  that didn't exist and so on and so forth.
• 9:58
  SPEAKER_4: And I hear people say glib things like, you know, LLMs are going to do your taxes for you.
• 10:02
  SPEAKER_4: Well, LLMs can't reliably count.
• 10:04
  SPEAKER_4: And so therefore, if they did your taxes, they would make a mistake and you'd get fined.
• 10:07
  SPEAKER_4: So that's clearly not true.
• 10:08
  What will probably happen is somebody will create a tax tool and then
• 10:13
  the LLM will drive the tool and the tool will do the arithmetic, right?
• 10:17
  So in the long run, yes,
• 10:18
  maybe your tax the way tax preparation is done will be fundamentally different,
• 10:21
  but it requires invention that's not here yet.
• 10:24
  Yeah, Steve,
• 10:25
  when you were working on Orange's guardrail programs
• 10:28
  and how does safety get designed into these models up front,
• 10:32
  I thought,
• 10:33
  between the two of you,
• 10:33
  the debate over regulation,
• 10:35
  too much,
• 10:36
  too little,
• 10:36
  what stifles innovation,
• 10:38
  what kind of constraints actually are better for innovation?
• 10:41
  Where were the two of you on regulation and on architecture in terms of trying
• 10:46
  to build in safety guard rails like Telescript had.
• 10:49
  SPEAKER_3: Yeah.
• 10:49
  SPEAKER_3: I'll start and JG be great to hear what you think too.
• 10:52
  On European regulation or Orange operating in 26 countries,
• 10:56
  many of those in Europe,
• 10:57
  our perspective was there were really good parts about
• 11:00
  the early European AI Act regulation in terms of requiring the model providers
• 11:05
  to say how much CO2 was generated in the consumption of the model and the operation
• 11:09
  of the model.
• 11:10
  So those were things that we could pass along to our employees so
• 11:13
  that they could make informed decisions about how they were using models.
• 11:16
  A lot of the challenge first of all was using things in
• 11:19
  an economically responsible way and an environmentally responsible way.
• 11:23
  So the regulation giving us the ability to be able to make those kinds of decisions
• 11:27
  as opposed to the models being a black box in terms of their CO2.
• 11:30
  SPEAKER_3: Those were useful.
• 11:31
  The notion of having high risk use cases where you say these are the kinds of things
• 11:36
  that you need to be very careful about.
• 11:37
  SPEAKER_3: You need to be careful with the bias of the model.
• 11:39
  SPEAKER_3: You have to be careful about the kinds of decisions that you take with the models.
• 11:43
  Just like JG was saying with tradespeople,
• 11:45
  the notion that the human is still actively evaluating the outputs of the model
• 11:50
  before you take an action.
• 11:51
  SPEAKER_3: Those are all really useful.
• 11:53
  I think where we saw challenges in over-regulation was the notion that if you use
• 12:00
  an open source or open weight model,
• 12:02
  then the person who even made small customizations of that model then had
• 12:05
  to adopt all of the liability of the original model provider.
• 12:09
  And so what that did is it just swamped a lot of the early European open source
• 12:14
  and open weight model innovation and favored the large companies who had teams
• 12:18
  of lawyers to manage all the complexity.
• 12:21
  SPEAKER_3: So that I thought was an overreach on regulation.
• 12:24
  And in fact that's one of the sections that's being actively reconsidered now
• 12:28
  because it really did stifle not just a lot of the investment in European AI,
• 12:34
  but also a lot of the innovative activity as well.
• 12:37
  A lot of the entrepreneurs that we were working with initially at Orange decided
• 12:41
  to shift strategy and instead of working on tools and models,
• 12:44
  they started working on simpler things.
• 12:46
  SPEAKER_3: So I think that that's not helping Europe to keep pace with the United States.
• 12:52
  It's also not helping Europe keep pace with China,
• 12:54
  where a lot of the innovation there in open weight models has been
• 12:58
  because they've been supply constrained on compute.
• 13:01
  And so since they can't get access to the latest NVIDIA infrastructure,
• 13:05
  they're focusing and doing an incredible job at making models much smaller,
• 13:09
  much more capable.
• 13:10
  Which we thought was going to be one of the great opportunities
• 13:13
  for European companies because they were so full of incredibly well-educated people
• 13:17
  with deep mathematics expertise.
• 13:19
  SPEAKER_3: So for me, Europe really did overstep.
• 13:23
  But the notion that you need to have penalties both for the model providers and
• 13:28
  for people misusing the models, I think those are right.
• 13:31
  SPEAKER_3: So but it's striking the balance is hard.
• 13:33
  SPEAKER_4: Yeah, I think appropriate regulation is necessary but hard to create, as you point out.
• 13:39
  You know,
• 13:39
  the Europeans,
• 13:40
  they had some head scratching stuff there,
• 13:42
  like they tried to define thresholds about the amount of compute that was used
• 13:45
  to create a model, like 10 to the 23 flops or something.
• 13:48
  SPEAKER_4: It's like that seems kind— I don't know where.
• 13:51
  However,
• 13:51
  the use of models for say high risk activities like facial recognition
• 13:54
  or credit scoring or something,
• 13:56
  these things,
• 13:56
  should be regulated and whether you need an AI law to do it or you can do it
• 13:59
  with existing regulations, It's a different question.
• 14:02
  The problem is,
• 14:02
  of course,
• 14:03
  regulating such a fast moving industry,
• 14:05
  you tend to either overdo it or miss the mark, right?
• 14:08
  And I think in the US there's been more of a belief of let's just let this develop
• 14:14
  a little bit more before we decide what we really actually care about.
• 14:17
  SPEAKER_4: Now, what I think is more important even than regulation is transparency.
• 14:21
  So parts of the European AI Act that I like are about model cards,
• 14:25
  disclosing the data used to train models, things like this, right?
• 14:28
  I know for a fact that all of the big Frontier labs,
• 14:31
  including the ones I've worked at, have huge safety teams, right?
• 14:33
  SPEAKER_4: I actually don't like the term safety.
• 14:35
  It's really more about a quality aspect,
• 14:37
  but nonetheless, some of the quality aspects are things like being harmful, right?
• 14:42
  SPEAKER_4: So in the field this is called safety.
• 14:45
  SPEAKER_4: But basically the people who work in these teams are very well meaning, right?
• 14:48
  SPEAKER_4: They actually care about making these models be humane, and not be inappropriate.
• 14:53
  And not,
• 14:54
  there are with some exceptions,
• 14:55
  there's one or two models that are deliberately designed to be inappropriate.
• 14:58
  SPEAKER_4: So these teams are good.
• 14:58
  SPEAKER_4: The problem is that they're marking their own homework, right?
• 15:01
  So what you need is an independent organization that can score models
• 15:04
  and say these models are appropriate for kids or these models are,
• 15:07
  harmful in certain ways.
• 15:08
  And so you're starting to see the infrastructure getting built out that way,
• 15:12
  whether it's you know standards organizations,
• 15:13
  say in the US government,
• 15:14
  or the companies themselves trying to standardize system cards and model cards
• 15:19
  to disclose things.
• 15:20
  But if you look at the model card for one,
• 15:21
  large model like say Anthropic Claude or ChatGPT,
• 15:24
  there's a lot of words there,
• 15:26
  but there's not a lot of actual concrete facts about
• 15:28
  which benchmarks were used internally.
• 15:30
  And so I think what I would like to see is more transparency in the industry
• 15:34
  and more independent benchmarks for these models so that we can have confidence
• 15:38
  that they're getting better.
• 15:41
  SPEAKER_1: I'm wondering what the inherent danger of AI is.
• 15:44
  I mean, a lot of people talk about its dangers in terms of outcomes,
• 15:50
  but what is it at the core of AI that makes it different as a technology in terms
• 15:55
  of its risks?
• 15:56
  SPEAKER_4: Well, I have an opinion about that, which is that we don't actually understand how it works.
• 16:00
  So we have,
• 16:01
  the empirical methods of generating models that contain,
• 16:05
  let's call it, 400 billion parameters.
• 16:08
  SPEAKER_4: What are these parameters?
• 16:09
  They're basically floating point numbers that were created by the process
• 16:11
  of training, pre-training and post-training, these models.
• 16:15
  SPEAKER_4: From a scientific point of view, we cannot explain why these models generalize.
• 16:20
  SPEAKER_4: And we don't have a good mathematical understanding of that.
• 16:23
  SPEAKER_4: We do know a few interesting things.
• 16:25
  Like, for example,
• 16:26
  there are experiments that have shown that you can just remove 90% of the parameters
• 16:29
  and the model still is as accurate.
• 16:32
  SPEAKER_4: We just don't know which 90% to remove, right?
• 16:34
  And so you wouldn't want to build a bridge with this understanding of
• 16:38
  the engineering here, right?
• 16:40
  So that's one risk is we're going headlong into using these models for all kinds
• 16:43
  of important things in our society,
• 16:46
  but we don't have a good brass tacks mathematical understanding of why
• 16:49
  the models work as well as they do.
• 16:50
  SPEAKER_4: So right now it's a pretty much purely empirical industry.
• 16:55
  SPEAKER_0: Yeah, it's market driven.
• 16:56
  SPEAKER_0: And you know, markets work when they work, but they often fail.
• 17:01
  SPEAKER_0: And both standards organizations and government bodies are struggling to keep up.
• 17:07
  SPEAKER_0: To me it's the issue of, you build a car without seat belts and then retrofit it later.
• 17:11
  SPEAKER_0: Or do you build cars with seat belts?
• 17:13
  SPEAKER_0: What would incent industry to build the seatbelts now.
• 17:17
  SPEAKER_4: Yeah, it's a great analogy, Mike.
• 17:19
  SPEAKER_4: Like, we used to drive cars without seat belts, right?
• 17:21
  SPEAKER_4: And then we mandated them.
• 17:22
  SPEAKER_4: We used to let people smoke in public places, now that's mostly banned around the world.
• 17:26
  SPEAKER_4: So yeah, it's a good analogy.
• 17:28
  I think there needs to be guardrails,
• 17:29
  there needs to be basic things that we all agree on that these models should not do.
• 17:34
  SPEAKER_4: And one of those things I've already alluded to, I think, is cyber offense.
• 17:37
  SPEAKER_0: Yeah, but how do we incent industry?
• 17:38
  SPEAKER_0: Because again, right now we're letting the market drive these decisions.
• 17:42
  SPEAKER_0: It's not being driven by government and it's not being driven by industry, self-government.
• 17:47
  SPEAKER_0: So what will drive the industry to do that as opposed to just leap ahead
• 17:53
  SPEAKER_4: I think transparency and metrics.
• 17:55
  SPEAKER_4: So, one of the organizations that I'm advising is a nonprofit group called Common Sense Media.
• 17:59
  SPEAKER_4: And they're actually funded in part by the frontier labs.
• 18:02
  And their goal is to be able to create an independent benchmark
• 18:05
  of certain categories of safety for models,
• 18:07
  particularly around appropriateness for children.
• 18:09
  So I think that,
• 18:10
  just like in America,
• 18:10
  we have Underwriters Laboratories (UL) that gives you some you know,
• 18:13
  assurance that the electrical device is not gonna catch fire.
• 18:15
  SPEAKER_4: I think we need something similar for this world of models.
• 18:18
  Maybe it's not a mandated seatbelt like that,
• 18:21
  maybe not the correct analogy, but maybe it's a mandated nutrition label.
• 18:24
  SPEAKER_0: Yeah, Steve, what was Orange doing in that space?
• 18:27
  A lot of the techniques that JG described were the things that Orange used,
• 18:31
  cards to describe what was inside the model,
• 18:34
  lots of testing before deployment to ensure that the models didn't show bias,
• 18:38
  ensuring that humans were in the decision loop before you took action based on
• 18:42
  the outputs of the models,
• 18:43
  and having a methodology where we would define these are the kinds
• 18:46
  of higher risk use cases, where we had to be very particularly careful about that.
• 18:50
  SPEAKER_3: So Orange was particularly good about that.
• 18:52
  I think one of the challenges is,
• 18:54
  as you were pointing out,
• 18:55
  Mike, not all of the even the US companies have the same safety motivations.
• 19:02
  SPEAKER_3: Among the large model providers, you see a lot of that.
• 19:05
  You see a lot of funding of the kinds of groups that JG was mentioning,
• 19:08
  and people self-policing.
• 19:10
  SPEAKER_3: But I don't think that's universally true.
• 19:13
  And when you consider the fact that the Chinese ecosystem is highly unregulated,
• 19:18
  and there seems to be the beginning of a split between the Western ecosystems and
• 19:25
  the Eastern ecosystems.
• 19:26
  I think is just gonna continue to be true,
• 19:28
  that you're gonna have continuing separation as Chinese hardware for GPU gets better
• 19:33
  and better,
• 19:34
  you're gonna see an even greater divergence between not only the hardware
• 19:37
  and software systems, but the regulation regimes too.
• 19:39
  SPEAKER_3: And so I—
• 19:40
  SPEAKER_4: Do you mind if I disagree with that a little bit?
• 19:42
  SPEAKER_4: I mean, I think the Chinese have the most regulated AI ecosystem.
• 19:46
  SPEAKER_4: They have extensive, laws and rules about what models may and may not do.
• 19:51
  They require companies to have a license to release a model,
• 19:55
  and that license involves extensive testing by
• 19:58
  at least seven different organizations in the Chinese government, right?
• 20:03
  SPEAKER_4: So it's a pretty draconian world.
• 20:06
  And, what I've read in the press recently is that some of the main researchers
• 20:10
  from their frontier labs are now prohibited from leaving the country.
• 20:14
  So China has a very,
• 20:16
  very unique point of view on how they want AI to be developed in their ecosystem.
• 20:20
  SPEAKER_4: And it's very locked down and very, very controlled by the government.
• 20:23
  Yeah,
• 20:24
  I think my concern there is that the open weight model ecosystem is superior even to
• 20:30
  the Western open weight model ecosystem today in China.
• 20:33
  And one of my concerns are bad actors and people using open weight models outside
• 20:38
  of commercial products, for example, in really negative ways.
• 20:41
  And so even though in the Western world more and more it's about commercial models
• 20:45
  and closed models where we can have strong regulatory oversight,
• 20:48
  my concern is that... Two concerns.
• 20:50
  One is that a lot of the open weight models coming out of China being used
• 20:54
  by bad actors can lead to really, really negative outcomes.
• 20:57
  SPEAKER_3: It'll be very difficult to enforce.
• 20:59
  And my other concern is the feeling of a kind of an arms race,
• 21:02
  where the concern being that maybe the government is doing things for military use,
• 21:07
  for example, in China and the West isn't.
• 21:09
  And does that create a sense of an arms race where we're worried
• 21:13
  that they may have capabilities that the West doesn't.
• 21:15
  And that leads to both sides in fact developing things outside
• 21:19
  of regulatory constraint that they shouldn't.
• 21:21
  SPEAKER_3: Similar to the kind of dynamics that we saw in the early atomic weapons programs.
• 21:27
  I wanna take a step back time wise, JG,
• 21:30
  and ask you about early in your career,
• 21:34
  both the career trajectory before, during, and after General Magic.
• 21:39
  And also a central thing that Mike and I have been examining is,
• 21:43
  for lack of a better phrase,
• 21:44
  the ethos of innovation,
• 21:46
  the sometimes subconscious framework that people had,
• 21:51
  exemplified by the Macintosh as a tool for self-empowerment,
• 21:57
  the General Magic vision of anytime anywhere communication as something
• 22:01
  that was liberating.
• 22:03
  So both with respect to the trajectory of your career and also with the philosophy
• 22:09
  of design,
• 22:09
  for lack of a better phrase,
• 22:10
  that you carried with you early on and how that may have evolved,
• 22:14
  if at all, over the course of your career.
• 22:16
  SPEAKER_1: I'd love to hear about that.
• 22:17
  SPEAKER_4: Yeah, I love this question.
• 22:19
  So I think one of the things that is not well understood is the role of values
• 22:24
  and values based motivation in the creation of technology in Silicon Valley.
• 22:29
  There's a really great book about this that was written by John Markoff from
• 22:32
  the New York Times called What the Dormouse Said.
• 22:34
  And it chronicles the history of early Silicon Valley in the 60s and the 70s,
• 22:38
  up through the 80s and the creation of the Macintosh.
• 22:41
  And makes a very,
• 22:42
  very strong case that most of the innovations that led us to
• 22:45
  where we are now were motivated by people who were trying to do something
• 22:50
  for humanistic reasons.
• 22:53
  SPEAKER_4: The whole notion of a personal computer was a heretical notion.
• 22:56
  SPEAKER_4: You had to have the vision that computing should not be like a utility like the power company.
• 23:01
  SPEAKER_4: It should instead be, a very personal thing.
• 23:04
  General Magic was another version of this,
• 23:06
  that there should be a pocket computer that you kept with you.
• 23:09
  Marc talked about it being like a piece of jewelry or something that you would have
• 23:12
  with you every day, right?
• 23:14
  SPEAKER_4: That is a vision completely devoid of whether the technology is available to create that vision.
• 23:19
  And that's how we got the iPhone,
• 23:21
  that's how we got the internet, that's how we got search engines, right?
• 23:25
  And so I think that's one of the bits of magic about Silicon Valley is
• 23:29
  that people go off and do things for the idea of doing it because they're passionate
• 23:32
  about inventing the future.
• 23:34
  I used to manage,
• 23:35
  well Vint Cerf was one of my employees when I was at Google and he used
• 23:38
  to say things like when we designed the internet,
• 23:40
  we designed it so you didn't need permission to connect.
• 23:43
  SPEAKER_4: And why did they care?
• 23:44
  It was because at the time you needed permission from AT&T to connect a telephone
• 23:48
  to their national network, right?
• 23:49
  SPEAKER_4: Famously the 1984 ad that Steve Jobs ran was like, a young woman throwing, a hammer at IBM.
• 23:55
  SPEAKER_4: Right.
• 23:55
  And so there's this humanistic,
• 23:58
  idealistic motivation behind most of the really important inventions,
• 24:03
  at least in my career.
• 24:04
  SPEAKER_0: But that has changed, hasn't it, John?
• 24:06
  I mean, I see the valley as having grown pretty rancid over the last decade with
• 24:10
  the emergence of the tech broliarchy,
• 24:12
  their quest for wealth and power,
• 24:14
  their disregard for humanity,
• 24:17
  their disregard for decency, their disregard for civility, their alliance with—
• 24:22
  SPEAKER_4: In some parts of the valley, yes, but not all of it.
• 24:25
  SPEAKER_0: You don't think it's the dominant ethos now?
• 24:27
  How do we go from a founder that Markoff characterized as being driven
• 24:32
  by humanistic 60s values,
• 24:34
  the values of the counterculture,
• 24:36
  to somebody like Musk, who's driven by Gilded Age values of power and wealth.
• 24:41
  I think that's a narrative that should be written,
• 24:44
  but I also know lots of idealistic people who are working on machine learning
• 24:47
  for drug discovery for material science because they want to solve climate change.
• 24:51
  SPEAKER_4: They want to solve the Pacific garbage patch, right?
• 24:54
  SPEAKER_4: There are still lots of idealistic technologists in the valley.
• 24:57
  I agree with you that a lot of the money now seems to be flowing to a certain subset
• 25:02
  of technologists.
• 25:03
  But even in the case of Musk, who,
• 25:05
  regardless of what you think of him,
• 25:06
  his motivation is for us to be interplanetary, supposedly, right?
• 25:10
  SPEAKER_4: I mean, that's not a completely dystopian vision, right?
• 25:14
  SPEAKER_4: I spent 10 years on the board of the SETI Institute.
• 25:17
  SPEAKER_4: It's a nonprofit scientific institute started by Carl Sagan.
• 25:20
  And the most likely explanation for why we have no evidence of life in
• 25:23
  the universe is it doesn't last very long.
• 25:25
  SPEAKER_4: Right.
• 25:25
  SPEAKER_4: That's Fermi's paradox.
• 25:26
  SPEAKER_4: And so the implications of that for us as a species are not good, right?
• 25:32
  SPEAKER_3: Yeah.
• 25:32
  SPEAKER_3: And actually Mike, I have a similar perspective to JG.
• 25:35
  When I think about AI,
• 25:37
  for me the thing that's most exciting is the notion that you can give people
• 25:41
  the power of a programmer with natural language.
• 25:45
  SPEAKER_3: So it just transforms the ability for anyone to use the full power of a computer.
• 25:50
  And it actually reminds me of our late coworker Bill Atkinson and his vision
• 25:54
  at Apple with HyperCard, and for me that's what gets me really excited.
• 26:00
  The notion that not only is it a means for people to become better educated
• 26:04
  in natural language, but actually to be able to create in natural language.
• 26:08
  And I don't know, JG,
• 26:09
  I'd be interested to hear you talk a little bit about your perspective on
• 26:12
  that transition from having to be this specialized expert in these obscure languages
• 26:17
  to be able to take full power of a computer versus just speaking to it or talking
• 26:20
  to it.
• 26:21
  Yeah, I often think about it in this way,
• 26:24
  which is that many products that come along go in this sort of cycle where
• 26:29
  the product didn't exist, my go-to example is a fax machine, right?
• 26:33
  So somebody invented a fax machine,
• 26:35
  it was very expensive,
• 26:36
  a few people had a fax machine,
• 26:37
  then people aspired to have a fax machine,
• 26:39
  then you had to have a fax machine to talk to all the other people with
• 26:41
  a fax machine,
• 26:42
  and then you had a fax machine,
• 26:43
  but didn't use it, and now nobody has a fax machine, right?
• 26:46
  SPEAKER_4: And that cycle, usually takes like 30 years.
• 26:48
  SPEAKER_4: And most products that are invented go through that boom and bust cycle.
• 26:53
  There are a few technologies that come along from time to time which don't follow
• 26:57
  that pattern at all.
• 26:57
  SPEAKER_4: So the internet, you wouldn't want to live your life without the internet.
• 27:00
  SPEAKER_4: You wouldn't probably want to live your life without something like a search engine.
• 27:02
  SPEAKER_4: You wouldn't want to live your life without something like a smartphone.
• 27:05
  And I feel like these large language models and AI's ability
• 27:08
  to let people code is kind of like a spreadsheet moment.
• 27:10
  SPEAKER_4: You wouldn't want to live without tools like spreadsheets.
• 27:13
  SPEAKER_4: And so as you say, now everybody can code.
• 27:15
  SPEAKER_4: It will take longer, as I predicted, for people to adopt it than you might think.
• 27:18
  SPEAKER_4: It is really interesting from a societal level to me at least to see the backlash with AI.
• 27:22
  You know,
• 27:23
  students at a commencement booing speakers who even mentioned the word AI
• 27:27
  because they think they find it dystopian and threatening
• 27:31
  to their own career prospects and just anti-humanity in some sense.
• 27:35
  SPEAKER_4: I think that's an overreaction.
• 27:37
  SPEAKER_4: Your average person is pretty skeptical about this technology.
• 27:39
  SPEAKER_4: They're not entirely convinced it's a good idea.
• 27:41
  And right now we're of course in the hype phase where people are plugging AI
• 27:44
  into everything, right?
• 27:46
  But I do agree that this is a transformative technology which will be
• 27:50
  with us forever and will help people do really, really useful things.
• 27:54
  And to be more technical for a minute,
• 27:57
  there are things that used to be open research problems like summarization,
• 28:01
  taking a long document and making a short document,
• 28:03
  which are now essentially a solved problem that's a big deal because it means
• 28:06
  that all of the books on your bookshelf,
• 28:08
  you can just get cliff notes for them, right?
• 28:10
  That's a very,
• 28:10
  very powerful capability when you consider that we only have so many hours in
• 28:14
  the week to pay attention to anything, right?
• 28:16
  SPEAKER_3: Yeah.
• 28:16
  SPEAKER_3: Yeah.
• 28:17
  What's your sense about the ability for us to reach a human brain class intelligence
• 28:25
  with the current architectures?
• 28:27
  Do you think we can get there by just being better and better with transformers,
• 28:29
  or do you think we have to do something radically different?
• 28:32
  SPEAKER_4: Yeah, that's a great question.
• 28:33
  SPEAKER_4: I think we don't know the answer to that question.
• 28:36
  There's obviously lots of people smarter than me who've raised billions of dollars
• 28:40
  to produce models that are not LLMs.
• 28:42
  SPEAKER_4: So Yann LeCun would be one example, David Silver from DeepMind.
• 28:46
  There is a very well funded effort to try and prove that transformers are not
• 28:51
  the end of the line here.
• 28:52
  And people call these things world models and they take a lot of inspiration
• 28:55
  from how children learn in an embodied world.
• 28:58
  They learn from less examples and less data
• 29:00
  than these massive systems are reading every token on the web.
• 29:04
  SPEAKER_4: And they seem to be able to learn from what machine learning people call single shot learning.
• 29:09
  SPEAKER_4: So learning from one example.
• 29:10
  An example I used in an interview a few years ago is
• 29:13
  when my daughter was very young,
• 29:14
  like five,
• 29:14
  six years old,
• 29:16
  we went to a Christmas fair thing in Silicon Valley called the Dickens fair.
• 29:22
  SPEAKER_4: And they take the Cow Palace and they basically dress it up like Victorian London.
• 29:25
  SPEAKER_4: You go and people are wearing costumes and so forth.
• 29:27
  And so I'm taking my daughter to this Christmas fair,
• 29:30
  and somebody rode past on a penny-farthing bicycle,
• 29:34
  and she had never seen one before.
• 29:35
  SPEAKER_4: And she looks up at me and she says, That's a weird bicycle, Dad.
• 29:38
  SPEAKER_4: And I was like, whoa, how does that work?
• 29:41
  SPEAKER_4: Like, how does the human brain know how to generalize, right?
• 29:44
  And in fact,
• 29:44
  Douglas Hofstadter,
• 29:45
  who was very famous in the 1980s has an entire book about this called Analogy,
• 29:50
  where he claims that analogy is the fuel and fire of true intelligence.
• 29:54
  SPEAKER_4: And so these LLMs are not doing analogizing and they're not learning from single shot examples.
• 29:59
  And so very prominent researchers like Yann LeCun will make the argument
• 30:02
  that you will hit a dead end.
• 30:03
  There's a different technical argument to be made,
• 30:05
  which comes from the folks who study causality,
• 30:08
  mostly students of people like Judea Pearl,
• 30:10
  who claim that these models don't know what they're doing,
• 30:13
  and therefore they can't reason about what they're doing,
• 30:15
  and therefore there's a sort of mathematical limit
• 30:17
  to how much reasoning they can do.
• 30:19
  My observation on this and managing lots of these research scientists is
• 30:22
  that there's a lot of tribalism in both computer science but
• 30:25
  in machine learning generally.
• 30:26
  There's the Bayesian people,
• 30:27
  and there's world model people,
• 30:28
  and there's the everything can be solved by a 10-layer neural network people.
• 30:32
  So these are fascinating working theories that people have about something
• 30:36
  that we don't have a good empirical theory for why it's working as effectively
• 30:38
  as it—
• 30:39
  Yeah, I suspect that the next major evolutions will be an ensemble
• 30:45
  of those different approaches.
• 30:46
  And I agree with you that the fact that the academics are so unwilling to think
• 30:51
  that there may be strengths and weaknesses to many approaches and they need
• 30:53
  to collaborate, I think that is going to slow us down.
• 30:56
  I'll say something even more opinionated about this,
• 30:59
  because I've been talking to a lot of investors recently and
• 31:02
  with all this huge build out that's going on.
• 31:04
  SPEAKER_4: So Sam Altman likes to say the models are as dumb as they'll ever be, right?
• 31:08
  SPEAKER_4: What he means by that is that the models will just get better.
• 31:10
  SPEAKER_4: So like if you're impressed by them now, just wait a few years and they'll be even better.
• 31:14
  SPEAKER_4: But I would say something different, which is the models are the most expensive they'll ever be.
• 31:17
  SPEAKER_4: Right.
• 31:18
  SPEAKER_4: These models are fully auto-regressive.
• 31:20
  SPEAKER_4: What does that mean?
• 31:20
  It means that you take all of the input tokens and you multiply it by all of
• 31:23
  the parameters in the model.
• 31:24
  So if you've got a 100-billion-parameter model and you've got
• 31:26
  a 100,000 input tokens,
• 31:28
  you're doing 100,000 times 100 billion floating point multiplications,
• 31:32
  which is why all these tens of billions of dollars are being spent on hardware
• 31:36
  and why NVIDIA is the world's most valuable company right now.
• 31:38
  I would predict that if we're having this conversation in three years' time,
• 31:41
  somebody will have cut that by 100x or 1,000x by coming up with different technique
• 31:45
  because it is literally the dumbest thing we could possibly be doing right now and
• 31:48
  the most expensive.
• 31:49
  SPEAKER_3: And we have a model, right?
• 31:50
  SPEAKER_3: The human brain operates on a very little amount of power.
• 31:54
  SPEAKER_3: So we know that it can be done with low power.
• 31:56
  SPEAKER_3: That's right.
• 31:57
  SPEAKER_4: On a ham sandwich, as we like to say.
• 32:01
  SPEAKER_1: You, both of you, have seen AI up close, have a deep understanding of it.
• 32:06
  SPEAKER_1: What tendencies do you think it promotes in terms of human thought and human behavior?
• 32:12
  SPEAKER_3: Yeah, for me, it helps me be far more creative.
• 32:15
  SPEAKER_3: So when I get stuck on an idea, the best thinking partner are AI models.
• 32:21
  SPEAKER_3: To take me new directions, to challenge my thinking.
• 32:24
  SPEAKER_3: I think that's incredibly powerful.
• 32:27
  SPEAKER_3: I also love being able to create things much faster than I could without the tools.
• 32:32
  SPEAKER_3: So for me, those are two really, really fundamental breakthroughs.
• 32:35
  I think that for me the challenge is that the models today certainly you need
• 32:40
  to be able to check their work because they make hallucinations
• 32:44
  and they make other kinds of errors.
• 32:45
  And I think one of the challenges will be not in the next five years,
• 32:50
  but in a generation,
• 32:52
  how will we encourage and incent young people to learn things the hard way so
• 32:59
  that they develop the cognitive skills to be able to direct and correct the models
• 33:07
  of the future?
• 33:07
  SPEAKER_3: And that for me is an unanswered question.
• 33:10
  SPEAKER_3: Certainly I think in the near term, the majority of the benefit is positive.
• 33:14
  SPEAKER_3: It helps people be a lot more effective with their jobs, learn faster, create faster.
• 33:18
  SPEAKER_3: There will be some job disruption as people have to learn these new tools and adapt to them.
• 33:23
  SPEAKER_3: But for me, this kind of generational question is unanswered.
• 33:26
  Yeah, I think the question of how we educate our young people
• 33:29
  to have better cognitive skills because of the availability of these technologies is
• 33:34
  a fundamental question.
• 33:36
  But to tie it back to General Magic and my career over the last 35 years,
• 33:40
  every time one of these things comes along,
• 33:43
  for example, I spent many, many years working closely with the Wikipedia people.
• 33:47
  And if you recall back then,
• 33:48
  there was a very strong sense from some corners of academia that
• 33:52
  a crowdsourced encyclopedia couldn't be better than Encyclopedia Britannica.
• 33:56
  And it was only when people actually did like the hard work
• 33:59
  to actually measure them side by side that they showed that actually
• 34:01
  the physics article while not perfect were actually better in Wikipedia
• 34:05
  than they were in Britannica.
• 34:06
  SPEAKER_4: Right.
• 34:06
  SPEAKER_4: But there was this initial feeling that we need experts to tell us how to do this stuff, right?
• 34:10
  SPEAKER_4: And some guy in a camper van can't be an expert on, some famous artist or something.
• 34:15
  SPEAKER_4: And they were just wrong.
• 34:15
  SPEAKER_4: Right.
• 34:16
  And so I am not only a technological optimist,
• 34:19
  but I'm also a libertarian in that sense,
• 34:21
  which I think that the technology will go wherever society wants to take
• 34:24
  the technology.
• 34:25
  SPEAKER_4: And society will find its own ways to push back on inappropriate uses of certain technology.
• 34:29
  But the reality is we live in a world where,
• 34:32
  most people who have access to a data plan and a mobile device have access to
• 34:35
  the sum of human knowledge.
• 34:36
  SPEAKER_4: That's not a small thing.
• 34:37
  And it's not something we'd want to go back to not having anymore,
• 34:40
  and you'd have to go to the library to go look up something.
• 34:42
  SPEAKER_4: Right.
• 34:43
  SPEAKER_4: So.
• 34:44
  SPEAKER_4: That seems like an unalloyed good.
• 34:45
  And in fact,
• 34:45
  the last stat I saw from the UN is more people have access to a mobile device
• 34:49
  than have access to proper sanitation.
• 34:51
  And that's them choosing with their pocketbooks,
• 34:53
  in, developing countries that they really want to have access to that data.
• 34:57
  SPEAKER_4: Right.
• 34:58
  SPEAKER_4: And so I think AI will be like that.
• 35:00
  SPEAKER_4: It won't be something that people will say, I don't want.
• 35:02
  SPEAKER_4: It'll be something that they'll end up depending on.
• 35:04
  And so then it was really important that we develop it in a way
• 35:07
  that is actually appropriate for the amount of use it's going to get.
• 35:10
  SPEAKER_4: I mean OpenAI says 900,000,000 people use their product weekly.
• 35:14
  SPEAKER_4: That's a significant portion of the planet.
• 35:16
  Yeah, and I think to your point earlier about how long is it gonna take AI
• 35:20
  to fully transform industries,
• 35:22
  the pace with which AI can be used to transform our own education systems,
• 35:27
  I think is one of the really crucial bottlenecks.
• 35:29
  John, does the advent of AI,
• 35:31
  and you've been living with it for a lot longer than other people,
• 35:35
  has it shifted the way you think about how to get work done,
• 35:39
  or shifted the way you think about what the interesting problems or the things just
• 35:44
  in daily life as a human being to focus on or pay attention to?
• 35:48
  SPEAKER_1: Has there been a shift or you don't feel that the tool has reframed your thinking?
• 35:52
  SPEAKER_4: I definitely turn to AI to answer questions or to get more nuanced perspectives on something.
• 35:59
  And so in that sense,
• 36:00
  is sort of as a replacement for a search engine, it's really quite remarkable.
• 36:05
  And as somebody who spent a long time working on search at Google,
• 36:07
  that's actually quite a big deal and that company has been navigating that change.
• 36:11
  But one thing I will say is that while you can use AI to help you be more creative,
• 36:16
  we don't yet have any evidence,
• 36:17
  or we have very, very scant evidence that AI can actually generate novel thoughts.
• 36:22
  SPEAKER_4: Right.
• 36:22
  So it's really regurgitating the sum total of what humans wrote down on the web
• 36:26
  and books and periodicals and everything else.
• 36:29
  And some of these models are,
• 36:30
  of course,
• 36:30
  multimodal and can do amazing things like reframe a photo that you already took.
• 36:35
  So there's lots of reasons to be excited as a technologist,
• 36:38
  but there's no evidence yet that AI,
• 36:41
  even though it could tell you what the 10 hardest problems in physics are,
• 36:44
  that it is better than humans at innovating in solving those 10 problems.
• 36:48
  Now, if that happens,
• 36:50
  I guess I'll be wrong, and that will be actually a very significant milestone.
• 36:53
  There's no shortage of mathematicians,
• 36:55
  for example,
• 36:55
  who are using AI to try and help them do their work,
• 36:58
  but the AI hasn't consistently outclassed them yet.
• 37:00
  You know,
• 37:00
  famously in the AlphaGo game,
• 37:02
  people said it was exhibiting intuition,
• 37:05
  which is not a very well defined technical term.
• 37:07
  SPEAKER_4: But what they were saying was it was surprising.
• 37:09
  SPEAKER_4: It had done something that humans had not actually seen in 2,000 years of playing Go games.
• 37:14
  SPEAKER_4: And so we haven't seen that yet, I think, in the sciences with these LLM models.
• 37:19
  I think, Tom,
• 37:19
  the other thing I'd mention is that the notion that the model can continue
• 37:24
  to reframe questions until you understand the answer.
• 37:28
  So I frequently find I don't really understand something well
• 37:32
  until I can explain it very, very simply.
• 37:35
  And so using the tools to challenge myself to do that and to uncover the areas
• 37:40
  where my knowledge is weak is incredibly powerful.
• 37:42
  And the notion that a kid in the Congo with a phone with
• 37:46
  a data connection can have access to that kind of power and that kind
• 37:50
  of personalized learning.
• 37:51
  SPEAKER_3: The Aristotle example is just awesome.
• 37:54
  There's a famous story
• 37:55
  about how Richard Feynman solved some important calculus problem and then went
• 38:00
  to go see his mathematician friend and said, Look, I've solved this problem.
• 38:03
  SPEAKER_4: And he— Yeah, that was solved in the 18th century.
• 38:07
  SPEAKER_4: But to him it was still important because he solved it himself, right?
• 38:10
  SPEAKER_4: From first principles.
• 38:11
  SPEAKER_4: And so these models actually can do a good job of creating curricula.
• 38:16
  SPEAKER_4: So you can tell the model, like, teach me how to do this thing.
• 38:20
  SPEAKER_4: Give me tests and examples as we go through it.
• 38:22
  SPEAKER_4: Don't give me the answer, give me the method of learning how to do this.
• 38:26
  SPEAKER_4: And models can do that quite well.
• 38:27
  So again,
• 38:27
  I'm optimistic that educators will figure out how to teach with this as
• 38:32
  a core technology.
• 38:34
  In terms of the allocation of capital within the global economy,
• 38:38
  I've taken away that both of you think there's a misallocation right now.
• 38:41
  SPEAKER_0: We're spending money on the wrong things in terms of these gigantic data center build outs.
• 38:47
  SPEAKER_0: And also in the wealth that's being created by the major AI arms race vendors.
• 38:53
  What's gonna stop— is it gonna be local fusion plants and local sources of energy
• 38:59
  and better architectures and better silicon that's gonna change that
• 39:02
  or does it take something else?
• 39:04
  SPEAKER_4: Well, I mean, we saw this movie before in the 1990s, right?
• 39:07
  With the dot com boom,
• 39:08
  all these infrastructure companies showed up and dug up all the streets in America
• 39:13
  and laid fiber optics.
• 39:14
  SPEAKER_4: Steve will know more about this than me, but like all of those companies went belly up, right?
• 39:18
  SPEAKER_4: The Global 360s of the world, the Level 3s, whatever, they spent so much money.
• 39:22
  SPEAKER_4: And what we were left with was a whole bunch of conduit in the ground.
• 39:24
  SPEAKER_4: It turns out it's been useful for the last 30 years.
• 39:27
  But it's not clear just because you're spending— Yeah,
• 39:29
  so I was reading some story about somebody's backtracking on
• 39:32
  a 40,000 acre data center in Wyoming.
• 39:34
  SPEAKER_4: It's like, that clearly just seems like an insane project to do, right?
• 39:39
  And then the only way to get the power for these projects is to actually do,
• 39:43
  so called behind the meter power grids.
• 39:44
  SPEAKER_4: And then those things are environmentally not good, right?
• 39:47
  SPEAKER_4: Gas turbines and so on and so forth.
• 39:49
  There's some controversy about Anthropic buying compute from xAI because
• 39:54
  that compute is not environmentally clean.
• 39:56
  So I think we're in the sort of frothy part of the cycle,
• 39:59
  but as I said,
• 40:00
  I've seen this movie before,
• 40:01
  and people used to make fun of pets.com and Webvan in the 1990s.
• 40:05
  SPEAKER_4: I get my dog food from Chewy and I get my groceries delivered by Instacart.
• 40:10
  So the model wasn't wrong,
• 40:11
  it was just done too quickly and was losing too much money too fast, right?
• 40:16
  SPEAKER_1: To circle back to General Magic yet again, that was my glory days, John.
• 40:20
  SPEAKER_1: So that's all I ever talk about in my life.
• 40:22
  SPEAKER_1: That and my kids.
• 40:23
  SPEAKER_3: They were good days for all of us, I think. I think we all—
• 40:24
  SPEAKER_4: Mine too, yeah, it was good.
• 40:26
  JG, are there lessons that stand out for you from General Magic,
• 40:31
  whether from design or business or anything else?
• 40:33
  Yeah, for me,
• 40:35
  a seminal moment in my career,
• 40:37
  which could have been an accident or could have been,
• 40:39
  me being smart or something,
• 40:40
  I don't know, but was the moment I decided to quit General Magic.
• 40:45
  SPEAKER_4: So we had been building, as you know, everything.
• 40:47
  And the Magic Cap operating system was of the order of magnitude of like
• 40:52
  a million lines of code.
• 40:53
  SPEAKER_4: And back then, a million lines of code was a lot, right?
• 40:56
  SPEAKER_4: And the Telescript system we were building was probably another million lines of code, right?
• 41:00
  SPEAKER_4: Order of magnitude.
• 41:01
  And the day I first saw NCSA Mosaic,
• 41:06
  back then to experience the web,
• 41:08
  you had to download this thing and compile it yourself.
• 41:11
  SPEAKER_4: So I download this thing, set it off to compile, went off to get lunch, came back and I ran it.
• 41:16
  SPEAKER_4: And I experienced the very, very early experience of using the web.
• 41:21
  SPEAKER_4: And to me, at least it was transformative that this was going to be a thing.
• 41:24
  But then I did something interesting to me at least,
• 41:26
  was I counted the number of lines of code in the software that I had downloaded,
• 41:31
  because it was all open source software.
• 41:33
  SPEAKER_4: And if I recall, it was like 23,000 lines of code.
• 41:37
  And so it's like, okay,
• 41:38
  we've got a million lines of code in the Telescript system,
• 41:40
  we've got 23,000 lines of code, right?
• 41:42
  And so there's a sort of phrase for this in computer science attributed to Berkeley,
• 41:48
  which is worse is better.
• 41:50
  SPEAKER_4: Right.
• 41:50
  SPEAKER_4: And back then in the early 1990s, people derided HTML as technically poor.
• 41:55
  SPEAKER_4: Apple and IBM were working on something called OpenDoc at the side.
• 41:58
  SPEAKER_4: If you talked to Adobe, they would have said that it was just horrible.
• 42:00
  SPEAKER_4: It was janky, it looked bad, the fonts weren't good.
• 42:04
  SPEAKER_4: Classic case of worse is better.
• 42:05
  It was good enough to cross the threshold to actually ignite one of
• 42:09
  the biggest technological revolutions in our lifetime.
• 42:13
  Anyway so I saw that,
• 42:13
  showed it to my cube mates,
• 42:15
  I went home, stewed on it for a couple of days and then just quit my job.
• 42:18
  SPEAKER_4: I just walked into my manager's office and said, I'm done.
• 42:22
  SPEAKER_4: This is clearly going to be the future.
• 42:23
  SPEAKER_4: And everything we're building is just not gonna work.
• 42:26
  SPEAKER_0: When you quit, did anybody try to stop you?
• 42:30
  SPEAKER_4: I think Marc tried to reason me out of it.
• 42:33
  SPEAKER_4: But no, I was a young junior person, so I wasn't critical to anything that was going on.
• 42:36
  SPEAKER_4: But it wasn't like I quit to go work on the web.
• 42:38
  SPEAKER_4: I quit, I went home and then I was like, I don't know what I'm gonna do next.
• 42:42
  And a bunch of my friends from Silicon Graphics,
• 42:45
  which I was at before General Magic,
• 42:47
  had actually joined what became Netscape, it was called Mosaic back then.
• 42:51
  SPEAKER_4: And so they called me, one thing led to another, and so I ended up working there.
• 42:55
  One of the questions we always try to ask people is,
• 42:59
  if you looked at your remarkable 30 year career, do you see a through line?
• 43:04
  Or was it more random than purposeful in terms of your own values and
• 43:09
  the things you wanted to accomplish?
• 43:12
  SPEAKER_4: It's probably not for me to judge, but I do like working on things that people are skeptical of.
• 43:18
  General Magic was a wonderful experience for that same reason,
• 43:21
  which is you're very passionate about something,
• 43:23
  you want to bring it into existence,
• 43:25
  and there's lots and lots of critics and haters who think it's not gonna work.
• 43:28
  SPEAKER_4: Right.
• 43:29
  And the web was that way for me,
• 43:30
  a lot of the startups that I did, people just didn't think it would work, right?
• 43:34
  SPEAKER_4: And it's like, well, that's why we're doing it, right?
• 43:37
  That's the magic of Silicon Valley is that,
• 43:38
  somebody can have a very well paying job for
• 43:41
  a couple hundred thousand dollars working at some big company in Silicon Valley and
• 43:44
  then quit their job to go do something that's very likely to fail.
• 43:47
  SPEAKER_4: Right.
• 43:48
  SPEAKER_4: What is the motivation, right?
• 43:49
  SPEAKER_4: And I believe fundamentally the correct motivation is creation of something, not monetary gain.
• 43:55
  SPEAKER_4: I tell research scientists all the time who want to quit.
• 43:58
  SPEAKER_4: It's like you want to make the most money, go work in a hedge fund, go work in New York, right?
• 44:02
  SPEAKER_4: You'll get paid even more than you get paid in Silicon Valley.
• 44:04
  SPEAKER_4: But it won't be very satisfying work, right?
• 44:06
  SPEAKER_4: So yeah, as I said, I'm a technological optimist.
• 44:09
  And it's a little bit of an accident of my career that I spent
• 44:11
  the last 16 years managing large teams.
• 44:14
  SPEAKER_4: That was more of an accident than my design.
• 44:16
  To the extent that you think Silicon Valley has lost that ethos,
• 44:21
  maybe you dispute that,
• 44:22
  but to the extent you think it has, how do we get back to that?
• 44:25
  SPEAKER_4: Yeah, it's a good question.
• 44:26
  SPEAKER_4: I do think it has materially changed.
• 44:28
  SPEAKER_4: And so a lot of young people today don't understand the history.
• 44:31
  SPEAKER_4: Like anything, there are notable exceptions to this.
• 44:33
  So for example,
• 44:34
  Patrick Collison and the Collison brothers who run Stripe,
• 44:37
  I've spent time with them.
• 44:39
  They have a very,
• 44:40
  very deep understanding about the history and how we got to where we are, right?
• 44:43
  So there are people who get it,
• 44:44
  and then there are people who are just like furiously trying to make money.
• 44:48
  I think that the only suggestion I have really is to tell this story,
• 44:52
  to tell the full history so people realize how we got here.
• 44:56
  SPEAKER_0: Do you lecture, do you teach?
• 44:58
  SPEAKER_0: Are you interested in doing that?
• 45:00
  SPEAKER_4: Yeah, there is a talk about this that I would like to give or a class I'd like to teach.
• 45:04
  Cause I do think it's important that people actually understand
• 45:06
  the really important inventions were values-based.
• 45:09
  And there's lots of ways in our current economy to do lots of incremental things
• 45:13
  and make a ton of money,
• 45:14
  but that's not how we get the world changing inventions
• 45:16
  that really move humanity forward.
• 45:18
  SPEAKER_0: You need to give a commencement address someplace.
• 45:22
  SPEAKER_4: But I can't mention AI or I'll get booed apparently.
• 45:26
  Mike, I think there's lots of reasons to be optimistic that at least the friends
• 45:31
  of my children,
• 45:33
  they are so world minded about culture, about the ecology, they're ethical.
• 45:42
  I think there is a lot of money driven activity around Palo Alto and Stanford and
• 45:48
  the rest of the valley.
• 45:49
  SPEAKER_3: That's certainly true.
• 45:51
  SPEAKER_3: But I think generally the young generation is really inspiring.
• 45:56
  SPEAKER_0: Unfortunately, we are approaching our time limit.
• 45:59
  SPEAKER_1: Well we have the most important question.
• 46:02
  SPEAKER_0: Yeah, it's yours, Tom.
• 46:03
  SPEAKER_1: And I almost fucking forgot it.
• 46:05
  SPEAKER_1: We have a rule on this podcast that I swear once.
• 46:08
  SPEAKER_1: So there we go.
• 46:10
  SPEAKER_3: Achieved.
• 46:11
  JG, if you were to have a bumper sticker that encapsulates either who you are
• 46:16
  or what you've done or what's meaningful to you, what would it be?
• 46:20
  SPEAKER_4: That's an easy one.
• 46:20
  SPEAKER_4: Hubris and humility.
• 46:21
  SPEAKER_4: You need both.
• 46:23
  SPEAKER_1: Good.
• 46:24
  SPEAKER_1: I like it.
• 46:24
  SPEAKER_1: I like it.
• 46:25
  SPEAKER_1: Steve, I don't know if we asked you that question.
• 46:28
  SPEAKER_3: Life is short, spend time with friends and family.
• 46:31
  SPEAKER_1: Excellent.
• 46:31
  SPEAKER_1: Excellent.
• 46:31
  SPEAKER_1: Well, there you go.
• 46:33
  SPEAKER_3: Which is what I'm doing right now.
• 46:34
  SPEAKER_3: I'm spending time with my friends.
• 46:35
  SPEAKER_1: There you go.
• 46:36
  Well, you may recall on the back of my car when I was at General Magic,
• 46:39
  I just had the refrigerator magnet alphabet letters.
• 46:43
  SPEAKER_1: So I would have a new message whenever fancy struck me.
• 46:48
  And sometimes those entertained passersby as I remember commuting with you
• 46:52
  and sometimes they offended passersby, if I remember correctly.
• 46:56
  SPEAKER_1: Sometimes they were not amused.
• 46:59
  SPEAKER_1: Once I had: The Miata is a total chump car.
• 47:02
  SPEAKER_1: That did not go over well with Miata drivers who passed me, but I stand by that statement.
• 47:07
  SPEAKER_4: I think Zarko had the coolest car, right?
• 47:09
  SPEAKER_4: He had a Lotus.
• 47:10
  SPEAKER_1: Yes, he did.
• 47:11
  SPEAKER_1: I don't know if you guys know the story of Andy Rubin and Phil Goldman.
• 47:16
  And after Phil cashed out at WebTV,
• 47:18
  he said, Andy,
• 47:19
  and Andy had not,
• 47:21
  you know, hit it yet, said Andy, you wanna go half ownership of a Ferrari.
• 47:26
  SPEAKER_1: And Andy was living in this tiny bungalow and he said, Phil, are you insane?
• 47:30
  SPEAKER_1: I can't afford that.
• 47:31
  SPEAKER_1: He said, No, I'll pay for the whole thing.
• 47:33
  SPEAKER_1: I just need a garage because Susan won't let me buy another car.
• 47:37
  SPEAKER_1: So I wanna thank you guys for what's such an illuminating and intelligent conversation.
• 47:44
  I tried to keep my questions to a minimum and that's
• 47:46
  because It seemed like everything you were saying was expanding my understanding
• 47:52
  exponentially.
• 47:53
  SPEAKER_1: And I trust that our audience will have a similar experience.
• 47:55
  SPEAKER_1: So great to see each of you and great to listen to both of you.
• 47:59
  SPEAKER_4: It was fun doing it.
• 48:00
  SPEAKER_4: Thanks for the opportunity.
• 48:01
  SPEAKER_3: Great to see you guys, as always.
• 48:01
  SPEAKER_0: Thanks for joining us.
• 48:01
  SPEAKER_0: Thanks, Steve.
• 48:03
  SPEAKER_3: Take care.
• 48:03
  SPEAKER_3: Spellbound Productions: Thank you for listening to General Magic and the Soul of Tech podcast.
• 48:08
  Tune into our Substack for more interviews and articles
• 48:11
  at spellboundproductions.substack.com.