Latest / Newcomer Pod / Chip War (with Chris Miller)
Transcript
- 0:09Welcome back to Cerebral Valley. I'm here with Max Child and
- 0:13James Woolsterman, my friends and Co founders of Volley.
- 0:17This episode we are talking about chips and big tech.
- 0:21My interview, after a conversation with Max and James
- 0:25is with Chris Miller, the author of Chip War, The fight for the
- 0:30World's most critical technology.
- 0:32He is an expert. We talk a lot about NVIDIA.
- 0:36Yeah, so stick around for that. Max James welcome back.
- 0:40Nice to be here. Eric, Glad to be here.
- 0:42The starting point question that I wanted to frame things up with
- 0:45is, do you think GPU capacity, the quality of these graphics
- 0:49processing units and what NVIDIA is putting out is the main
- 0:53driver for the sort of generative AI revolution we're
- 0:57seeing right now? What do you mostly attribute it
- 1:00to? The research papers, The new
- 1:02approaches. How?
- 1:04How do you? How do you?
- 1:05Who do you give? Credit.
- 1:07I guess I give more credit to the research papers.
- 1:11Not that I don't discount what NVIDIA has done here.
- 1:16I just, I think they were in the right place at the right time
- 1:19with the right technology to accelerate what was possible
- 1:23through due to the research. But yeah, the research was the
- 1:27breakthrough. I do think there was some
- 1:29intentionality. I mean, it's always hard to tell
- 1:31with AI when there's so much. It's a hypey thing to talk
- 1:35about. And this is ACEO that also made
- 1:37a bunch of money off crypto mining.
- 1:39So happy to dive into wild speculation.
- 1:42Clearly I don't know. Max, you have a take here.
- 1:45I mean I think I agree with James.
- 1:46I just think that, you know even four or five years ago we were
- 1:50already starting to see the beginnings of this with GPT 2
- 1:52and you know earlier versions of the the foundational models,
- 1:55right. Also in areas outside of large
- 1:58language models you saw big progress in, you know our hobby
- 2:01horses of speech synthesis, speech recognition and other
- 2:04areas. So I think that even with five
- 2:06year old GP US, you still could pretty to do pretty cool stuff
- 2:10with the new research that was outright whereas.
- 2:12If the research never comes out, I don't think that you're making
- 2:15this kind of progress in, in any of these kind of large large
- 2:18language model areas that we or or any of these areas of the
- 2:21stack that we've seen so far. So right.
- 2:24I mean, chips, even though there's potentially exponential
- 2:28improvement, it's still like, you know, on a trajectory,
- 2:32whereas a paper sort of comes out of nowhere, gives people a
- 2:35new approach and then sort of revolutionizes what's possible.
- 2:40I mean, I think a different way to think about it would be like,
- 2:42OK, GPT has gotten like 1000 times better in the last three
- 2:46years and like the chips haven't gotten 1000 times better in the
- 2:49last three years. So like, what's driving that?
- 2:51Like, it's probably the software and then the people like
- 2:54figuring out how to make GPT, right?
- 2:55So yeah, I just think it's clearly based on the ideas in
- 2:59the software. The chips are great, no doubt,
- 3:01but it's not the key driver. I I do think it so NVIDIA has
- 3:06clearly gotten rich off this stuff.
- 3:08It is now a $1.1 trillion company.
- 3:11In one year the stock is up 258% and in five years it's up 564%.
- 3:21So this, this has been a great run for the company.
- 3:27I mean what do you, what do you guys make of NVIDIA investing in
- 3:32all these companies? I mean similarly if it's into,
- 3:37you know Amazon, Google, like all these companies are making
- 3:41investments. The money that they're spending
- 3:43obviously comes back to their business, right.
- 3:45NVIDIA wants AI to be super active, so puts out money that
- 3:50gets spent on NVIDIA chips. Yeah.
- 3:52What do you make of, I mean it's round, it's round tripping in
- 3:56the OR it's you know spending money to make money, but it's it
- 4:00doesn't seem as sort of negative as in some cases.
- 4:04I mean, isn't the investment NVIDIA is putting out into the
- 4:07ecosystem like a rounding error compared to their revenue or
- 4:09compared to their profits? I mean, like, I I mean I think
- 4:12that they're making such ludicrous quantities of money
- 4:14that kicking back like 1 to 2% into the startup community that
- 4:18might invest in NVIDIA chips in the future is, is obviously just
- 4:21a very good business practice, right.
- 4:23Right. Well, it seems like they're
- 4:24literally playing kingmaker, right.
- 4:26I mean like companies like Core Weave and who are sort of chip
- 4:30doing chip adjacent stuff depend on their access to H1 Hundreds
- 4:35and a lot of these foundation models seem like they're
- 4:37fundraising on the premise that hey we have access like this is
- 4:43differentiated partially because we have the deal to get GPU
- 4:47access when other people don't. And so there's this hope that
- 4:51you know, you're you have a Moat in that you have access to to
- 4:54chips that other people don't. The scarcity today of the best
- 4:58in class chip, the H 100 is. Has to be a temporary situation.
- 5:02I just like the economics of like making more of these chips
- 5:05is a very good idea for NVIDIA. So they're going to do it.
- 5:07But they they clearly want to build like a Amazon Web Services
- 5:12type competitor. And so having eyes and ears in a
- 5:15bunch of startups and seeing maybe who they could acquire to
- 5:18do that is smart. I do think the interesting
- 5:20question again is like Intel basically had like 2 decades of
- 5:24like total monopolistic dominance of of PC chips, right?
- 5:29And. The reason basically was that
- 5:31it's like really, really, really hard to build fabrication plants
- 5:35to to build CP US. And it took a long time for like
- 5:39TSMC and other people to sort of catch up on that front and then
- 5:42eventually kind of surpass them in many ways.
- 5:43But the question I guess for GP US I think is like, is it that
- 5:47technically hard to make GP US that like whatever the next
- 5:51version of the H 100 is? You know that they're going to
- 5:54be on to that before anyone's even caught up to the H 100, and
- 5:57so they'll have this insurmountable 2 two to
- 5:59three-year advantage where if you want the best AIGPUS, it's
- 6:02always going to be NVIDIA for the next 20 years.
- 6:04Because that seems like a plausible case to me.
- 6:06Do you think Microsoft was smart to invest 10 billion in open AI
- 6:09given it's clearly going to distract the company as has been
- 6:13reported? I think my friend in the Wall
- 6:15Street Journal reported as much that they directed sort of their
- 6:18attention to open AI away from Microsoft.
- 6:21And and part of what you're saying is they they are giving
- 6:26up access to their own GPUs to Open AI.
- 6:29Is that accurate to to say so? Yeah, I mean, I think it was.
- 6:34I think it was a good decision and.
- 6:36I agree. Definitely a good decision.
- 6:39I I can't imagine they would be, as you know critical to where we
- 6:45are in the in the AI era right now had they not made that
- 6:48decision they. Have you know early access to
- 6:53all the technology that Open AI is creating.
- 6:56They got GPT 4 into Bing before it was in public in you know in
- 7:02front of customers for in front of Open AIS customers and seems
- 7:06to be continuing with what they are trying to do with Office and
- 7:11Windows, you know the Office suite.
- 7:14So I think. Just that early access to Open
- 7:18AI products seems worth the investment.
- 7:21I think, I think the sort of meta question you're asking that
- 7:24I think is really interesting is like.
- 7:26Is big tech going to win the AI race or whatever, right?
- 7:29Like is like, I mean we always used to talk about Fang right?
- 7:32Like Facebook, Amazon, Apple used to be Netflix and Google
- 7:34right? You could replace the N and Fang
- 7:36with NVIDIA and it would be perfect.
- 7:37So Fang OK and like, goodbye streaming.
- 7:40Hello. Chips.
- 7:41Yeah, Goodbye streaming. Hello.
- 7:42Chips. Right.
- 7:43And I love Netflix, So they never got over like a couple 100
- 7:45billion, which is where you said, you know, NVIDIA is a
- 7:47trillion now. So pathetic. 200 Three $100
- 7:51billion company. Yeah.
- 7:53Is fan going to win AI? Like, I think that's a really
- 7:56interesting question. And I think like I'm kind of
- 7:59leaning, yes, I guess to your point, like it seems like
- 8:02Microsoft's been able to grab, you know, Open AI, which is, per
- 8:06our last podcast, still with the most valuable asset in the
- 8:08entire industry, right. It seems like Amazon's getting
- 8:12very cozy with Anthropic, in case that's actually the other
- 8:15big asset in the foundation model category.
- 8:17Facebook is absolutely like ripping on open source models.
- 8:21You know, Llama. They're going to put like LLMS
- 8:24inside their ad products so you can have automated ads on
- 8:27Facebook and Instagram and all that stuff, you know?
- 8:29They're also putting, they're also putting LLMS into consumer
- 8:35chat. You know, products, right?
- 8:36You can chat with celebrities, and I'm sure they're doing a lot
- 8:39with image generation. Yeah, exactly.
- 8:42And then Google seems to be like fully ripping on, just like
- 8:44dumping large language models into every single Google
- 8:46product. Like Google.
- 8:47You know, Google Docs, Google Photos, I don't know if that is
- 8:51the conventional wisdom. And so on.
- 8:53OK. Google ripping for Google.
- 8:55For the record, you have to like grade them on the Google curve,
- 8:58which is they're in a competent company coming out with new
- 9:01ideas. Given that Google has not come
- 9:03out with a new idea in 15 years, they are doing a tremendous job
- 9:06putting AI products into their existing, into their existing
- 9:10products, right? And so, yeah, for them, the
- 9:13number of launches this year has been tremendous, right, Graded
- 9:16on the Google curve, right. And then Apple has basically
- 9:18done nothing. But Apple always does things
- 9:21like 2 years late and then hopefully they get it right.
- 9:23So the, the great dream of Apple as I understand it is just that,
- 9:27you know, we get to a point where a lot of these small
- 9:29models can be run local, local and that they have these great
- 9:34M1 and M2 whatever generation we're on now chips.
- 9:38And that even though they're not GP, they're sort of combined
- 9:41chips, right. So there is some graphical
- 9:44component, right. And so that those chips would be
- 9:47sort of pretty capable at handling sort of small local
- 9:50models? Totally.
- 9:52And I mean. Apple's argument would be we're
- 9:54using transformer based models in photo recognition stuff,
- 9:57we're using it in speech recognition stuff, we're using
- 9:59it in auto correct. They just put it in the iPhone,
- 10:02right. They're like their their take is
- 10:04always like well we will do it when there's a real customer
- 10:06need and this tool can really provide value.
- 10:09So I think like Jury is still out there.
- 10:10I also think with Apple Vision Pro it'll be really interesting
- 10:13to see what kind of generative AI and and also just you know
- 10:18the AI driven tool set that launches with Apple Vision Pro.
- 10:20So I think like. It's a really interesting
- 10:23question as to whether or not like this is whole AI revolution
- 10:26is just going to entrench the power of the big 5 tech
- 10:28companies because I think like they look pretty good right now.
- 10:32You didn't. You didn't mention Amazon,
- 10:33right? I I said Amazon was like cozying
- 10:36up to anthropic. And also, I mean, you can
- 10:38elaborate on the other stuff they're doing, which is quite
- 10:40copious. On on Alexa specifically.
- 10:42Well, and also AWS, right? Bedrock and everything, right?
- 10:45Yeah, right. Well, it feels like Amazon, I
- 10:48mean Microsoft is clearly ahead like they with Azure and AWS for
- 10:53those two companies I think are the most important.
- 10:56I get you guys really care about Alexa and stuff could happen.
- 10:59But for the businesses today, like Amazon Web Services is so
- 11:03essential. And like Microsoft can go to
- 11:06people and say like Azure has a direct relationship with Open
- 11:10AI, whereas Amazon, it feels like they've been pretty slow on
- 11:15this. But like like you said, they
- 11:16have Bedrock and they're trying to come up with these sort of
- 11:19partnerships, but. Yeah.
- 11:21Well, their strategy is they're going to put everyone else's
- 11:23model on AWS, right. So they're going to be the
- 11:25middleman, like they're going to sell you other access to other
- 11:28people's models. So the anthropic models and I
- 11:30think to the Llama models and to whoever other models they can
- 11:33get on there, they're going to get everything on there.
- 11:34And the only like maybe they will, maybe they won't end up
- 11:38with open AIS models on there. I think that's an interesting
- 11:41ongoing business discussion. But like, in the end, they're
- 11:43going to try to be. You know the supermarket for,
- 11:46for, for models in the cloud, right as they have been in the
- 11:49the the last generation of all the different web tools they
- 11:52sell. So like.
- 11:53But maybe maybe Azure is not allowing Open AI to put that
- 11:58model on AWS essentially right? Yeah.
- 12:01So is Open AI so good that that will cripple Amazon?
- 12:06But it is interesting that I imagine most developers that who
- 12:10are using GPT models are. Doing that directly through the
- 12:14open AIAPIS. Yeah, probably.
- 12:17And there is, you know, there's this world now that was already
- 12:21trending this way of big companies wanting to have like a
- 12:23foot in Azure and AWS so that you can sort of get the best of
- 12:28both worlds. But that would seem to be good
- 12:31for Microsoft given that they were not the first place player.
- 12:34So even saying OK this further creates this world where you
- 12:38want hybrid cloud or you want to dual app clouds is is sort of
- 12:43good for Microsoft. I I can't believe I'm talking
- 12:46cloud computing on the podcast. I feel like even even even as
- 12:51long as we're willing to go it's funny to be in that room.
- 12:55OK. Facebook, I think part, you know
- 12:57Facebook is releasing Llama which is sort of their, they're
- 13:01sort of the king public company and open source models.
- 13:05Their their their top AI person is Jan Lacun who's sort of he's
- 13:10been at this forever and has had a lot of spicy and open takes.
- 13:15I mean I guess the the core question is just like what do
- 13:18you make of Facebook which is sort of a closed social network
- 13:23being so pro like let's just give, let's give foundation
- 13:28models away, make it easy, let's commoditize them.
- 13:31Let's just like give away. What do you make of the Llama
- 13:33strategy? Like what's?
- 13:34What's the point of giving away Foundation models and making
- 13:38that such a core piece of your approach?
- 13:41I mean, I love it. I think it's good.
- 13:44Yeah, I think it's great. I think it's really, it's really
- 13:48probably primarily about developer relations and creating
- 13:56goodwill in the open source and the developer communities.
- 14:00That for them is more important overall to the like longevity of
- 14:06their business than. Owning you know, or monetizing
- 14:12these APIs, these models through APIs which would be a just an
- 14:17entirely new business for them. So I don't know, that's that's
- 14:22my take. I don't know.
- 14:23Do you guys have a different perspective?
- 14:26I just think that I think that there's basically 2 strategies
- 14:29here with like open and closed source software, right?
- 14:31One is like closed source where you keep it private.
- 14:34You're like we have the best stuff.
- 14:36And people will have to come to us even though they won't know
- 14:39the sort of how the internal workings of the software are
- 14:41because it's the best, right. And that's pretty much the open
- 14:43the eye strategy, right, like and some of the other foundation
- 14:46model companies like Anthropic whatever.
- 14:48And then Facebook is like, OK, somebody already took the closed
- 14:51source. We're the best strategy.
- 14:52We're going to take the the, you know, opposite strategy which
- 14:55is. We're going to open source all
- 14:57these models, right? And we're going to hopefully
- 15:00build developer communities around these models so that
- 15:02people can make them better. They can, you know, fine tune
- 15:05them. They can like, you know, help
- 15:07optimize them over time. And then we will have like the,
- 15:12you know, the. Thousands or, you know, 10s of
- 15:15thousands of developers who are working on top of these models,
- 15:18like build and improve them and build a network of of plugins
- 15:22and, you know, optimize them and all this good stuff.
- 15:25And so we're going to get value out of the developer community
- 15:27and and therefore the models that our business Facebook is
- 15:30built on top of will will get better and better that way,
- 15:32right. And so I think it's like it may
- 15:34not work, but I think it's better than just trying to.
- 15:38It's differentiated. Yeah, It's differentiated,
- 15:40right. Yeah.
- 15:41And you know, there are lots of great startups that have been
- 15:44built. You know where it's like we're
- 15:45going to give away a foundation, we're going to give away, sorry,
- 15:48an open source project and then you could build a business
- 15:50around if it's super successful. Also, for Facebook, they
- 15:56probably benefit a lot from hiring great AI talent in the
- 16:00long run. And and if.
- 16:02If that alone just gets them, you know marginally, you know
- 16:07percentage. I know Reels is getting better.
- 16:09You know, I I feel like the. The reels is good.
- 16:11Reels is a good algorithm. When if Facebook and Google nail
- 16:15AI generated ad advertisements, right, or add just even the
- 16:19words in the advertisements, the copyright like that, you have to
- 16:23imagine that's a huge tailwind for the business, right?
- 16:26Like like. You know, personalized ads that
- 16:29are created by AI in real time as you scroll to Instagram
- 16:32feeder or the Facebook feed or whatever.
- 16:34Like, you have to imagine that's.
- 16:36Yeah, yeah, you have to imagine that's a crazy good business for
- 16:40them, right? Or improvement to their business
- 16:42and lowers the friction to actually buying the ads.
- 16:45It's kind of interesting if you think of it that way, because
- 16:48what you're saying Max is AI just improving and getting
- 16:52better faster. Is, is is what's best for
- 16:56Facebook overall, almost like if we had a breakthrough in, you
- 17:01know, energy production of energy or clean energy or
- 17:04something that would be great for Facebook too, probably
- 17:06because they would, you know, have a lot, we could run their
- 17:09data centers a lot cheaper and you know all these things,
- 17:12right? So it's almost just like better
- 17:14for them to encourage the advancement of this industry
- 17:18faster cause a lot of the gains will accrue to them.
- 17:21We we jumped into this with business strategy which I am
- 17:24enjoying. But to me what's interesting
- 17:27about Facebook's positioning is that you know Google, Microsoft
- 17:32with Open AI to some degree they're playing the responsible
- 17:36steward right. It's we are going to make sure
- 17:40our language model behaves appropriately, doesn't get off
- 17:44the rails. We're our brands are at stake
- 17:46and we need to really guard them.
- 17:48You know, Mustafa Suleiman at Inflection, who's going to be
- 17:51speaking at the conference has sort of staked out and you know,
- 17:55we're going to interrogate this more.
- 17:56And so I don't want to caricature it before he speaks.
- 17:58But like a somewhat open source, skeptical position, right?
- 18:02It it undermines the idea that we need to really be thoughtful
- 18:06and protective about who gets access to what and what these
- 18:10systems can do. And so Facebook is playing sort
- 18:14of AI, don't know flamethrower fire thrower type role here
- 18:19where they're throwing fuel to the fire on open source in an
- 18:23area where a lot of the other big tech companies would say the
- 18:26thing we can do is like do this really well and then keep it
- 18:29really closed off so that, you know, we have basically we're
- 18:32almost as functioning as a state ourselves where we're really
- 18:36being thoughtful. What do What do you take of
- 18:39Facebook, the big company playing this role of
- 18:42democratizing? I do worry, I guess for them if.
- 18:48Someone uses Llama for nefarious purposes like.
- 18:51Essentially, it hasn't. It's a brand, but.
- 18:53Yeah, I think that could happen. I think that this could be like
- 18:57a long running issue now for them going forward the next few
- 19:01years. If for if, if, as we've talked
- 19:04about in previous episodes, these models can be used for
- 19:07nefarious, really is laying the groundwork for reporters here.
- 19:10You're like, OK, people should be held accountable for what
- 19:12their foundation models, their open source project does.
- 19:19I don't. I don't.
- 19:20Know if I I don't know if I personally agree that I'm saying
- 19:24that will be the. That could be a narrative for
- 19:27sure, and it just depends on how how much harm can be done with
- 19:32open source models, and I think we don't know yet.
- 19:35I mean. Facebook's like, look, our brand
- 19:38for trust and safety is already so bad that there's nothing we
- 19:41can do that could possibly make it worse.
- 19:43So why don't we just go for it On the business front, they're
- 19:46like Cambridge Analytica. Wait till they see what people
- 19:49will do with this model. Well, Cambridge Analytica is a
- 19:52great similar analogy because they thought they were.
- 19:57At the time they were opening up the Thorngren graph and they
- 20:00were, you know, giving data to, you know, other developers and
- 20:04companies. And they were, and that was, you
- 20:06know, encouraged almost at the time, right, by developers.
- 20:10And it blew back on them entirely.
- 20:13And it wasn't even good for their business to be doing that,
- 20:15to be giving, leaking that data. So it is there is kind of a
- 20:19similar thing occurring here potentially.
- 20:23Facebook's willing to be wild. Well the narrative kind of can
- 20:26change on you right that with at the beginning of social
- 20:30networking and and technology this was encouraged that we
- 20:33would want you know more more data sharing or something with
- 20:37developers and that and then Fast forward 5-10 years and it's
- 20:41not have we all staked out points of view.
- 20:44I think we're all on the same page on do we believe in open
- 20:47source like data bricks, super pro open source, like a lot of
- 20:50the, you know, obviously hugging face clam at CV1, super pro,
- 20:56open source. Like, I feel like generally
- 20:58we've heard from a lot of pro open source people.
- 21:01Do either of you have reservations?
- 21:04Are you fully in the camp of like it's an arms race and just
- 21:08like get the best stuff out and let good people sort of take
- 21:10advantage of? It my take is that open source
- 21:15right now is great for the community, for developers, it's
- 21:19awesome to be able to. You know, edit weights or use
- 21:23smaller models, run, run in or your own inference, all these
- 21:27things fine tune, right? And that just might not be the
- 21:33case going forward. I don't know, at some point
- 21:35maybe I will change my mind and be like, no, the models are too
- 21:37powerful. Like I don't want these open
- 21:40source models in the hands of nefarious actors or you know,
- 21:45China or something. So I I guess my take is it's
- 21:48great right now. I don't know, but it's hard to
- 21:51pull. You know, that's sort of the
- 21:53once they they have the models and it's gone too far, it seems
- 21:56hard, hard to unwind the clock or whatever.
- 22:01Yeah, I guess I I just believe the nefarious actors in this
- 22:04scenario are unstoppable. And as we discussed in The AI
- 22:09Kills Us All episode that somebody's going to figure out
- 22:12how to you know, some bad, bad actor is going to get control of
- 22:15a a really powerful model regardless of whether or not it
- 22:18was open source or not so. Yeah, I mean give the, so you're
- 22:22you're very pro give the pro open source give the give the
- 22:25fruits of the labor away to the community.
- 22:27I mean why do open source? Why do we think?
- 22:30Why do we think Open AI changed its tune so dramatically on
- 22:34this? Question.
- 22:35Awesome business strategy. Just like, just protect your
- 22:38business. Good for the front runner,
- 22:40right? Like like like regulate it.
- 22:43Like let tell. Make sure you have to spend a
- 22:45lot of money, like following rules like that's good for big
- 22:49companies. Because once they realized how
- 22:50good the stuff they were cooking was, why give it away?
- 22:54I mean, open source is the strategy when you're not the
- 22:56leader. Like not, you know, you never,
- 22:59never give away the the high quality stuff, yeah.
- 23:02Or or the strategy that makes sense for meta because they have
- 23:06this these huge other businesses right like you would.
- 23:09It doesn't really make sense. To give away models as a start
- 23:13up I guess I I mean maybe maybe that's what stability is kind of
- 23:17doing or mid journey but you said they're kind of like
- 23:21becoming more close source. Imagine a world where like these
- 23:25algorithms are near AGI like and then they are locked down.
- 23:31You know that's the ultimate elite.
- 23:33So like a a small group of companies gets to control like
- 23:37this sort of Infinity resource like it seems that seems insane.
- 23:41You know what I mean? I feel like that would become
- 23:44sort of one of the great freedom questions of our time, that just
- 23:48in any way restricting the set of people who have limited to
- 23:52access to like a God like reasoning and information
- 23:55system. I feel like we're we're
- 23:56backtracking on Oh yeah kills us all episode but but I mean I do
- 24:01think open source was key to Facebook and that's how we got
- 24:03down this path. But reasonable that we've gone
- 24:06far from that though you've been the one that makes the point
- 24:10that like often these discussions undercount the AGI
- 24:14potential and how disruptive that would be to any of the
- 24:16other arguments that we're. Having I total, I totally agree.
- 24:19And that's where I would question your assumption that
- 24:23everyone should have AGI. Like Without knowing how
- 24:25dangerous that could be for the world, that seems like pretty
- 24:29presumptuous to just assume that that it should not be regulated
- 24:33by a handful of companies. Yeah, I mean it's a lot just.
- 24:37Give away the God like powers. Everyone deserves God like
- 24:41powers. I mean, everyone deserves what
- 24:42can go around. Yeah, I mean they they
- 24:46presumably they will have some counter, you know
- 24:50countervailing. Presumably, sure, yeah.
- 24:52I guess if that's the case, you know, but, oh, I guess here's an
- 24:57easy way to make sure we haven't missed anybody.
- 24:59Let's Let's do a collective ranking of who we think benefits
- 25:03the most from AI in terms of market cap movements experienced
- 25:09and to be experienced. This is I'm putting, I'm putting
- 25:13in no we're doing it together. I'm putting.
- 25:15I think NVIDIA should be our number one.
- 25:18I mean, Microsoft would be the close one there, I think.
- 25:21I mean, Microsoft stock isn't actually up anywhere nearly.
- 25:25So, so NVIDIA then probably, I mean to your point I guess, but
- 25:29I guess the question. What about Google?
- 25:30I I would put Google pretty high here, Microsoft.
- 25:34Depends on what time frame. 34% over the year and 187% / 5
- 25:40years? Google you're putting Google.
- 25:45I don't know. What to me?
- 25:47What time rules in though like tech becoming more important and
- 25:51Google being all over it is good for tech is good for Google.
- 25:54But like Google's relative dominance, which was so powerful
- 26:00in search, feels threatened by this, so I I just.
- 26:04No, I think that's worth talking about because I just feel pretty
- 26:10confident that that whole search impact of AI has like is like
- 26:17very, very early and Google will have plenty of time to
- 26:22potentially continue dominance with you know using LMS and and
- 26:27AI and multimodal models in in in their search products, so.
- 26:33They have, they have the the front page right that everyone
- 26:37goes to for search and they have really powerful AI internally.
- 26:42It just hasn't played out yet. But I don't.
- 26:45I believe that they will get there.
- 26:47All right. So I mean you're you want Google
- 26:50first it just. Depends what we're talking
- 26:53about. Like are we talking about which
- 26:55of the biggest companies will have the highest market caps
- 26:58from AII mean or just like relatively like you know?
- 27:03Is their position improved by AI?
- 27:05How much the percentage of their position is improved by AI?
- 27:10That's what you're asking. Yeah, I mean NVIDIA.
- 27:16Yeah, NVIDIA for sure. Microsoft second agree.
- 27:24Facebook Third I would go what? Yeah, I don't.
- 27:27I don't agree, because algorithms are core to what they
- 27:29do in their discovery. Mechanism.
- 27:31I think if you actually, if you oh, because of algorithms are
- 27:34core to what they do, OK. I just think if you believe
- 27:37there can be a 50% improvement in advertising at scale through
- 27:41AI generation, then Facebook benefits more from that than
- 27:45anyone except Google and and they might think more than
- 27:48Google. I have just put Google before.
- 27:50OK, I mean, that's fine. It's, but it's a totally.
- 27:52Agree with Eric, but but. It's just about search versus
- 27:56the Instagram feed. Which one gets more benefits out
- 27:59of AI like, I think that's the real question here, like or
- 28:03search versus the Instagram feed plus the Facebook feed, right?
- 28:06And I guess it also has YouTube which is competing in like
- 28:09shorts. Yeah.
- 28:11But it's going to take a while to do video generation, I guess.
- 28:13I don't know, like it's sort of about the latter of how quickly
- 28:17generated personalized ads kind of get scaled up.
- 28:19So it's. Yeah.
- 28:21Facebook, you're mostly talking about their ad products and.
- 28:24Ranking algorithms. But there's also an argument
- 28:27around on the content generation piece itself that as we've
- 28:31talked about before, there's, you know, ability to create more
- 28:35compelling content and social media posts and I.
- 28:38I really think the only way you can make the case for Google is
- 28:41that you would say the ads on Google are going to get much
- 28:43better, right? I mean like.
- 28:44Well, what about search itself? You think search is going to get
- 28:47so much better I. Think search itself is going to
- 28:50get so dramatically better? Yes.
- 28:52Like I think, I don't know, I mean time spent in search you
- 28:55would think that go up if they're chat type products.
- 28:57Well, well, there is an argument that time spent goes down,
- 29:01right? Because you're just getting your
- 29:02answer faster at an end. But it's you're not.
- 29:05It's no longer a link. It's like less of a link system
- 29:09and more like in the search, which is where Google ads are.
- 29:12Well, it just depends. If it's more, it becomes more
- 29:15like a ChatGPT model where you're chatting with Bard or
- 29:18instead of searching. I guess that's where I'm coming
- 29:20from. I I see me and myself doing
- 29:23that. Like almost more than.
- 29:25I searched Google today in the future, and I think they're the
- 29:27best positioned to be that like true assistant that I use every
- 29:33day to talk to and instead of searching by typing into AI mean
- 29:38I was down on Google before. But like Google versus Facebook,
- 29:40I'm getting heated up just in the sense that, like, if you
- 29:42think of every product that Google offers like if Gmail was
- 29:46amazing from AI and like the prompts were great and there was
- 29:49writing like could become an even more insane product.
- 29:52Obviously Like Google cloud services.
- 29:56Wait, sorry, it's Google. Cloud platform, global cloud
- 29:59platform is key. If they can improve their like
- 30:03search, YouTube, like they're just so many pieces of their
- 30:06business where they have audience and they could put this
- 30:10to work. The only question we all have
- 30:13probably is execution on all of these areas, but the opportunity
- 30:17to me seems massive. I mean, I guess I'm just going
- 30:21based on the fact that Google makes no money on anything
- 30:23except search and the other stuff is great, but it's really
- 30:25just a data vacuum for for search ads, like they really
- 30:28don't make any money. I mean, they make one on YouTube
- 30:30now, so that would be the other big thing.
- 30:31But like search ads is the whole ball game if you're just purely
- 30:35talking about the business of Google, pretty much.
- 30:37And so you have to believe that search ads are going to get way
- 30:40better and or that people are going to spend way more time in
- 30:43search, which again is an execution question about like
- 30:45Bard versus all the other, you know, large language models and
- 30:48chat models. So like I buy the case, I just
- 30:50don't think it's as clear cut as you guys do because I just think
- 30:52Facebook like Instagram ads and Facebook ads getting better
- 30:55would like to immediately dump right into the business.
- 30:57Which and what you're saying is that there's more of an obvious
- 31:02trajectory for. The Instagram ads and feed to
- 31:06get better from AI than there is for search ads to get better
- 31:09from AI. Yeah, yeah.
- 31:10I mean, it's close. They're, you know, they both are
- 31:13going to get a lot better. So, Yep.
- 31:15But it's about which one's bigger and what's the bigger
- 31:17impact, I guess, Yeah, notably, notably like, so yeah, we, we,
- 31:23it's Facebook or I don't know, I Facebook wouldn't necessarily be
- 31:27my next. Either.
- 31:28Oh, really? Interesting.
- 31:29I'm. I'm willing to.
- 31:30I mean, I. Just feel I would bet it forth
- 31:32at least, but what? What's next?
- 31:35We have Apple and Amazon, you know, like are now towards the
- 31:38bottom of our at least tech list.
- 31:40Obviously tech being better in some ways lifts all boats, but I
- 31:44don't know what do you make of those two and like what it means
- 31:49for them. I would put Amazon next.
- 31:52I think they have an obvious role to play with AWS and they
- 31:58have made a lot of great partnerships there with
- 32:00Anthropic already. They are probably working
- 32:03internally on models themselves. They are the number one cloud
- 32:09platform. They will benefit in their other
- 32:12core businesses through AI potentially, you know Amazon.com
- 32:19and ads there, which is a increasing part of their
- 32:21business and not to ignore it, you're allowed to talk about it.
- 32:27Yeah, Alexa. Which is critical to our
- 32:30business. They have an end point in 30% of
- 32:36US homes that is an AI assistant.
- 32:40It historically hasn't been amazing at talking to you
- 32:44beyond, you know, controlling smart home features and and
- 32:48telling you the weather and playing volley games.
- 32:51But it's it's completely possible that they will be able
- 32:56to. AD LLMS and and AI experiences
- 33:00as they've already demoed that would transform that device in a
- 33:04way that would be you know essential would would would be a
- 33:08game changer for the Alexa business.
- 33:10So I think they're the furthest ahead in that area on like
- 33:15potentially integrating AI with, you know, your average day
- 33:19through a Voice Assistant. And Apple, I mean Apple, I mean
- 33:23the argument Apple gets away with everything in that we can
- 33:27put them in like basically last place and it's still like, well,
- 33:30they move slowly and maybe they'll figure something out.
- 33:32And like, I don't know, it's definitely their wheelhouse.
- 33:35It's like it's not, it's a lot of data.
- 33:38It's not about privacy. It's sort of not a device.
- 33:41I mean, I guess, well, I mean, if we really want to criticize
- 33:44Apple, we saw this report that, you know, maybe Sam Altman and
- 33:48Johnny I've and Masa son Masayoshi, son of SoftBank are
- 33:54getting the team together weird. The weirdest team headline you
- 33:57could imagine to build a phone or some device that's sort of
- 34:02got AI at its core, that that's a threat.
- 34:05I mean, I could buy, you know, a next generation sort of AI first
- 34:09device. I I I just think Apple's Moat
- 34:11has always been how unbelievably difficult it is to build
- 34:14hardware at scale, right? Especially like high margin
- 34:17hardware. I mean, there's almost no other
- 34:19large hardware businesses in the industry.
- 34:21And even if Johnny is on board and Masa is on board, you still
- 34:26need like Masa's got a great history.
- 34:29You still need, yeah, you still need about three to five years
- 34:32and 30 or $5 billion to build even like a small scale, high
- 34:35quality hardware device. And just like I think that's
- 34:37just a long way away and anywhere where you know Johnny
- 34:42and and. Sam are playing like Tim Cook
- 34:45will pay some attention to that and make sure that they're not
- 34:47completely caught off guard, right.
- 34:49You know, they all talk to the same people manufacturing the
- 34:51stuff in China, so but would you?
- 34:52Agree though that like Apple, while they're not threatened
- 34:56that much, like relatively, they are probably lowly ranked here.
- 35:00Yeah, I think. I I think.
- 35:01They're the most, most protected on the downside in some case, in
- 35:04some cases you could argue and then also they don't have huge
- 35:07outside this is all bullshit. They're fine.
- 35:09And if it's good, you know, it doesn't feel like they're going
- 35:12to disrupt it, yeah. Right.
- 35:14I would like to throw another company in the mix here.
- 35:17What do you guys think about Tesla?
- 35:20Ah, yeah. Self driving.
- 35:22The old self driving and the general xai suite.
- 35:26Now, you know, it's like he's somehow allowed to like, merge
- 35:31AI research, like it seems like across all his companies.
- 35:36I mean, I just think that I think I'd be much longer way MO
- 35:39and cruise than than Tesla or whatever.
- 35:41I mean like, I just think that. Tesla's fundamentally built
- 35:44around a hardware business model where you sell cars and like if
- 35:47you believe we're entering a world where all every car is
- 35:49self driving and it can take you anywhere.
- 35:51Like why wouldn't you be long? Like, yeah, Uber or Lyft or
- 35:54Waymo or Cruise or somebody who's built around this model of
- 35:56like providing cars as a service rather than cars as like a
- 35:59jewelry object that sits in your garage 99% of the time.
- 36:02Like, I just don't like. I think Tesla's a great product,
- 36:04but is it an AI for self driving first product?
- 36:07I always thought. The idea that it was gonna turn
- 36:09into a self driving taxi fleet was like pretty absurd, but.
- 36:13I think that's a good point. I think Elon, his hype or
- 36:19marketing spin on this is that the AI that they've built to.
- 36:24Successfully get self driving. Working at Tesla is so valuable
- 36:29in future endeavors, right? Like robots, robotics.
- 36:32But didn't they literally admit they gave up on all the AI
- 36:35they've been building for? Until a year or two ago, they
- 36:38had been using the last generation of deep learning
- 36:41tools or whatever. And then at two years ago, there
- 36:42was a. Like recently there was a story
- 36:44that came out that basically said, yeah we had to redo the
- 36:46whole thing once we realized like transformer models, large
- 36:49language models are better, right.
- 36:50So like they they've been saying that for like 7 years that like
- 36:53we're way ahead and everyone on building a self driving models,
- 36:56we have gazillion trillion miles of data blah, blah, blah.
- 36:59Which is fine is true. But like they had to throw it
- 37:01all in the trash can like 18 months ago.
- 37:03So like I I don't find that like a super compelling like case
- 37:06that they have some Moat around that sort of stuff.
- 37:08But you know, who knows? I mean, they do have.
- 37:11They don't have a lot of trust with like regulators.
- 37:13Like, I don't. I feel like, you know, Waymo and
- 37:16Cruz have just tried to be as responsible as possible.
- 37:21You know, I mean, Waymo has been very conservative in deployment.
- 37:25And you know, Cruz is like General Motors, like a company
- 37:29that I feel like every American politician is rooting for,
- 37:32right. I mean, Tesla, it seems like now
- 37:35has been able to position itself as very close to the Republican
- 37:39Party, but it feels like the administrative state and
- 37:42Democrats are going to be, like, extremely skeptical of them just
- 37:46throwing cars on the road. I mean, that's sort of cynical
- 37:51even about, I guess, the politicians I like.
- 37:53But I yeah, I think I agree with Max's point of technology.
- 37:58I also think having a reputation for being a responsible, sane
- 38:04actor is important when you're doing something as dangerous as
- 38:07deploying the first self driving cars.
- 38:11I think it's a fair, fair assessment.
- 38:12I mean we'll see if they pivot the attack, they're going to be
- 38:14a stock premium for it. I'm sure like I guess they pivot
- 38:18the you couldn't you can't I don't know how you can ever
- 38:21measure this because it's like well the stock can stay inflated
- 38:24for like forever So if they if they pivot.
- 38:26The entire company to robo taxis and they start selling $5 Tesla
- 38:30rides all around America. I will totally change my tune on
- 38:32this, but you pretty much have to embrace a disruptive business
- 38:35model in the next three to five years, which is historically
- 38:38been very, very difficult for any type of company because
- 38:41you're kind of saying all this money we're making a day.
- 38:43That's going to be 0 in five years and we're just going to
- 38:45have to jump on the new thing and ride it out, which is really
- 38:48tough. I to be clear, I don't think the
- 38:51self driving deployment cycle will be anywhere near as fast as
- 38:55that would suggest. So I don't think it's going to
- 38:57destroy their like existing car business.
- 39:00But I agree with the idea it's hard.
- 39:03The the classic start up situation which you're
- 39:06articulating is that like people don't disrupt their own good
- 39:09businesses and so Tesla's in sort of the worst situation for
- 39:12that. Just a counterpoint to that, do
- 39:16we believe that other companies are going to be able to sell
- 39:20fully self driving vehicles anytime soon?
- 39:23It doesn't doesn't seem like it, at least besides the three we've
- 39:27talked about. No, I I mean those other
- 39:28companies seem to be only focused on the robo taxi market
- 39:32as opposed to getting consumer vehicles that can fully self
- 39:37drive themselves into the world like if.
- 39:40Tesla is the only game in town there, and I'm not sure if
- 39:43that's the case, but right now it seems to be the only one that
- 39:46is going in that direction. That could end up being a very,
- 39:50very compelling offering if the only car you can buy that will
- 39:55drive itself anywhere is a test big if we'll see.
- 39:59I feel like Waymo and crews are going to deploy city by city
- 40:02geofence where they know they can do it with no deaths, you
- 40:06know? Yeah.
- 40:08Anyway. Yeah, are there any other, I
- 40:11mean this is great. I I'm glad you brought up Tesla.
- 40:13Like what Any more edge case companies we haven't considered.
- 40:18I mean this is part of the problem I think right now and
- 40:19why I think a data bricks IPO and the Mosaic deal will be
- 40:23welcomed. It's like there's a public
- 40:26market desperation for a way to bet on AI, which is why I think
- 40:31NVIDIA has done so well because, you know, these big tech
- 40:36companies are already very highly valued.
- 40:39You know, the startups are private.
- 40:41Cool, great. Thanks for coming on.
- 40:44And now we're going to Chris Miller with Chip War, who's a
- 40:48real genuine expert in the wonky world of chips.
- 40:53And so we go deep in NVIDIA sort of their history and you know,
- 40:58the, the big geopolitical question around the race with
- 41:03China. So give it a listen.
- 41:09Chris Miller, author of Chipwar. Thank you for coming on the
- 41:12show. Thank you for having me.
- 41:14I feel like so much of what sort of the regular person or even
- 41:19you know, I don't know, the average startup founder, venture
- 41:22capitalist in Silicon Valley thinks of this artificial
- 41:26intelligence phenomenon starts and sometimes ends with chat GPE
- 41:303 and it's certainly it's very much the like.
- 41:33What bots can we play around with?
- 41:34Like how am I talking to some sort of AI but you know in the
- 41:39background doing reporting on some of these conversations
- 41:43NVIDIA and like H1 hundreds and like weird chips come up over
- 41:48and over again and sort of people's ability to get access
- 41:51to those chips. So I really wanted to have you
- 41:53on just to sort of like interrogate that and understand
- 41:57sort of the technology behind it.
- 41:59Can we just start off like, when when does NVIDIA start to matter
- 42:04as a company? Or just give us like, the brief
- 42:06history of this company, because it was like a weird, weird sort
- 42:09of gaming chip making company, right for a long time.
- 42:13Yeah, that that's right. NVIDIA was founded over 3
- 42:15decades ago, but its earliest couple of years was all in
- 42:20graphics, and graphics was most important for games, computer
- 42:24games, video games. Which, you know, think back two
- 42:28decades, they had the most complex graphic demands, figures
- 42:30moving across the scene, pictures changing very rapidly.
- 42:34And there were special types of chips that were produced by
- 42:37companies like NVIDIA for processing graphics.
- 42:39And if you think back, not so long ago, people would buy
- 42:43specific computers to have the best graphics capability.
- 42:46Now we sort of take it for granted, but for a long time
- 42:49that was a real differentiating factor.
- 42:50Can you explain sort of the difference between AGPUA
- 42:55Graphics Processing Unit and ACPU?
- 42:59So it's ACPU, which is the the workhorse of traditional
- 43:01computing and computers and data centers.
- 43:04They're very good at doing many different types of things, but
- 43:07they do every computation serially, 1 after the other.
- 43:11And for most use cases that's what you want because you're
- 43:14undertaking different types of calculations.
- 43:16But for for training AI systems, you want to undertake the same
- 43:21type of calculation repeatedly. And so parallel processing,
- 43:24which is what a GPU does, is capable of doing multiple things
- 43:28in parallel at once, which is why they're vastly faster for
- 43:32this type of calculation than a CPU is.
- 43:35But why? Why does AI?
- 43:37Why do AI tests need to be parallel while like a normal
- 43:42computer task is sequenced? Well, they don't have to be.
- 43:45It's just really inefficient, right?
- 43:49You know, I think the key trend, there's been great research
- 43:51looking at the amount of data on which cutting edge AI systems
- 43:55are trained and and what you find, if you look at this for
- 43:57the last like 10 years or so and chart it, the amount of data
- 44:00used in cutting edge systems is doubling every six to nine
- 44:04months. There's extraordinary demands
- 44:06for putting as much data as possible into training.
- 44:10And in order to make sense of all this data, to train your
- 44:13model to learn from the data, it has to undertake lots of
- 44:16calculations. And so the the computing demands
- 44:19of training have shot upwards not just exponentially, but
- 44:23exponentially, at at a rate that's faster than Moore's Law.
- 44:27And that's why having the right ship for training has become
- 44:30very important. Did NVIDIA see this coming?
- 44:33Like, you know, I their stock is up year to date.
- 44:37I think it's some like 200%. I I feel like Jensen Huang, the
- 44:41CEO, you hear about them all the time.
- 44:42Like was this a happy accident? It's like we're we're making
- 44:46great video games and all of a sudden people want to buy our
- 44:48chips for the hottest thing in the world or did they have some
- 44:52foresight here? How much was this a plan on the
- 44:55part of NVIDIA? You know, it certainly was a a
- 44:57bit of an accident and if you talk to the company they'll
- 45:00they'll they'll literally admit that when they founded NVIDIA.
- 45:03They had no idea that AI would be an application.
- 45:06But over a decade ago, they began to realize that there were
- 45:10a a bunch of PhD students and researchers at places like
- 45:12Stanford and Berkeley who were using their chips for things
- 45:15other than gaming for complex calculations.
- 45:20And that NVIDIA realized that if they tried to build out chips
- 45:24around this application and no less importantly, build out a
- 45:28software ecosystem around it, that they could.
- 45:32Provide the chips that AI would require.
- 45:33And so they've been investing very heavily for now, over a
- 45:36decade. And for a long time that
- 45:38investment seemed actually quite foolish because most of their
- 45:42money was still in in in graphics and in gaming.
- 45:46It's only quite recently that a data center in AI has become
- 45:49their their primary. Well, you just, people talk
- 45:52about artificial intelligence as a way to boost their stock price
- 45:56or to, you know, seem like a more futuristic company than you
- 45:58are for NVIDIA. It's like, OK, you're a gaming
- 46:01company guy. Like, chill out and now you know
- 46:04it's been totally validated. That sort of all this talk of
- 46:07artificial intelligence wasn't total wild hype, hypemanship or
- 46:13whatever. Well, and in between though,
- 46:14they were also a a Bitcoin mining.
- 46:16Yeah, I know, yeah, exactly. They did have the crypto.
- 46:19They're definitely good at seizing the moment, but they
- 46:22were they were being used for crypto mining, right?
- 46:24Is that what it was? That's right.
- 46:27And like, yeah, how do you see Jensen the CEO, like he he's one
- 46:30of the Co founders writer. What's sort of his role at the
- 46:34founding and how do you see him as sort of ACEO today?
- 46:39He was one of the three Co founders of a video they they
- 46:41met in a Denny's in San Jose to devise the business plan that in
- 46:46the the 1990s and and Jensen I I think deserves a lot of credit
- 46:51for realizing. A decade ago, that AI could be a
- 46:56real growth driver. At the time, people thought it
- 46:58was a really irresponsible thing to do plow all this money into
- 47:01building a software ecosystem around chips.
- 47:03No one else did that. Most companies just made chips.
- 47:07And if you go back to the debate in 20-15 around that time
- 47:13period, you can find lots of equity analysts saying, you
- 47:15know, what's this company doing spending all this money on
- 47:17software rather than selling chips?
- 47:18It's a waste of money And and worse, they were giving the
- 47:21software away for free. But it turned out to be a
- 47:24brilliant move because everyone started using the the CUDA
- 47:27ecosystem, which is is the the software layer on top of NVIDIA
- 47:32chips. And that's put it at the center
- 47:35of the the AI world. A key piece of the chip world is
- 47:38this sort of like are you designing chips, are you the
- 47:42fab? Can you just sort of explain
- 47:44that and where NVIDIA sort of fits in?
- 47:46Like, are they? Are they actually we say build
- 47:49the chips. Are they like building the
- 47:50chips? Yeah that that's a that's a key
- 47:53distinction. They're not actually building
- 47:54the chips. They only do chip design from
- 47:57day one. They were, they were what's
- 47:58called a fabulous chip company. They didn't have a fab.
- 48:01They only did the design and and chip design is in many ways like
- 48:06a a software design type of business.
- 48:08You you basically write the code of the design and then you
- 48:11e-mail it off to the the factory or a fab and semiconductor
- 48:15parlance and and the fab actually does the manufacturing
- 48:18and so in the case of NVIDIA. Their most important
- 48:23manufacturing partner has been TSMC, the Taiwan Semiconductor
- 48:26Manufacturing Company, which manufacturers most of their
- 48:29chips. And so today for all of Nvidia's
- 48:31key AI training chips, they're all manufactured by one company
- 48:36in Taiwan. And do they they make all
- 48:39basically all NVIDIA chips or? They make, they currently make
- 48:43all of the key AI chips that NVIDIA produces and they.
- 48:47Been the most important producer of video chips from day one, and
- 48:51is NVIDIA like, how far ahead is NVIDIA right now in this sort of
- 48:55AI chip? Are they singular?
- 48:58Like, yeah, where where are they today?
- 49:01Well, in terms of market share, Nvidia's far, far ahead.
- 49:04So it's estimated that up to 90% of cutting edge AI systems are
- 49:09trained on NVIDIA chips. And you can debate what's the
- 49:12definition of cutting edge, but Nvidia's market position is.
- 49:15That is extraordinary right now and that's reflected in in in
- 49:19their stock price. I I think if you asked are they
- 49:21technologically ahead of of competitors, whether AMD, which
- 49:25is another chip maker or Google, which has a chip called the TPU,
- 49:29which is also used for AI training, that that's a harder
- 49:31question to answer. Exactly.
- 49:34Yeah, exactly, exactly. And that's a hard question to
- 49:37answer because there's both, you know, technical specifications
- 49:40you could look at, but there's also ease of use and so if
- 49:43you've built a system. Trains it on NVIDIA if you
- 49:45yourself as a engineer have gotten comfortable with
- 49:49invidious products and with CUDA.
- 49:51There's also a a a switching cost of moving to somebody else,
- 49:54and so Nvidia's both got their own capabilities.
- 49:56They also got the first mover advantage.
- 49:59Everyone's used to their products, but but they're really
- 50:02scarce right now, right? I mean, my sense talking to
- 50:05startups and investors, I mean, you hear Nvidia's like in every
- 50:09venture round now because as part of the funding, they can
- 50:12sort of promise people their chips, which then they make
- 50:15money back from the startups. It's like a great world.
- 50:17They're playing kingmaker. Like what?
- 50:20What's your sense of how scarce their artificial intelligence
- 50:24chips are? Right now they're scarce.
- 50:28I hear the same anecdotes that you're hearing and NVIDIA and
- 50:31TSMC, the manufacturer partner have have both said it's going
- 50:35to be a year or so until supply normalizes.
- 50:39And the problem is actually not that TSMC can't produce enough
- 50:42chips, but they can't package them because they use a a very
- 50:46complex, a new packaging technology to put the chips
- 50:49right next to the specialized. I assume you don't mean in a
- 50:52box. Yeah, you're like, you place the
- 50:54chip on something, right? Yeah.
- 50:56That's right, yeah. It's a Traditionally chips were
- 50:59packaged in. That'd be funny though, if it's
- 51:00like. Oh, the Apple cardboard box is
- 51:02like holding them up at the end anyway, so go ahead.
- 51:06Yeah. So, so packaging traditionally
- 51:08was a really boring part of the chip business.
- 51:10You take a PIC a chip, you put it in a a plastic or a ceramic
- 51:13package and that was that was done.
- 51:14But now it's it's increasingly important because you need to
- 51:18get your your processor and your memory as close as possible with
- 51:21as fast and interconnect as possible.
- 51:24And so companies like NVIDIA are leaning more heavily on having
- 51:29the right packaging capabilities to provide the most advanced.
- 51:34Outcomes. And so it's actually TSM, CS
- 51:36packaging capabilities that are the limiting factor right now
- 51:39for producing more NVIDIA GPUs. And and what is an H100 or like
- 51:45what what, what are, how different are these chips from
- 51:48the the GPUs that everybody got excited about in the 1st place?
- 51:51Like are they server chips Are explaining, you know, what is an
- 51:55H100? But it's just the newest version
- 51:58of of AGPU that NVIDIA offers specifically for AI training
- 52:02and. The reason people get excited
- 52:04about new versions of chips is because the rate of change is so
- 52:07fast that a new version is always not just incrementally
- 52:10better, but a lot better than the old version.
- 52:14You know, it's it's it's not like the iPhone 15, which is a
- 52:17bit better than the iPhone 14. But we've stopped noticing and
- 52:20I'm like, this is not enough necessarily to get me from 13
- 52:25Pro or whatever. Yeah, yeah.
- 52:27The chips are different because of the the Moore's Law dynamic,
- 52:31which is, you know, says you roughly double your processing
- 52:34power every two years. And that's not a perfect, a
- 52:37perfect metric, but ballpark that tells you just how rapidly
- 52:40the technology improves. And the H100 is the most
- 52:43advanced chip that NVIDIA offers.
- 52:46So yeah, to put Moore's Law is still alive today.
- 52:50Moores law is actually not a law.
- 52:52It's a prediction that was set out by Gordon Moore, who founded
- 52:55Intel in 1965. He predicted that the number of
- 52:59transistors per chip would double every year or two, which
- 53:03means that the computing power of chips would double every year
- 53:06or two. And that's been and he said it
- 53:09would be a decade, but then it's been true for a long time,
- 53:11right? That's right, I'm just stealing
- 53:14from your book. Of course it makes me sound
- 53:17smarter when you get to read the book in advance, so you know all
- 53:20the answers. Anyway, go ahead.
- 53:22But yeah, that's right. So Gordon Moore thought, would
- 53:24last for a decade, and here we are half a century later.
- 53:27And Moore's law is is still basically holding up the the
- 53:31rate of change increases and decreases at certain times.
- 53:34But it's basically true that if you wait 2 years, a new chip
- 53:38will have twice as many transistors and therefore twice
- 53:40as much computing power. I mean, this is sort of a hard
- 53:44question, so I don't know if you there's an easy answer, but
- 53:48yeah, how much credit would you give then the improvement of
- 53:52GPUs for what's going on in artificial intelligence right
- 53:56now relative to, you know, attention is all you need or you
- 54:00know, the papers in terms of the approaches to actually use these
- 54:04chips to to run, run the software?
- 54:09Well, you know, I think it's certainly it's the case that
- 54:11that there's there's multiple factors driving innovation.
- 54:14But I just the way I like to think about it is, you know
- 54:17what's improved most over the last decade?
- 54:19Is it the case that that software engineers are 16 times
- 54:23smarter or is it the case that? Algorithms are 16 times better
- 54:28or is it the case that because of Moore's Law 2 to the fifth,
- 54:32we've got, you know, a vast increase in the number of of
- 54:36transistors? You know, I think it's it's
- 54:38primarily computing power that's that's driving us.
- 54:41That's what you want. The chips guy comes on, he says
- 54:44it's the chips. Yeah.
- 54:46A key theme, you know, obviously in the book and sort of for
- 54:49anyone thinking about the situation is, you know, the
- 54:52rivalry with China and and sort of the the delicacy of the
- 54:59global supply chains and you know, the, there's the chip
- 55:03rivalry with China. But also in artificial
- 55:05intelligence, you know, if we're worried that this is the
- 55:08potential sort of path to some sort of, you know, generalized
- 55:12intelligence, there's also the real question of whether the US
- 55:16gets there first. Yeah.
- 55:20So I mean, there's so much to hit at.
- 55:22I mean, I guess the first question for me is, is the
- 55:25United States getting most of Nvidia's chips or like how much?
- 55:29How many of those chips are going to China?
- 55:31Is China getting access to those right now while they're scarce?
- 55:36Well, the the US last year imposed new rules that said
- 55:41NVIDIA can't transfer its most advanced GPU to China.
- 55:44And not just NVIDIA. Any company producing chips
- 55:47above a certain threshold can't transfer them to China, so H1
- 55:51hundreds are illegal to. To transfer to China, China's
- 55:55not getting any of those. But but NVIDIA has design chips
- 55:58that go basically right up to the threshold called A8
- 56:01hundreds. And Chinese firms are reportedly
- 56:03buying very large volumes of those.
- 56:06So China's getting a lot of GPUs, but they're less advanced
- 56:09than what AUS firm can buy. What would their next best
- 56:12option be? Well, the next best option is
- 56:15Nvidia's second best chip. Oh, man, OK, right.
- 56:20And China tried to build its own fab, right?
- 56:24A competitor of TSMC. What's, what's the status of
- 56:28that and how much? You know it's fun, especially in
- 56:31the US and given the market returns, like to talk about
- 56:33NVIDIA, but like how much does this all hang on TSMC?
- 56:38So China has been investing very heavily in its own chip
- 56:40industry. It's got both companies doing
- 56:42the design side and the manufacturing side.
- 56:46There there are a number of Chinese GPU designers that seem
- 56:50to have pretty competitive products, although even in China
- 56:54Chinese firms prefer to buy invidious chips, so they're
- 56:57they're not as good it seems. But the big challenge in China
- 57:01is the manufacturing side because TSMC, the Taiwanese
- 57:04firm, has been around five years ahead of the leading Chinese
- 57:09firm SMEC. For a very long time, for at
- 57:12least a decade, there's been a five year gap.
- 57:13So both companies regular regularly improve the
- 57:16manufacturing processes, but the rate of change in both is
- 57:20roughly the same. So TSMC is always half a decade
- 57:23ahead, and that holds to up the Today.
- 57:25And what about the United States?
- 57:26Like it hadn't Intel been trying to become a fab?
- 57:29Like what's what's the state of our ability to actually
- 57:32manufacture these chips if you know, there's a war in Taiwan,
- 57:35God forbid? Well, yeah, that that's that's
- 57:38the dilemma. So Intel does make its own
- 57:40chips, but around five years ago it faced some severe problems
- 57:45with its manufacturing operations.
- 57:47And so for its most cutting edge chips today, Intel now turns to
- 57:51TSMC for the manufacturing. Now Intel right now is is trying
- 57:55to reformulate its manufacturing processes and it hopes that by
- 58:00they're saying by 2025 they're going to be producing.
- 58:04Have their most advanced chips back in house again, and that
- 58:06those chips will be as capable as a TSMC made chip.
- 58:09But right now, Intel's most advanced chips are actually
- 58:12manufactured by TSMC. And then there's like a key, the
- 58:16people who make the machines that help T that allow TSMC to
- 58:22do what they do, There's like one company of those or what's
- 58:24that company? Yeah, that that's right.
- 58:27So TSMC, they know how to use the machines to make chips, but
- 58:30the machines themselves are produced by a handful of other
- 58:33companies, a couple in California, a couple in Japan,
- 58:36and then one in the Netherlands called ASML, which produces the
- 58:39most complex of these chip making tools.
- 58:43So is quantity. Everything here with the H1?
- 58:48Hundreds like, I mean, you know, I'm at, you know, Microsoft has
- 58:51this cloud computing offering. You know, Google has its, Amazon
- 58:55obviously has Amazon Web Services.
- 58:57How much are those services trying to stockpile AI chips?
- 59:01And you know, then these startups, whether it's, you
- 59:05know, anthropic or open AI, how much is their game?
- 59:08Just like we're going to win if we assemble the most H1
- 59:11hundreds, Is quantity sort of the game right now in terms of
- 59:15getting access to these chips? So long as there are shortages,
- 59:19I think Quantity's a key part of the game.
- 59:22And and what you see is that NVIDIA now realizes it's got a
- 59:25lot of influence over the future of the cloud computing market.
- 59:29And so NVIDIA has been, I I think, pretty actively trying to
- 59:33build up other cloud computing firms by giving them exactly
- 59:39exactly giving them access to GPUs and NVIDIA.
- 59:43I think quite rationally wants a more fragmented cloud computing
- 59:46market because that's a market in which it has more market
- 59:49power relative to Microsoft, to Amazon and to Google.
- 59:54So so we alluded to this, but sort of can you walk me through
- 59:57the tech giants in terms of credible competitors to NVIDIA?
- 1:00:03I mean we talked, I guess start with Google, it sounds like
- 1:00:05maybe they're in the lead. I want to know about Intel.
- 1:00:07And then is there anybody else who's who's relevant or even
- 1:00:10close? Yeah.
- 1:00:12So Google has a a in house designed chip.
- 1:00:16They've got a big design team that has designed a chip called
- 1:00:18ATPU which stands for tensor processing unit but fairly
- 1:00:22similar to AGPU that is used for for AI applications and in
- 1:00:28Google has a vast cloud computing business as well as
- 1:00:31its own, its its own vast data center.
- 1:00:34So lots of experience running very complex computing
- 1:00:39operations but empirically we know that companies prefer to
- 1:00:44use NVIDIA chips at least they do right now.
- 1:00:46You know that could change as Google's chips get better as
- 1:00:50scarcity drives up the price of GPUs.
- 1:00:53But that hasn't been the the trend, the preference has been
- 1:00:55for NVIDIA. Then AMD is the other company
- 1:00:57that that is a competitor that in producing GPUs it's a a chip
- 1:01:02design firm. They also manufacture with TSMC
- 1:01:06that in. They're an American firm.
- 1:01:08That's right. Based in Texas and and they
- 1:01:12manufacture fairly competitive GPUs.
- 1:01:14But the key difference with between their chips and Nvidia's
- 1:01:18is that NVIDIA has the the ecosystem around it, the CUDA
- 1:01:21ecosystem. And so once you're bought into
- 1:01:24that ecosystem, right now the switching costs are substantial
- 1:01:28enough where you just prefer to stick with NVIDIA if you can get
- 1:01:30access to enough chips. Is Intel relevant at all?
- 1:01:34Intel is is trying hard to produce competitive chips and I
- 1:01:39think over the next couple of years we'll see it roll out a
- 1:01:41series of new products that are designed to compete.
- 1:01:44But right now it's a small player.
- 1:01:46In my understanding like in cloud computing, like part of
- 1:01:49the beauty of like an AWS is that you can sort of you know,
- 1:01:53it's flexible, it's elastic, we can sort of have some people do
- 1:01:56the computing, then you sell it to another customer at a
- 1:01:59different time. Why hasn't that been the case
- 1:02:02with this artificial intelligence computing, I mean?
- 1:02:05I hear people talk about it like you're you're running full bore,
- 1:02:08like these data centers are super hot.
- 1:02:10Like what's what's so different about AI processing?
- 1:02:14I think the the, the key difference is that the demand
- 1:02:18for compute power in AI training is so large that there's just a
- 1:02:23deficit of it. There's just a deficit.
- 1:02:25And so the the systems that firms like Open AI are trying to
- 1:02:28train are so vast that they they can't share their infrastructure
- 1:02:34because they need all of their infrastructure and they need
- 1:02:36even more of it than they actually can get access to.
- 1:02:38And so the cloud computing business has been a business, as
- 1:02:42you say, about engineering more efficient systems via sharing.
- 1:02:46But in AI training, no one wants to share because everyone needs
- 1:02:50more compute than they can actually get access to.
- 1:02:53Right. And then there is also this sort
- 1:02:54of perceived almost like 0 sum, like you want to deny your
- 1:02:58competitor access. One company we haven't talked
- 1:03:02about, you know, that we talked about in the laptop world for
- 1:03:07chips is Apple. Like what's what's the status of
- 1:03:10Apple on this? Sometimes people talk about
- 1:03:12Apple and artificial intelligence on like local
- 1:03:15processing. Is that a possibility or how do
- 1:03:17you see Apple as being relevant or not in this race?
- 1:03:23Yeah, you know, thus far we've been talking about AI training
- 1:03:26and and AI training happens almost exclusively in big data
- 1:03:29centers. But there's a big question about
- 1:03:31what inference will look like in the future.
- 1:03:34Some people think that inference will mostly happen in data
- 1:03:37centers, that you're gonna ask a question of ChatGPT, you'll send
- 1:03:41it back the Internet to the data center, tips to the data center.
- 1:03:44We'll think about it and give you an answer back.
- 1:03:46And that is efficient in some ways.
- 1:03:48You have all the compute, all the chips in one big data
- 1:03:50center. And so you can start to set up
- 1:03:52some of the sharing and efficiencies that that we just
- 1:03:54discussed. But the downside is you have
- 1:03:56latency issues if you're sending all the data back and forth and
- 1:03:58there's costs associated with moving data.
- 1:04:00And so an alternative paradigm is doing more of your inference
- 1:04:04on the edge of networks in your phone, in your car, for
- 1:04:07autonomous driving systems, in in in your PC.
- 1:04:11And that's where companies like Apple could start to play a much
- 1:04:14bigger role. Right now, I think the the
- 1:04:17market for inference on the edge is still very much in flux.
- 1:04:21We're in the process of seeing many new types of chips rolled
- 1:04:25out precisely for that purpose. So, so it's just sort of depends
- 1:04:29on where it's one of these like sort of personal computer versus
- 1:04:34server level questions back in the 90s where it's like OK, is
- 1:04:38computing going to continue to happen?
- 1:04:40In the cloud? Or is there some model where
- 1:04:42local computers are going to do and you're saying that if it is
- 1:04:45local, then maybe Apple's in a better position because they
- 1:04:48have these great chips for for individual computers.
- 1:04:51Is is that right? Yeah, that that, that's right.
- 1:04:54And there's I think a pretty straightforward cost equation
- 1:04:56of, you know, if the cost of data transfer is high, you'll
- 1:05:01try to do as much as your processing can on the edge.
- 1:05:03If the cost of the transfer is low and the benefits of having
- 1:05:07big centralized facilities are are substantial in terms of the
- 1:05:10efficiencies you can reap, then you'll have a much more
- 1:05:12centralized inference process. And it it might vary even
- 1:05:16between different use cases. In a car, for example, if you've
- 1:05:20got a pretty autonomous car, your willingness to to depend on
- 1:05:25a data center to tell you to turn left or to turn right, it's
- 1:05:28going to be pretty limited. I would, I would, I would
- 1:05:30hypothesize. Whereas for your phone maybe.
- 1:05:32You want to make sure no matter what you can get the answer but
- 1:05:35you know if it's for mid journey I can wait till exactly you
- 1:05:41know, I mean we we talked about NVIDIA earlier on in terms of oh
- 1:05:46there was a brief period where they were essential to crypto
- 1:05:49mining and I and one of the key storylines I think think for the
- 1:05:54non crypto world that was just watching it.
- 1:05:56Was all the energy consumption around those crypto mines is?
- 1:06:02Is that true for AII haven't heard that same environmental
- 1:06:05story get picked up in this case.
- 1:06:08But are are we destroying the environment and our pursuit of
- 1:06:12artificial intelligence? Why, I think we we haven't heard
- 1:06:16much about it just because there's such an extraordinary
- 1:06:18demand for computing that companies haven't gotten around
- 1:06:23to thinking about the consequences.
- 1:06:24But the short answer is, is yes, it's extraordinarily energy
- 1:06:28intensive and and we're going to get more efficient at it as as
- 1:06:32time passes. But right now there are, you
- 1:06:36know, such demands for energy in data centers that there are some
- 1:06:39places in the US where it's difficult to build a data center
- 1:06:42because there's no spare electricity capacity for the
- 1:06:45data center. It's it's already a gating
- 1:06:47factor. Have you seen any?
- 1:06:49Yes, it's about energy use or anybody trying.
- 1:06:53Well it's it's, it's tricky because everything depends on
- 1:06:56how much more efficient you think things will get over time.
- 1:06:59So you've got to assume we're going to be a lot more efficient
- 1:07:01making the same compute available at lower energy demand
- 1:07:05in 10 years time. But at what curve it's it's very
- 1:07:08difficult to say. Yeah.
- 1:07:10And to be clear, I think what what is the point of energy if
- 1:07:14not to sort of push the cutting edge of like computing?
- 1:07:17It seems like a very valuable use to me, but it yeah, it'd be
- 1:07:21interesting. I mean, you obviously want to
- 1:07:23make sure companies aren't being sort of cavalier about it.
- 1:07:26And yeah, well, there's certainly not cavalier because
- 1:07:30if you look at the cost of running a data center, 'cause I
- 1:07:32don't know what it is, yeah, critical cost.
- 1:07:35Well, I mean, I don't know if you're Open AI, you raised $10
- 1:07:38billion, it's you can become sort of irresponsible.
- 1:07:42You know there isn't the same market pressure if these
- 1:07:45companies are able to raise what are basically like research
- 1:07:47grants to to figure this all out.
- 1:07:51Can you, I mean, we sort of jumped into artificial
- 1:07:53intelligence, but I mean can the broad theme of the book, the
- 1:07:57idea that sort of that we often think of global infrastructure
- 1:08:02and the sort of weaknesses and vulnerabilities in terms of like
- 1:08:05oil, right. I mean it's like oh, did we go
- 1:08:07to war in Iraq over oil? What what is sort of the real
- 1:08:12sort of what are the weaknesses with chips and what what why do
- 1:08:16you think sort of this chip's arms race is?
- 1:08:19So important when we're thinking about sort of geopolitics.
- 1:08:24Well, the reason I, I, I wrote chipwork was when I realized
- 1:08:27first, that we're surrounded by thousands and thousands of chips
- 1:08:31over the course of our daily lives, Not just our phones and
- 1:08:34our PCs, but it's cars and dishwashers and coffee makers
- 1:08:37too. Second, when I began to
- 1:08:39understand how critical they are for AI.
- 1:08:42And then third, when I realized how concentrated the production
- 1:08:45is in just a tiny number of countries and really just a
- 1:08:47handful of companies, Taiwan being the the, the, the, the
- 1:08:52most surprising example of that. And so when you look at the the
- 1:08:56US, China race to develop more advanced AI systems and in tech
- 1:09:01in general, then realize that both China and the US are
- 1:09:04dependent to a shocking degree on ships made in Taiwan to power
- 1:09:07their most advanced AI systems. It's an extraordinary situation
- 1:09:11that we've all found ourselves in and it's a very dangerous one
- 1:09:14given the concentration and given the dependency.
- 1:09:18What would be the consequences of China having a chip edge
- 1:09:22broadly and an artificial intelligence?
- 1:09:25Well, just like we're preventing China from accessing our most
- 1:09:29advanced ships, I think one has to assume that any country with
- 1:09:32that position would do the same to us.
- 1:09:35And right now, Chinese firms that are trying to develop AI
- 1:09:38systems are doing so at least severe disadvantage because
- 1:09:41they've got much less access to high quality hardware than does
- 1:09:45Open AI or or Google. And that's I think the only way
- 1:09:50to accelerate into the future as countries, including the US
- 1:09:53become more restrictive in terms of controlling access to
- 1:09:56different types of AI hardware. And so it's a good thing that
- 1:10:00we've got most of it designed here and most of it manufactured
- 1:10:04in friendly countries, because what we're going to see, what
- 1:10:07we're already seeing, what we'll keep seeing is more
- 1:10:10politicization, I think, of the hardware that makes AI possible.
- 1:10:13Is the United? I mean the United States still
- 1:10:16seems great. I guess that chip design, given
- 1:10:18we have so many of the leading firms, could, do you think we
- 1:10:22could ever produce a fab or like what?
- 1:10:24What makes TSMC so uniquely capable at building a fab?
- 1:10:29Like, we're talking culture, money, like, what's what's
- 1:10:34driving that in your opinion? Yeah.
- 1:10:36You know it's it's it's not it can't be a cultural thing
- 1:10:38because Taiwan hasn't always been the center of the chip
- 1:10:41industry. And if you go back four decades
- 1:10:43ago, it was Japan that was the world's biggest producer of of
- 1:10:46advanced semiconductors. So it's primarily I think the
- 1:10:50result of TSMC business model which has let it scale and the
- 1:10:54scale has let it Dr. efficiencies.
- 1:10:57And so when TSMC was founded in 1987 at that time almost all
- 1:11:01companies both designed and manufactured chips in house.
- 1:11:04They did, both sides of the of the equation, And the founder of
- 1:11:08TSMC, Morris Chang, realized that as it was getting more
- 1:11:11complex to manufacture, companies would prefer to
- 1:11:14outsource it. And so he decided that he wanted
- 1:11:16to be sort of like Gutenberg was for books.
- 1:11:18Gutenberg didn't write any books, he only printed them.
- 1:11:20Morris Chang and TSMC, they didn't design any chips.
- 1:11:23They only manufactured them. And what that meant is that he
- 1:11:25could produce chips not just for one company, but for lots of
- 1:11:29companies. For AMD, for Apple, for
- 1:11:32Qualcomm, for NVIDIA. And so today, TSMC is the
- 1:11:34world's largest chip maker. And precisely because it's the
- 1:11:37largest, it's also the most advanced because it can hone, it
- 1:11:40processes over every single silicon wafer that it
- 1:11:43manufacturers. How can you can you make any
- 1:11:47predictions about how much you think AI chips are going to
- 1:11:51improve? Or like what do you see sort of
- 1:11:53the next couple years looking like in terms of like do
- 1:11:57availability problems resolve and and how much more do you
- 1:12:01think these chips will improve over the next couple years?
- 1:12:06Yeah, on the the availability front, NVIDIA and TSMC has said
- 1:12:10they expect it to be a year or so until the shortages get
- 1:12:14resolved, but I think they're going to get resolved.
- 1:12:17Prices are such that there's a very strong incentive to solve
- 1:12:20the availability problem and sell more.
- 1:12:23Sometimes people think they want to keep a shortage to keep sort
- 1:12:27of demand high. You you think they want to
- 1:12:29produce as many chips as they can get out there to make the
- 1:12:31money while it's there to be made or yeah I think that's
- 1:12:35right because they're they're it looks like they're a
- 1:12:37monopolistic producer. They've got 90% of the market
- 1:12:40but if they can't supply chips there are competitors there is
- 1:12:43Google with TPUS there is AMD. And so if there are long run
- 1:12:47shortages that that are are in the supply chain, people will
- 1:12:50turn to competitors. And so they've got a strong
- 1:12:52incentive to get get supply increase.
- 1:12:54And actually part of their strategy is again to put their
- 1:12:57chips at the center of the AI world and make it the the gold
- 1:13:01standard. And the more people that design,
- 1:13:04the more people that train, more people that study using their
- 1:13:08system using the Kuda ecosystem, the more likely NVIDIA is to be
- 1:13:11at the center going forward. And then I guess sort of the
- 1:13:14next chip they have is hard to know or have there been any
- 1:13:17leaks about what what the future looks like in terms of these
- 1:13:21artificial intelligence chips? I mean I think the the best
- 1:13:24guide is is to look at the past and and what you find is that
- 1:13:28that for over the past half decade they've released a series
- 1:13:33of better and better chips that you know ballpark Moore's law
- 1:13:37gives you a pretty good guide as to where things should be in a
- 1:13:40couple of years time bigger, more powerful hopefully somewhat
- 1:13:44more power efficient and and the other area focuses the the
- 1:13:49interconnect speed between semiconductors.
- 1:13:52So there's the GPUs themselves, there's all the networking
- 1:13:54equipment that moves data between chips.
- 1:13:57That's also an area of really intense focus right now.
- 1:14:01And we're I think we're gonna see a lot of progress for the
- 1:14:03coming years. Jensen I mean, did you interact
- 1:14:06with him at all for this book or no, I didn't actually.
- 1:14:10I I interviewed his his Co founder but not not.
- 1:14:12Interesting. Yeah.
- 1:14:13I mean, just like observe. Like it must be crazy to go from
- 1:14:18sort of being seen as this, like, I don't know, wild sort of
- 1:14:21futurist to all of it. Seeming to come true or the
- 1:14:25world sort of agreeing with your futurism.
- 1:14:29I mean he sort of he he wears like leather jackets and dresses
- 1:14:32all black or like what? What do you make of him?
- 1:14:35And yeah, how much has this been sort of a vindication for him?
- 1:14:40Well, I think he he sort of sees himself like like Steve Jobs was
- 1:14:44for the iPhone moment, he is for the AI moment.
- 1:14:47And I think that's right. I I think he he has played that
- 1:14:50role because he does with a lot of credit for investing in the
- 1:14:54types of products that that have made AI possible at a time when
- 1:14:58most people thought it was either crazy or possible but
- 1:15:02only relevant for niche academic uses or something that would
- 1:15:07never be financially viable. And look, he's built one of the
- 1:15:10world's largest tech companies around it.
- 1:15:13And already there are products being produced on a regular
- 1:15:16basis using chips that his company has pioneered.
- 1:15:20I believe I agree with you. You can make a fortune.
- 1:15:23Silicon Valley can love you. But I feel like the world will
- 1:15:26only love you if it's your device in their hands or it's
- 1:15:29your brand. You know, it.
- 1:15:31I, I, it just doesn't seem like they have a path for like
- 1:15:34consumer love, right? I mean, do you see them trying
- 1:15:37to take advantage of this and go, you know, offer other
- 1:15:42products or, you know, they're so good at chips, they're
- 1:15:45they're gonna, they'll win the technologist's heart and be
- 1:15:47happy with their cash mile. Yeah.
- 1:15:51They're so far from the computer in terms of where they sit in
- 1:15:54the tech stack that like most consumers don't even know they
- 1:15:58exist. Or gamers do is sort of the
- 1:16:02funny gamers do. Right.
- 1:16:03Yeah. How did how did you get into
- 1:16:07this? Like what were you sorry, I
- 1:16:09don't know as much of your background.
- 1:16:11Yeah. Before you wrote this book, were
- 1:16:12you a chip nerd or what? No, not at all.
- 1:16:15Oh, really? OK what was your story to the
- 1:16:17book? So I I'm a an economic historian
- 1:16:20by training written a couple of books on different aspects of
- 1:16:22economic history and and decided to write the book when I came to
- 1:16:26realize how important ships were and was shocked myself that I'd
- 1:16:30never paid any attention to this industry.
- 1:16:32And so it was the combination of learning that almost all
- 1:16:36advanced ships are made in Taiwan, which was shocking.
- 1:16:39Learning that chips are so extraordinarily difficult to
- 1:16:42manufacture and transistors in your iPhone, for example, are
- 1:16:46small in the size of a coronavirus manufactured by the
- 1:16:49billions. That was an extraordinary fact
- 1:16:52that got me into it. And then and then finally
- 1:16:54looking at the ramifications for AII, like most people have
- 1:16:57thought about AI as a as a question of of of either
- 1:17:00software or of data. Data is the new oil I'd read a
- 1:17:03million times, but it turns out that's not exactly right.
- 1:17:06Did you go dump all your money into NVIDIA or I should have?
- 1:17:11Yeah. Did you see the stock?
- 1:17:13I mean I I know. Well you're if you're an
- 1:17:15economic historian like do you have a view on whether, OK, this
- 1:17:19is a great company but it's overvalued.
- 1:17:21I mean you know, yeah it's do you, I mean you shouldn't turn
- 1:17:24to me for for stock market advice.
- 1:17:26But I mean I think the, the key question about NVIDIA is, is
- 1:17:30will it. I, I think it's clear it will
- 1:17:33continue to play a dominant role in AI training.
- 1:17:36Maybe it'll lose market share, maybe it'll win market share
- 1:17:38with Google, but it'll play a central role.
- 1:17:40But in the inference market will NVIDIA be as central there or
- 1:17:44not? That's I think the key question
- 1:17:46that we're gonna find the answer to over the next couple of
- 1:17:48years. And do you think China is close
- 1:17:51to catching up? You know, I I think China's is
- 1:17:53not that far behind, but China has been not that far behind for
- 1:17:58some time. And because the rate of advance
- 1:18:01is so rapid in terms of design in the US and manufacturing in
- 1:18:04Taiwan, it's very, very hard to catch up.
- 1:18:08And so long as that companies like NVIDIA and TSMC keep racing
- 1:18:12forward to catching up is just going to be extraordinarily
- 1:18:15challenging for China and no matter how much money they pour
- 1:18:16into it. And are Japan and South Korea
- 1:18:19relevant to this conversation? Well, it's that's an interesting
- 1:18:22question. Japan is relevant because if you
- 1:18:26look inside of a chip making facility, there is a ton of
- 1:18:29Japanese machine tools and there's also a wide variety of
- 1:18:34ultra specialized chemicals made in Japan.
- 1:18:37So a lot of the chemicals used in chip making because of the
- 1:18:40precision required have to be purified to the 99.99999% level.
- 1:18:46A lot of those are only made in Japan.
- 1:18:49I wanted to go back and I know this is like a complicated
- 1:18:53question, but like what are the tasks that are being thrown at
- 1:18:58these chips, right? Or you talked about sort of
- 1:19:01matrix computing or? Is there like 1 task over and
- 1:19:04over again that these chips are being asked to do or it's a lot
- 1:19:07of different things or how? How are the foundation models
- 1:19:11sort of translating to the chips as best you can explain it?
- 1:19:16I I guess the way I would explain it is you know if you're
- 1:19:18trying to train a a a chat bot like ChatGPT, you train it by
- 1:19:24taking all of the language data in a a vast data set like
- 1:19:27Wikipedia. You essentially read Wikipedia
- 1:19:30and identify patterns, and so and.
- 1:19:33That's and that's like inference that you've been taught.
- 1:19:36That's training. That's training.
- 1:19:37So training is is identifying the pattern.
- 1:19:39So you know the what's the what? What is the next word I'm going
- 1:19:48to say? You know, you know what say, and
- 1:19:52Chachi Piti knows that because it's identified thousands of
- 1:19:55other instances where sentences like that ended with the word
- 1:19:59say. And it did that because it read
- 1:20:02lots and lots of books and articles and Reddit posts
- 1:20:06online. And then when I you go back and
- 1:20:08ask a question which is inference, it gives you a pretty
- 1:20:11good answer because it has a pretty good sense of what the
- 1:20:14most likely word is going to be. So you're saying they're they're
- 1:20:18training the chips based on a a lot of data and then it makes an
- 1:20:23inference about what it thinks the the sort of fill in the
- 1:20:26blank of of the prompt? Basically, yeah, inference is
- 1:20:30just an educated guess, a probabilistic guess.
- 1:20:33And what's cool about Chachi PT is that it's not just fill in
- 1:20:37one word, it's fill in dozens of.
- 1:20:39Words. I know, it's insane.
- 1:20:41One thing I'm still trying to get my head around is, you know,
- 1:20:44like DeepMind, right? Like when they trained, you
- 1:20:50know, computer systems, I guess to to compete on chess or Go or
- 1:20:55Dota. Like my sense is that you know
- 1:20:58they're using. Sort of this sort of scale data
- 1:21:01approach, but they also have like specific ways they're
- 1:21:04teaching it to think about specific problems.
- 1:21:08Or like, do you, are those types of approaches to AI also using
- 1:21:13these same chips, or are they still sort of invested in the
- 1:21:17same arms race? Yeah, well, the chips just do
- 1:21:21the math for you. And so they'll do what other
- 1:21:22math? Whatever math you ask them to
- 1:21:24do, Yeah. Exactly.
- 1:21:27And they're not. And then the interaction between
- 1:21:31like Nvidia's like programming language is super interesting
- 1:21:36because they're not like the the foundation models are not being
- 1:21:38written in that language, right? But they're they're sort of
- 1:21:40bridging. That's right.
- 1:21:43Yep. And it's it's just about, it's
- 1:21:46fundamentally allows them to optimize what's happening on
- 1:21:48these chips much more than maybe they would on like an AMD chip
- 1:21:52even if it were and it's it's it's it's easier, it's more
- 1:21:55familiar and therefore there's a big Moat around invidious
- 1:21:58position. Yeah.
- 1:22:01So I have you like become like were you bullish on like AI
- 1:22:06before writing this book or I'm curious how much your view has
- 1:22:10really changed on, yeah like I don't know like generalized
- 1:22:13intelligence or like the imminence of self driving cars
- 1:22:16or any of that. You know I I think that actually
- 1:22:19we've I I'm I'm in the long run, I'm very bullish but I think in
- 1:22:23in the the short run the last 12 months there's been all a whole
- 1:22:26lot of excitement of the imminent arrival of AGI which
- 1:22:30I'm I'm not so sure I see it happening in 2023.
- 1:22:34I think if you look at the trends, the trends are all very
- 1:22:37positive. AI system is getting better,
- 1:22:39more capable, but I think there there there was due to ChatGPT a
- 1:22:44popular sense that we're going to get some extraordinary
- 1:22:47products in our hands right away.
- 1:22:48And actually, it's going to be a couple of years before we see
- 1:22:51AI, the current wave of AI products deployed in a big way
- 1:22:55that meaningfully impacts economy or society.
- 1:22:57Yeah, I think we're all watching to see if this hype wave is
- 1:23:00about to end or not. I mean there's there's a world
- 1:23:03where this is sort of peak NVIDIA, right?
- 1:23:06Like we have this sort of mania around AI, everybody's running
- 1:23:10towards it and then it turns out, you know, chat GPD doesn't
- 1:23:15improve that much and self driving cars still have a lot of
- 1:23:18edge cases. And you know, the chips don't
- 1:23:21solve everything. Even if they do get better.
- 1:23:23Like you're saying, like, I don't know.
- 1:23:24How likely do you think that is? Or do you have a sense of, yeah,
- 1:23:27whether better years are ahead or this is sort of a peak moment
- 1:23:31for the company. Well it's it's been a very, very
- 1:23:34good year for the company. So I wouldn't be surprised if
- 1:23:36it's a it's it's at least a localized peak.
- 1:23:38But I I think you know for for AI applications most people
- 1:23:43think of what's the consumer application for AI, you know
- 1:23:46what's what's the iPhone of AI. But it seems to me that for the
- 1:23:50next couple of years actually most of the application and most
- 1:23:53of the money in AI will be in enterprises because enterprises
- 1:23:56that have vast data, they've got the desire to use it efficiently
- 1:23:59to monetize it. And so that's where I think
- 1:24:02we're actually already seeing a lot of the investment and a lot
- 1:24:04of the early products are happening in pretty boring
- 1:24:07places inside of enterprises. But that that's probably OK
- 1:24:10that's that's what's going to drive I think the the, the
- 1:24:14building of effective products in the long run.
- 1:24:17Enterprises, you know, businesses just translate it for
- 1:24:19like the regular person, you know, I used to work at
- 1:24:21Bloomberg. They've actually come out, I
- 1:24:23think with their own foundation model to show how they can train
- 1:24:26financial data. And you you have a good example
- 1:24:29of where you think corporations will put this approach and data
- 1:24:33to use. If you imagine a company like
- 1:24:37Walmart, they have to everyday decide what price to put all of
- 1:24:41their products, and they decide pricing based on a whole variety
- 1:24:45of different factors. What their competitors are
- 1:24:46pricing? The availability of products.
- 1:24:48Very complex process and that seems like a perfect use case
- 1:24:53for trying to use more advanced AI algorithms to set better
- 1:24:57prices more rapidly. Boring is it?
- 1:24:59Kmart this blue light discounts. What's Walmart?
- 1:25:01I forget. But yeah, who gets the discount?
- 1:25:03You know where, right? A lot of products to keep track
- 1:25:07of. Have you like?
- 1:25:10It feels so absurd to ask this, but like, what the chip business
- 1:25:14looks like in a world of AGI, generalized intelligence?
- 1:25:18Or like, have you gained that out at all?
- 1:25:20I mean, there are people who think it could happen, you know,
- 1:25:24five years from now or less. Like, do you have a view on it
- 1:25:28first of all and then second? Yeah, what?
- 1:25:30What would it mean for the chip world?
- 1:25:34Well I I guess my sense is that we're we're we're a long way
- 1:25:37away. I'm I'm still waiting for
- 1:25:38ChatGPT to accurately complete all my senses.
- 1:25:41You know still at a 90% rate and the other 10% are pretty bad.
- 1:25:46I think it's gonna require tremendous advances in
- 1:25:50semiconductors to make possible the tremendous increase in
- 1:25:54computing that more advanced AI systems will need.
- 1:25:56I mean that that's been the trend.
- 1:25:57The trend has been AI systems only advance.
- 1:26:00We apply more computing to them, and so if we want systems to be
- 1:26:03twice as good as they are today, we're gonna need something not
- 1:26:07too far off from twice as much compute to make it possible.
- 1:26:10Are there, you know there was this I I covered Righetti and
- 1:26:14sort of the quantum computing world.
- 1:26:17Do you have much optimism there? Do you spend time on them in in
- 1:26:21the book? You know, I not not in the book.
- 1:26:25I my sense in in, in, in speaking to people in the
- 1:26:29quantum world and then looking also at how in history new
- 1:26:32computing technologies have disseminated is that actually
- 1:26:36even revolutionary technologies, they're implemented slowly.
- 1:26:40And so suppose we get to the next couple of years, the first
- 1:26:43practical use case of quantum computing.
- 1:26:45It's going to be a years long process of beginning to
- 1:26:49implement that in all sorts of different computing use cases.
- 1:26:51And so I think we shouldn't expect to have, you know, a
- 1:26:54quantum powered iPhone anytime soon.
- 1:26:56It's it's insane to think like on the one hand we have.
- 1:27:01We have public companies that people are sort of speculating
- 1:27:04and betting on they yeah produce quantum computers but they
- 1:27:08really do nothing practical. I I you can go with I you know I
- 1:27:13swing back and forth. On the one hand it's like great
- 1:27:15it's amazing that our system will invest in such bleeding
- 1:27:19edge technology and give it a chance.
- 1:27:21On the other hand for the shareholders, you know like
- 1:27:24everybody could decide to give up on the effort with like
- 1:27:26higher interest rates, it's, I don't know, it's a crazy
- 1:27:29function of. The global economy, well, I I
- 1:27:33think it's interesting that a lot of the companies that are
- 1:27:35investing the heaviest are also big cloud computing operators.
- 1:27:39And that's I think because they believe that quantum will be
- 1:27:42actually most useful in a context where it's closely
- 1:27:46interlinked with huge volumes of classical computing.
- 1:27:50And so actually, we're going to need more advanced silicon chips
- 1:27:53to make quantum practically applicable.
- 1:27:55And do you know, is there any sense whether Quantum and the
- 1:27:58GPU sync up, or where they fit in that story?
- 1:28:03There There are many different paradigms for how you
- 1:28:06specifically sync classical and quantum computing, but right now
- 1:28:11nobody knows which of the many paradigms will win, if any of
- 1:28:15them. To wrap it up, I mean, what do
- 1:28:18you think the regular person should take from all this?
- 1:28:21Like it feels very far. From their lives like they they
- 1:28:26maybe don't even know what NVIDIA is.
- 1:28:28Like they tried ChatGPT, their kids using it for homework.
- 1:28:32It's not in their life yet. Like you know Silicon Valley
- 1:28:35loves to get itself worked up about things that don't
- 1:28:38sometimes translate. Don't like what what do you
- 1:28:40think the lesson or the the key thing for like the regular
- 1:28:44person right now is in terms of what's happening with these
- 1:28:47chips. I think if you went back to
- 1:28:491965, which is when Gordon Moore first set out the the phrase
- 1:28:53Moore's Law and you asked what's the impact of of of Moore's Law,
- 1:28:57the average person, the the answer in the short run was
- 1:29:00approximately nothing. But the answer in the long run
- 1:29:03is that it totally transformed society, economy, technology,
- 1:29:06everything, because we put computing and therefore
- 1:29:10semiconductors into basically every product that we rely on.
- 1:29:13And I think we should expect the same to be true for AI.
- 1:29:16You know, what does it mean for me tomorrow?
- 1:29:19Probably not much. And what is it going to mean in
- 1:29:21in 10 years and in 20 years when every aspect of human life is
- 1:29:25being impacted by it? Well, it'll be transformative in
- 1:29:28all sorts of ways, most of which we probably can't even imagine
- 1:29:30today. For the startup entrepreneur,
- 1:29:32where do you think there's opportunity to build a business?
- 1:29:36You know they, I mean maybe you think they should go out and
- 1:29:38start the next NVIDIA like it's a heavy lift, but like where
- 1:29:42where do you think really with the progress that we've made in
- 1:29:46these chips that there's real business opportunities still?
- 1:29:51Well, I think if you've got the idea of the next NVIDIA, you
- 1:29:54should absolutely go do it. But I I think you're, you're
- 1:30:00right that, you know, companies like NVIDIA, they they create
- 1:30:03the infrastructure on which many different types of systems can
- 1:30:06be built. And and if you go back to, you
- 1:30:09know, the smartphone for example, smartphones were
- 1:30:11themselves a platform owners could build lots of different
- 1:30:14things. And I think we're still in the
- 1:30:16Super early stages. We're basically in stage zero in
- 1:30:18terms of figuring out what are the ways you can create products
- 1:30:21out of generative AI systems. Right now, there are hardly any
- 1:30:24companies that make money selling their AI systems.
- 1:30:27And of course, the challenge for entrepreneurs is, you know,
- 1:30:30people who built early iPhone apps and Facebook apps, you
- 1:30:33know, they don't all pan out. Sometimes you come too early.
- 1:30:36You can be right that a technology is transformational
- 1:30:40and get the timing wrong. And I guess that's.
- 1:30:43The sort of confluence of luck and insight in, in the business
- 1:30:47world, it's a tough one to predict.
- 1:30:50Chris, this was awesome. Thank you so much for coming on
- 1:30:53the show. It's great.
- 1:30:54Great to talk to you. Thanks for having me.
- 1:30:56That's our episode on chips and big tech.
- 1:30:59Thanks for listening. I'm Eric Newcomer, your host,
- 1:31:03author of newcomer. Thanks so much to Max Child and
- 1:31:06James Wilsterman. Co founder Zavali and my long
- 1:31:08time friends shout out to Scott Brody, our producer Riley
- 1:31:12Kinsella, my Chief of Staff Gabby Caliendo at Volley who's
- 1:31:15helping organize the conference and playing a big role behind
- 1:31:19the scenes. Thank you to young Chomsky for
- 1:31:21the theme music. Please like, comment, subscribe
- 1:31:24on YouTube, give me a review on Apple podcast and subscribe to
- 1:31:29the sub stack newcomer.co. Go try out Volley on your Alexa
- 1:31:34play song quiz maybe we're going to have.
- 1:31:36Couple more episodes before Cerebral Valley.
- 1:31:39Cerebral Valley is on November 15th.
- 1:31:42We will publish our conversations to YouTube.
- 1:31:46We will probably publish them all.
- 1:31:47That's what we did last time. You can see the conversations
- 1:31:50from Cerebral Valley One, which is on March 30th.
- 1:31:54And then yeah, we'll play some of our favourites probably.
- 1:31:58We did a sort of distilled version in our podcast feed, so
- 1:32:02follow the podcast feeds for those.
- 1:32:04And yeah, I'll be covering it in the newcomer newsletter.
- 1:32:07So newcomer.co, Thanks so much.