Latest / Deep Values: Value Investing Deep Dives / Ilya Sutskever: Post-Scaling Paradigm and Its Financial Implications for the AI Stack
Transcript
- 0:00Welcome back to the Deep Dive. Today, we're cracking open a
- 0:03concept that really isn't just another tech trend.
- 0:07Researchers are calling this a structural inflection point of,
- 0:10well, historical magnitude in the AI sector.
- 0:13For the last decade, you know, AI progress felt predictable,
- 0:17almost mechanical. But now it feels like we're
- 0:19standing at the edge of a Cliff where the fundamental rules have
- 0:22just changed completely. And this chain has these
- 0:25profound long term financial consequences for the three
- 0:29giants really dominate this landscape, NVIDIA, Google and of
- 0:33course Apple, right. So our mission today is a
- 0:35serious one. We're going to try and
- 0:37synthesize the highest level technical research.
- 0:40I mean the stuff coming directly from the architects of modern
- 0:44AI. And pair that with the hardcore
- 0:45executive commentary, the financial modeling.
- 0:47Exactly. We're trying to give you a
- 0:49detailed road map for navigating this next era of computing
- 0:52because, well, the old assumptions about how AI grows
- 0:56just don't hold up anymore. And that road map, it really
- 0:59begins with a declaration from the very front lines of AGI
- 1:02research. It's articulated most clearly by
- 1:05Ilya Sutsgiver, the Co founder of Open AI.
- 1:08He says the age of scaling is definitively yielding to the age
- 1:12of research, or as some call it, the age of discovery.
- 1:16OK, let's define that for everyone right away.
- 1:18What exactly was the age of scaling?
- 1:20The age of scaling was an era of brute force.
- 1:23It was all about reliably getting better performance just
- 1:26by throwing, you know, exponentially more computation
- 1:29and vast amounts of data at the problem.
- 1:32It is an extremely successful paradigm.
- 1:33It gave us the foundational models we have today, but we've
- 1:36hit a ceiling. That era where just being bigger
- 1:39was the main measure of progress is it's over now.
- 1:42We're in a phase where ingenuity, new algorithms and
- 1:45just fundamentally different architectural approaches are
- 1:48needed to move the needle. And that is the crucial pivot
- 1:50because the old market anxiety, the one that still fuels a lot
- 1:53of that AI bubble talk, it rested on this idea that if AI
- 1:57research slowed down or if model training plateaued.
- 2:00Then compute demand would eventually slow down with it.
- 2:03Right, leading to a hardware collapse.
- 2:04Proceed some, and that thinking seems to be completely
- 2:07backwards. Now that we understand the
- 2:09physics of this new age of research, the pivot to these
- 2:13reasoning heavy models, as people call them.
- 2:16System 2 thinking, which we'll get into it doesn't signal less
- 2:20demand at all. Quite the opposite.
- 2:21It signals radically different and maybe most critically,
- 2:25continuous demand. This is where we have to
- 2:28introduce a key economic concept from our sources.
- 2:31This shift catalyzes A Jevons paradox in compute consumption.
- 2:35Yeah, the Jevons paradox, for anyone who's not familiar, it
- 2:38originally referred to the phenomenon where, you know,
- 2:40technological efficiency gained actually increased the total
- 2:43consumption of resource. Instead of decreasing it.
- 2:46Exactly because the resource becomes cheaper and more
- 2:48valuable to use. The classic example is making
- 2:51steam engines more fuel efficient instead of reducing
- 2:54coal use. People just found more valuable
- 2:56things to do with steam engines and the total demand for coal
- 2:59exploded. And in AI, the paradox says it's
- 3:02identical. As the models get more capable,
- 3:05more efficient at reasoning, meaning they can do more
- 3:08valuable tasks, the demand for the underlying compute just
- 3:12skyrockets. Exactly.
- 3:13We're willing to pay significantly more for a query
- 3:16that solves a complex, high valued business problem than we
- 3:19ever were for a simple search query.
- 3:21It's a structural demand driver. So the value proposition changes
- 3:25from simple retrieval. To sophisticated agency, it
- 3:28moves the industry from these episodic massive training runs,
- 3:32which you can think of as building a huge skyscraper, to
- 3:35continuous heavy duty inference, which is like the perpetual high
- 3:39load utility company that's powering an entire city 24/7.
- 3:43And that fundamental shift completely alters the investment
- 3:45thesis for our three core companies.
- 3:47NVIDIA, who supplies the power? Google, who's struggling to host
- 3:51and use it within their old economic model.
- 3:53And Apple, because they control that last mile of intelligence
- 3:56brought on the user's device. Let's start by really unpacking
- 3:59the technical reality that defined the last decade.
- 4:02It was all governed by the scaling laws.
- 4:04If you were an AI practitioner in the 20 tens, this formula was
- 4:08basically your Bible. The formula was, like I said,
- 4:11just incredibly elegant in its simplicity.
- 4:13It made AI progress feel almost manufactured performance gains.
- 4:19You'd measure them by things like reducing predictive error
- 4:21or cross entropy loss. They were a predictable power
- 4:24law function. You just plotted it on a graph.
- 4:27You did. You plotted performance based on
- 4:28three inputs. The models parameter count, so
- 4:31it's size, the data set size, the quality and quantity of
- 4:34text, and the compute budget you allocated, the energy and time
- 4:37spent. So if you needed a 10% gain in
- 4:39performance, you knew roughly that you had to crank up one of
- 4:43those variables by a factor of X.
- 4:45It just it took the R&D risk way down.
- 4:47Absolutely, and his predictability gave rise to what
- 4:50Rich Sutton famously called the bitter Lesson.
- 4:53I find the bitter lesson just a fascinating concept.
- 4:55It really encapsulates the entire ethos of that era.
- 4:58It was this humbling realization that just leveraging computation
- 5:03and massive data reliably and consistently outperformed any
- 5:07clever human design heuristics or complex hand coded
- 5:10algorithms. All that careful engineering.
- 5:12Right. Researchers spent years trying
- 5:14to engineer AI with explicit logical rules.
- 5:17But when the big transformer models came along, their simple
- 5:20goal of just predicting the next token backed by all that data,
- 5:24it just crushed those engineered systems.
- 5:26You didn't need a smarter algorithm, You just needed a
- 5:28bigger checkbook, more time, and, critically, access to the
- 5:32entire human corpus of knowledge.
- 5:34It turned AIR and D into a reliable manufacturing process.
- 5:39But every manufacturing process needs a reliable raw material,
- 5:42and this is where we hit the hard bottleneck, the thing that
- 5:44ends the age of scaling the data wall.
- 5:47The data wall is the exhaustion of the core input, high quality
- 5:51human generated text. Subskeever explicitly calls this
- 5:54the fossil fuel of AI. The original training corpus
- 5:56included everything unique, the public Internet, GitHub,
- 5:59academic papers, Wikipedia, all the digitized books.
- 6:02The sheer scale of that effort means we have effectively
- 6:06tokenized the entire unique, non repetitive corpus of human
- 6:10knowledge in about a decade. That is genuinely A staggering
- 6:14thing to realize when you stop and think about it.
- 6:16We burn through hundreds of millions of years of accumulated
- 6:20human thought. All the unique stories, code,
- 6:23arguments, history in 10 years. And now what?
- 6:27Exactly. Now what?
- 6:28And now we're seeing the brutal reality of diminishing returns.
- 6:33Simle scaling is hitting a wall. Not because models hysically
- 6:36can't get bigger. We can build models with
- 6:38trillions of arameters. It's because the unique signal
- 6:41in the training data is exhausted.
- 6:43If you keep training on the same pool of data, you just get
- 6:46overfitting, not deeper intelligence.
- 6:48The model starts memorizing the noise, including all the biases
- 6:51and errors, instead of learning new generalizable concepts.
- 6:54And this introduces the structural long term problem
- 6:57that's going to plague future training runs.
- 6:59The slot feedback loop. Exactly.
- 7:01This is an insidious form of data degradation, as these AIS
- 7:04we've just trained start to proliferate across the Internet,
- 7:07writing articles, generating product descriptions, you know.
- 7:10Filling up forms. The data available for training
- 7:12the next generation and becomes increasingly filled with content
- 7:15that was actually generated by the previous AIS.
- 7:18This is the slop, as researchers call.
- 7:20It so the data just becomes homogenized, synthesized.
- 7:23Precisely, If you train the next generation on the last
- 7:26generations output, you're basically feeding the model a
- 7:29diet of recycled, statistically homogeneous and increasingly
- 7:33bland information. It dilutes the novelty and the
- 7:37true human ingenuity that was in the original corpus.
- 7:40This is now a massive, expensive engineering problem.
- 7:44The biggest models risk becoming experts at sounding fluent but
- 7:47in a closed loop without any real external causality or
- 7:50originality. And the ultimate manifestation
- 7:53of this plateauing, the structural weakness of the age
- 7:56of scaling, is what researchers are calling the generalization
- 7:59gap. Right.
- 8:00Despite training on trillions of tokens and unimaginable volume,
- 8:04these state-of-the-art models still have fundamental flaws.
- 8:06They hallucinate, they struggle with novel logical puzzles, and
- 8:10most importantly, they fail to apply knowledge outside the
- 8:12specific distribution they were trained on.
- 8:15They can't generalize efficiently.
- 8:16And this is the inefficiency that the age of research has to
- 8:19solve. It's the key bottleneck.
- 8:21Now a human can grasp a brand new complex concept from just a
- 8:25few examples. A paragraph in a textbook,
- 8:28right? A large language model, even a
- 8:29state-of-the-art 1, still needs billions of tokens to get that
- 8:33same level of robust generalization.
- 8:36And now that data volume is maxed out this inefficiency, the
- 8:39sheer computational cost of gaining new knowledge, becomes
- 8:43the primary problem. So if the old recipe is broken
- 8:46just scaling massive static data sets, we have to fundamentally
- 8:50shift where we look for progress.
- 8:52We transition into the age of research, which is defined by
- 8:55the search for entirely new algorithms or new recipes that
- 8:59move beyond the simple passive task of just predicting the next
- 9:02token in the sequence. This is the move from sounding
- 9:04like a human to, ideally, thinking like an expert.
- 9:07Exactly. It's the necessary shift from
- 9:09system one to system 2. Thinking System 1 is fast,
- 9:13intuitive, pattern matching. That's what the last generation
- 9:16of LLM's mastered. Incredible fluency system 2
- 9:19though is slow, deliberative, reasoned, and crucially, it
- 9:23prioritizes verifiable accuracy over speed.
- 9:27The goal is to make the AI perform rational multi step
- 9:30deduction, not just fluent single step prediction.
- 9:33To do that, the model has to engage in active search.
- 9:36It can't just rely on its stored weights.
- 9:38Active search is a requirement for System 2.
- 9:41It means the model has to learn to dynamically explore solution
- 9:44spaces, to verify its own tentative answers internally and
- 9:48apply logic dynamically rather than just spitting out the most
- 9:51probable next word. It needs the capacity to hit a
- 9:54dead end in its thought process, backtrack, and try a different
- 9:57line of reasoning, just like a human solving a complex math
- 10:00problem. And this active multi step
- 10:02search is the technical engine that's driving the next wave of
- 10:05hardware demand. It brings us to the most
- 10:07critical financial concept in this whole shift test, time
- 10:10compute or TTC. Test time compute is the
- 10:13fundamental game changer for the hardware economy.
- 10:16It means allowing the model to spend exponentially more
- 10:19computational resources at the moment of inference when you ask
- 10:23the question to improve the quality, accuracy and depth of
- 10:27the answer. So you're literally paying the
- 10:29model to take time to think through the problem before it
- 10:32replies precisely. So instead of just an immediate
- 10:35response, the model is generating all these internal
- 10:38hidden steps that are invisible to the user but are consuming
- 10:41massive amounts of compute. That's it exactly.
- 10:45The mechanism for this involves techniques like chain of thought
- 10:47prompting or the even more intensive tree of thoughts and
- 10:51search algorithms. Before giving you the final
- 10:54output, the model generates a complex sequence of intermediate
- 10:57reasoning steps. Think of it a planning phase, a
- 11:00calculation phase, a critique of that calculation, maybe
- 11:03exploring two or three different internal solutions in parallel,
- 11:06and then finally selecting and revising the best path before it
- 11:09outputs the final answer. These intermediate latent
- 11:12thought processes consume massive compute, but they are
- 11:15absolutely essential for achieving that system to
- 11:17accuracy. This is where the financial
- 11:20physics of the entire sector changes dramatically.
- 11:22This is the move from linear to exponential growth in compute
- 11:26demand. Let's talk about that economic
- 11:28multiplier of System 2, that infamous 23 X spike.
- 11:32This is the core thesis for why compute demand isn't slowing
- 11:35down, it's intensifying. Let's just walk through the
- 11:37numbers from the sources, comparing a typical interaction
- 11:40today versus a system 2 interaction tomorrow.
- 11:44In the traditional age of scaling inference model, say
- 11:47asking a simple factual question like what's the capital of
- 11:50France? The model process is maybe 100
- 11:52input tokens and generates 50 output tokens.
- 11:54The total computational steps or token passes is around 150 OK.
- 11:58150 passes that defines the low cost economics of classic search
- 12:02or a simple chat bot. Cheap, fast, scalable.
- 12:05Now consider a high value, complex reasoning query, the
- 12:09kind of task that actually justifies the high cost of the
- 12:12machine. For instance, asking an agent to
- 12:15draft a comprehensive multi stage business strategy for a
- 12:18software launch in three different regulatory markets,
- 12:21identify core risks and provide three potential mitigation
- 12:25plans. That's a huge shift from a
- 12:27simple query. It is.
- 12:28The model can't just retrieve an answer, it has to generate an
- 12:31answer using logic. So it's internal thought process
- 12:34might look something like this. Generate an initial strategic
- 12:37plan that's 500 tokens. Critique that plan for legal
- 12:40conflicts and other 500 tokens. Explore market option A in
- 12:43depth. That's 1000 tokens.
- 12:45Explore option B another thousand tokens.
- 12:47Verify the chosen strategy against financial models, 500
- 12:50more tokens and then finally it outputs the concise 50 token
- 12:54executive summary. Wow.
- 12:55OK. So if we tally that up, the
- 12:57internal plan, the critiques, the parallel searches, the
- 12:59verification, we're looking at something like 3550 token passes
- 13:03for just one interaction. Exactly.
- 13:05The source analysis shows that a single high value user
- 13:08interaction in the age of research consumes approximately
- 13:1123 times times more compute than the traditional simple model.
- 13:15This is the structural driver for Nvidia's continuous hardware
- 13:18demand. It's not just more users, it's
- 13:20users asking for exponentially more deep continuous effort per
- 13:24query. And we'll pay for it because the
- 13:26value it generates is immense. So if human data is now scarce
- 13:30and expensive, where does the new proprietary high quality
- 13:35training material come from to feed these 23 X reasoning
- 13:37models? This leads us to synthetic data
- 13:40and self play. Right, we're moving from
- 13:42harvesting human data, the fossil fuel, to refining data
- 13:45internally. You could call it synthetic
- 13:47solar power. Self play, which was famously
- 13:49used by DeepMind in systems like A0 to master chess and go by
- 13:53playing against itself, is now being applied to general
- 13:56reasoning. The model acts as both the
- 13:57student and the teacher at the same time.
- 14:00How does that work in practice for, say, an enterprise language
- 14:03model? You define two critical
- 14:04perpetually running roles within the compute cluster, the
- 14:08generator and the verifier. The generator attempts to solve
- 14:11a problem, say writing a massive piece of bug free code or
- 14:15solving a complex logic puzzle and the.
- 14:17Verifier checks its homework. Essentially, yes.
- 14:21The verifier then checks the solution's correctness in
- 14:23domains like math or code. Truth is objective.
- 14:27The proof holds up, the code compiles and runs, The logic is
- 14:30sound. And if the solution is correct,
- 14:32that entire internal multi step reasoning trace, the attempt,
- 14:36the exploration, the self critique, that becomes a new
- 14:39proprietary high quality training example for the next
- 14:41generation of the generator. Precisely this is the virtuous
- 14:45cycle of data generation. It converts electricity and
- 14:48silicon into valuable non dilutable intellectual property
- 14:51high quality synthetic data. Companies that own massive
- 14:54compute clusters like Google and Microsoft Open AI are basically
- 14:58running data refineries 24/7, and this creates an unassailable
- 15:02Moat because having the best model lets you generate the best
- 15:04data, which lets you train the next best model, and so on.
- 15:07And you mentioned the verifier. It seems like the need for a
- 15:10highly reliable verifier model hints that we haven't truly
- 15:14overcome the bitter lesson. If you have to spend all this
- 15:17compute building a rule checker, aren't you just building a new,
- 15:21more expensive kind of human designed heuristic?
- 15:23That is a really insightful question and it captures the
- 15:26tension in the age of research right now.
- 15:29The difference is that the verifier isn't necessarily hand
- 15:32coded with human rules. It's trained using reinforcement
- 15:35learning and self play to be an objective truth seeker.
- 15:39But yes, the verifier is the algorithmic bottleneck.
- 15:42Sutskeever highlights that these proper verifier systems are
- 15:45critical because now the ability to reliably check facts and
- 15:48discriminate between correct and correct reasoning is as
- 15:51important as generating the steps in the first place.
- 15:54So to synthesize this whole technical pivot, the unit of
- 15:56production is no longer the parameter the size of the.
- 15:59Model. It's the reasoning trace.
- 16:01And that high value trace costs 23 times more compute.
- 16:06Before we move into the profound financial implications this all
- 16:09has for NVIDIA, Google, and Apple, if you're finding
- 16:13yourself wrestling with the complexities of this structural
- 16:17inflection point, this shift from scaling to reasoning,
- 16:20you're probably thinking hard about the overall health and
- 16:22temperature of the market. That's why we encourage you to
- 16:25check out tools that can help cut through the noise.
- 16:27Indeed, we highly recommend you welcome to search and download
- 16:31the iOS app Buffett Indicator from the Apple App Store.
- 16:35It provides daily updates on the critical metric Warren Buffett
- 16:37uses to gauge overall market valuation against GDP.
- 16:41It's an essential tool for monitoring market frothiness,
- 16:44especially during times of rapid structural change like this.
- 16:47Right. Whether you're worried about an
- 16:48AI bubble or celebrating this new compute paradigm, keeping
- 16:51track of the total market temperature is just crucial.
- 16:54So search and download the iOS app Buffettt Indicator from the
- 16:57Apple App Store. Now that we've established the
- 16:59physics of reasoning that 23 X spike and continuous demand,
- 17:03let's pivot hard to the financial road map.
- 17:06This technical shift directly addresses the biggest financial
- 17:09anxiety surrounding NVIDIA. The AI bubble anxiety.
- 17:12Exactly the fear that demand collapses once the initial
- 17:15training clusters are built and amortized.
- 17:17And the age of research is the definitive counter narrative.
- 17:21It shifts the entire story from cyclical risk, the lumpy demand
- 17:25of training, to structural, sustained utility like demand.
- 17:30We need to detail that shift from episodic training to
- 17:33continuous reasoning demand because that is the foundation
- 17:36of Nvidia's future valuation. Training demand was lumpy,
- 17:39predictable, but finite. You build a massive H100 cluster
- 17:43for a model, you train for three months, you pay the bills, and
- 17:46that revenue disappears until the next model generation.
- 17:48That's project revenue. Right, it's prone to cycles, but
- 17:51inference demand, the actual usage by billions of people is
- 17:54constant. It's utility.
- 17:56Like every interaction, every query, every agent execution
- 18:00consumes compute and the shift to test time.
- 18:02Compute doesn't just make inference consumption constant,
- 18:06it makes it explode in intensity.
- 18:08Importantly, financial projections consistently show
- 18:11that the aggregated cost of inference over a model's
- 18:14lifetime is now forecast to greatly exceed the initial one
- 18:18time cost of training it. This solidifies that continuous
- 18:22utility like demand for Nvidia's hardware.
- 18:25Let's revisit the Jevons paradox for NVIDIA here.
- 18:28Why do efficiency gains and chips like doubling the
- 18:30performance with Blackwell not reduce total demand?
- 18:33Why does it actually increase it?
- 18:34It's purely a function of value. Think about historical
- 18:37parallels. When the efficiency of lighting
- 18:38improved dramatically, moving from oil lamps to electricity,
- 18:41people didn't use less. Light.
- 18:42No, they lit up entire cities. They lit up their homes more
- 18:45brightly. The cost per unit of
- 18:46illumination dropped, but the total consumption of energy for
- 18:49lighting skyrocketed because we found exponentially more
- 18:52valuable uses for it. So applied to AI, the value of
- 18:55the output has increased exponentially.
- 18:57Exactly. When AI was just giving you a
- 19:00slightly better search result, the value is marginal.
- 19:03Now with system 2 reasoning, AI can replace or augment
- 19:07specialized high cost human labor.
- 19:09Lawyers, engineers, developers. If a reasoning query costs a
- 19:13dollar but saves a user an hour of specialized labor that costs
- 19:16hundreds of dollars, we will pay that dollar thousands of times a
- 19:20day. This massive increase in value
- 19:23justifies the continuous aggressive infrastructure build
- 19:26out. Which means Nvidia's hardware
- 19:28strategies, specifically the Blackwell architecture B200, is
- 19:32perfectly aligned with the physics of reasoning.
- 19:34It's not just a speed bump. Blackwell is not just a general
- 19:37performance increase, it is optimized for the specific
- 19:39demands of reasoning. Let's look at two key features,
- 19:42FP4 Precision and the Envy link switch.
- 19:45OK. Starting with FP-44 bit floating
- 19:47point precision, we've heard about 16 bit for training, but
- 19:50why is 4 bit suddenly so critical for reasoning?
- 19:52Doesn't reducing the bit depth reduce accuracy?
- 19:55It's a trade off, and it's specifically tailored for
- 19:58inference. Training needs high precision,
- 20:00like 16 bit, because models need that fine granularity to adjust
- 20:05their weights correctly. However, once the model is
- 20:08trained, you can quantize those weights down to a much lower
- 20:11precision, like 4 bit. Without a major loss in output
- 20:14quality. Why is that?
- 20:16Because reasoning relies less on those fine grained weight
- 20:18adjustments and more on rapid data movement and throughput,
- 20:22FP4 allows the chip to handle four times the amount of data
- 20:25and calculations per cycle compared to 16 bit.
- 20:28It effectively quadruples the speed of the reasoning engine.
- 20:31And the second feature at the end of Link Switch, connecting
- 20:33up to 72 GPU's into a unified Super Chip.
- 20:37Why is that so critical for test time compute?
- 20:39It's a requirement for sophisticated System 2 thinking.
- 20:42Remember our Tree of Thoughts example?
- 20:44The model is running parallel searches, exploring multiple
- 20:48branches of logic at once. These branches need to
- 20:51communicate massive amounts of intermediate data.
- 20:53The thought traces between the different ships.
- 20:56Traditional networking has too much latency.
- 20:58The Envy link acts as the corpus callosum.
- 21:01A high speed interconnect let's the AI brain think as one
- 21:04massive unified entity. Let's look at the numbers cited
- 21:07in the synthesis, because this paints the whole picture for
- 21:10Nvidia's transition the $300 billion data center.
- 21:13The projection is aggressive, but it's grounded in the
- 21:15structural demand transformation.
- 21:17You can trace the transition clearly.
- 21:19In fiscal year 2024, Nvidia's data center revenue was $47.5
- 21:23billion. This was mainly the initial
- 21:26episodic training boom, the big H100 clusters in FY20 25 that
- 21:32ramps up to an estimated $110 billion.
- 21:35This phase is fueled by the large scale deployment of those
- 21:37trained models into high volume inference.
- 21:39So the immediate surge is still about deployment, not the full
- 21:42reasoning spike just yet. Correct.
- 21:44The real inflection point, the moment the age of research truly
- 21:47hits the balance sheet, is fiscal year 2026.
- 21:51Estimates suggest revenue of $170 billion to $200 billion as
- 21:55the Blackwell ramp begins, driven specifically by the Test
- 21:58Time Compute era. And the long term forecasts?
- 22:00By FY20 27, with the next architecture Ruben likely
- 22:05entering the market and ubiquitous deployment of System
- 22:072 reasoning, we could see data center revenue surging to an
- 22:11estimated 250 to $300 billion annually.
- 22:14So the conclusion on NVIDIA is clear.
- 22:16They're not facing a demand Cliff or a bubble bursting.
- 22:18No, They're facing a structural demand transformation where
- 22:21they're recurring continuous revenue of inference, now
- 22:24intensified by reasoning, far outstrips the initial episodic
- 22:28revenue of training. OK, let's pivot to Google.
- 22:31Because Google is technically, I mean, exceptionally well
- 22:33positioned for the age of research.
- 22:35Deepmind's culture is perfectly aligned with this whole thesis.
- 22:38It is. But this shift threatens the
- 22:40golden goose search. This is the classic innovator's
- 22:44dilemma. The new technology is
- 22:46technically superior but economically incompatible with
- 22:49the existing highly profitable business model.
- 22:52And the conflict between cheap retrieval and expensive
- 22:54reasoning is just brutal for Google.
- 22:56It's a volume versus value problem.
- 22:59Search economics are built on massive volume, 8.5 billion
- 23:02queries a day and extremely low cost.
- 23:05The cost per query is less than a penny, latency is
- 23:08milliseconds, and monetization through ads is incredibly
- 23:12efficient. A money printing machine.
- 23:13The definition of one right reasoning economics flips that
- 23:16entire thing on his head. Using our 23X multiplier, the
- 23:20cost per query jumps from under a penny to maybe deer dollars
- 23:23and $20.50. Latency moves from milliseconds
- 23:26to seconds or even minutes. And monetization is really
- 23:29difficult because inserting ads into a single cohesive,
- 23:32generated answer is disruptive. The cost threat is immense.
- 23:35It is, the sources cite, an estimated $36 billion impact on
- 23:39operating income. If only 10% of their daily
- 23:41searches shift into that high cost reasoning mode like AI
- 23:44overviews, that's a massive imminent friction point.
- 23:47Yeah, Google has technical hedges that nobody else has.
- 23:50Their research DNA is maybe their greatest asset.
- 23:53Their research culture is ideal for this new age.
- 23:56DeepMind has always prioritized fundamental AGI research,
- 24:00finding those new recipes that set Skewer talked about.
- 24:04They're uniquely positioned to solve the algorithmic complexity
- 24:07of System 2 through breakthroughs, not just brute
- 24:10force. And their multimodal advantage
- 24:12becomes critical now that the text data was drying up.
- 24:15Correct, if human text was the fossil fuel.
- 24:18Google owns the world's largest untapped oil reserve of video
- 24:21and audio data. YouTube video is the next
- 24:25frontier for training reasoning models because it intrinsically
- 24:28teaches the model about physics, causality, and time, things text
- 24:32alone can't teach efficiently. This gives Google a proprietary
- 24:35data mode for the age of research.
- 24:37And their final critical hedge, their secret weapon against that
- 24:40$36 billion cost threat, is the TPU, the Tensor Processing Unit.
- 24:44The TPU is their defensive maneuver.
- 24:47Because Google is vertically integrated designing their own
- 24:50chips, they completely bypass the massive NVIDIA margin.
- 24:54That's 75% profit premium NVIDIA charges.
- 24:57This drastically lowers their internal cost for test time
- 25:00comute. And the Tus are technically
- 25:02superiore for their secific tasks, right?
- 25:05They're custom built for transformer inference.
- 25:07Reports say they're consistently 15% to 30% more efficient for
- 25:11this reasoning workload than general purpose GPU's.
- 25:14This efficiency, combined with the margin savings, gives them
- 25:17strategic sovereignty over their own cost curve.
- 25:20So what's the financial outlook for Google?
- 25:22Best tech, Worst financial position?
- 25:24You have to expect continued elevated CapEx.
- 25:27They'll keep spending $50 billion plus annually to build
- 25:30out this TPU infrastructure. In the short term, expect margin
- 25:34compression as the cost for search rises, but the long term
- 25:37play is revenue expansion if they can monetize agency.
- 25:41Let's elaborate on agency. That's the key to escaping the
- 25:43dilemma, right? Absolutely.
- 25:45If the model is just providing links, the value is low.
- 25:49If the model can book your flight, schedule your meeting,
- 25:51or write the legal brief if it acts as a digital agent, the
- 25:55value is far higher than an ad click.
- 25:57They have to shift the users perspective from free search to
- 26:01paid agent services. So they have the best map, but
- 26:04they're sailing through the roughest financial waters.
- 26:07No, Apple. Apple has often been seen as
- 26:09late to this party, but the physics of reasoning, especially
- 26:13the high compute requirements of TTC, play perfectly into its
- 26:16core strategy, edge computing and privacy.
- 26:19The main physical constraint is obvious.
- 26:21You can't run a trillion parameter model on an iPhone,
- 26:24but the age of research provides the technical escape route
- 26:27through techniques like distillation and quantization.
- 26:30Explain what that process does for the average user.
- 26:32Distillation is where a massive, powerful teacher model is used
- 26:36to train a much smaller, resource efficient student
- 26:38model, say 3 or 7 billion parameters.
- 26:41The goal is to transfer the sophisticated reasoning
- 26:43capabilities of the large model into the tiny model, making it
- 26:47runnable on a phone. And this relies on a hybrid
- 26:49architecture, which is the cornerstone of Apple
- 26:52intelligence. Yes.
- 26:53The approach is brilliant. The on device model handles
- 26:56personal context and basic reasoning, prioritizing absolute
- 27:00privacy. The complex queries are routed
- 27:02to their secure private cloud compute.
- 27:05However, even the on device model, when it's doing System 2
- 27:08reasoning demands massive sustained memory.
- 27:11And this brings us to the core driver of the consumer side of
- 27:15the next cycle, the hardware super cycle where RAM is the new
- 27:19gold. This is a critical point.
- 27:20To run any reasoning model locally, even a distilled one,
- 27:24you need huge amounts of fast memory for two reasons.
- 27:27First, the context window. The model has to hold all your
- 27:30relevant personal data, emails, messages, calendars in active
- 27:33memory. 2nd, and more critically for test time compute is the KV
- 27:37cache. OK, let's take a moment to
- 27:39explain the KV cache. It's a technical concept, but
- 27:42it's crucial for memory consumption.
- 27:44The KV cache is basically the models short term memory or it's
- 27:47scratch pad. When a model processes a prompt,
- 27:50it generates key and value vectors for every token.
- 27:54When it engages in a complex reasoning chain like our 3550
- 27:58Token pass example, it has to store every intermediate thought
- 28:02trace in that cache. So if the reasoning chain is 23
- 28:05times longer, the KV cache grows exponentially.
- 28:08Exactly. For sophisticated 7B parameter
- 28:10model running a deep System 2 reasoning trace, the cache can
- 28:13quickly consume HEB10GB or even more RAM.
- 28:17Current iPhones with 8GB of RAM are borderline.
- 28:20They can run a simple model, but they cannot run sophisticated
- 28:23high fidelity System 2 agents with acceptable speed.
- 28:26So to truly run high fidelity reasoning locally, future
- 28:29iPhones, the iPhone 17 or 18, will need 12 GB or even 16
- 28:32gigabytes of RAM as a structural necessity.
- 28:35And that turns RAM capacity into the ultimate obsolescence
- 28:38driver. People used to upgrade for a
- 28:39better camera, now they will upgrade because their old phone
- 28:41literally cannot think fast enough to run the new generation
- 28:44of egentic apps. And the Neural engine NPU is the
- 28:47primary metric now. Right, the NPU becomes the main
- 28:50performance metric. Chips like the A 17 Pro or M4
- 28:54are marketed on their MPU capability because that's what
- 28:57determines the speed of on device reasoning.
- 28:59This drives the shift from dumb apps to agentic apps.
- 29:03So the final verdict on Apple, How do they monetize this?
- 29:06Through their traditional strength, hardware margins and
- 29:09premium pricing, they don't need an AI subscription.
- 29:12They just need to make the hardware mandatory for the new
- 29:15experience. And critically, privacy becomes
- 29:18a valuable luxury good. Their promise, what happens on
- 29:21your iPhone stays on your iPhone, is an increasingly
- 29:24valuable defensive Moat. They are the defensive growth
- 29:26pick and shovel for the consumer side of this new age.
- 29:29Let's broaden our lens, because this shift has implications far
- 29:32beyond just tech. This explosion in test time
- 29:34compute has massive long term energy constraints.
- 29:37It fundamentally converts electricity and silicon into
- 29:40intelligence, and while inference is more efficient per
- 29:42calculation than training, the sheer unrelenting volume of
- 29:46continuous reasoning that 23 X spike exacerbates the energy
- 29:51veracity of the entire AI ecosystem.
- 29:53We're creating a new industrial demand for power that is growing
- 29:56exponentially. And the sources frame this as
- 29:59driving huge derivative industries.
- 30:01Yes, the forecast for infrastructure spend, often
- 30:05cited as $3 trillion over the next decade, is largely driven
- 30:08by power generation and cooling requirements.
- 30:11This means energy infrastructure itself, grid modernization,
- 30:15nuclear power becomes a direct derivative play on this thesis.
- 30:18And let's address the elephant in the room, the timeline to
- 30:21AGI, and what this age of research suggests about its
- 30:23financial meaning. Sudskeber provides a broad
- 30:26timeline convergence on AGI within 5 to 20 years.
- 30:30The distinction is critical for investors.
- 30:32If it's five years away, the market is in a singular frenzy.
- 30:36However, the shift to the age of research suggests that the 20
- 30:38year timeline is highly plausible because there are
- 30:40still hard scientific problems left to solve.
- 30:43A 20 year trajectory supports a long sustained industrial cycle
- 30:47similar to the build out of the Internet, justifying the
- 30:49continuous CapEx we're seeing. Absolutely.
- 30:51It's not an imminent singularity event.
- 30:54It's a decades long revolution requiring sustained
- 30:58infrastructure investment. So, final synthesis, what is the
- 31:01new alpha for the investor in the age of research?
- 31:04The age of scaling was simple by the biggest clusters.
- 31:07The age of research demands architectural nuance.
- 31:10The new alpha lies in understanding the specific
- 31:13requirements for high latency, high value thought processes.
- 31:17Who can deliver system to thinking most efficiently?
- 31:19Let's summarize the conclusions for our Trinity based on the
- 31:22physics of reasoning. NVIDIA is facing demand
- 31:24transformation, mitigating their cyclical risk and setting them
- 31:28up as the sovereign of continuous utility compute.
- 31:31That remains A bullish outlook. Google has the best technical
- 31:34blueprint, but they face immense margin pressure from the cost of
- 31:38reasoning. They have the capability, but
- 31:40the execution risk is high. We're cautiously bullish.
- 31:44They have to solve the innovator's dilemma, and Apple
- 31:47is driving the necessary consumer hardware super cycle.
- 31:50They're leveraging the high memory demands to turn privacy
- 31:53and on device compute into a luxury commodity.
- 31:56That's a strong defensive growth play.
- 31:58The core insights Setskeeper provided is that the industry
- 32:01has exhausted hundreds of millions of years of accumulated
- 32:04human thought in one decade. Now they're building machines
- 32:07that generate their own proprietary knowledge.
- 32:09So the final thought I want to leave you with is this Given the
- 32:12efficiency of self play and verifier systems, how quickly
- 32:16will the rate of synthetic data generation accelerate the
- 32:18timeline for entirely new scientific discoveries?
- 32:21Will we look back on the age of scaling as the last time human
- 32:24knowledge was the limiting factor?
- 32:25It truly changes everything. And that leads to a profound
- 32:30final thought experiment for you to consider.
- 32:32What are the deep ethical and economic implications of the
- 32:35verifier model? If we are successfully building
- 32:39AI systems that are demonstrably better at checking facts and
- 32:42verifying correctness than they are at generating the initial
- 32:44ideas, what new societal structures will that enable?
- 32:48What happens when our ultimate most reliable source of truth is
- 32:52a synthetic automated reasoner? Fascinating question for the
- 32:55next decade of discovery. We hope this deep dive provided
- 32:58you with the clarity you needed to navigate this incredibly
- 33:01complex and incredibly exciting inflection point.
- 33:04Thank you for joining us for this deep dive into the age of
- 33:06research.