Latest / The Neon Show / Why Coding is the Fastest Path to AGI | Turing CEO Jonathan Siddharth

Transcript

Why Coding is the Fastest Path to AGI | Turing CEO Jonathan Siddharth · The Neon Show

• 0:00
  SPEAKER_01: OpenAI came to us when they were training 253 and they wanted to teach 253 to COVID.
• 0:04
  SPEAKER_01: So we collaborated with them on teaching the models to code, and then after we launched, it was the fourth of the API.
• 0:14
  SPEAKER_00: Turing AI enterprises have secured significant funding, including a$111 million investment and raising a valuation of$2.2 billion.
• 0:21
  SPEAKER_01: So Turing is the one company that works with the Frontier AI labs and Fortune 500 enterprises.
• 0:28
  SPEAKER_01: A lot of problems in life are math problems, and code is a way to express and solve math problems.
• 0:36
  SPEAKER_01: If you can solve coding, tool use, reasoning, and multimodality, you have the keys to superintelligence.
• 0:43
  SPEAKER_01: I used to think about why do we need so much compute?
• 0:46
  SPEAKER_01: Now it's clear to me.
• 0:47
  SPEAKER_01: I think humanity is going to be reasoning bound and compute bound to solve some of our biggest problems.
• 0:54
  SPEAKER_02: Which are the kind of companies that you would invest?
• 0:56
  SPEAKER_01: I would bet on companies that are an input to I would also bet on companies that are betting on a future where.
• 1:07
  SPEAKER_02: Hi, this is Sadhhat Alwaliya.
• 1:09
  SPEAKER_02: Welcome to the Neon Show.
• 1:11
  SPEAKER_02: I'm your host and managing partner of Neon Fund, a fund that has invested at seed stages in some of the best enterprise AI companies coming from India, building for the US market and for the globe, like Atomic Work, Spot Draft, CloudSec.
• 1:25
  SPEAKER_02: Today I have with me Jonathan Sidhat, co-founder and CEO of Turing.
• 1:30
  SPEAKER_02: Turing is accelerating superintelligence.
• 1:33
  SPEAKER_02: Jonathan, Turing is one of the most defining AI companies of our era.
• 1:38
  SPEAKER_02: It's such a proud thing to see that, you know, the founders who are from, you know, you're from Chennai, you know, and achieving so much today.
• 1:48
  SPEAKER_02: So first of all, congratulations on all the success.
• 1:51
  SPEAKER_01: Thank you, Suddart, and thank you for having me.
• 1:53
  SPEAKER_01: It's great to be here.
• 1:54
  SPEAKER_01: And it's impressive how large the neon show is in terms of subscribers.
• 2:00
  SPEAKER_02: No, thank you.
• 2:01
  SPEAKER_02: So, Jonathan, uh, want to understand from you.
• 2:04
  SPEAKER_02: Turing started in the hiring business when you initially started.
• 2:09
  SPEAKER_02: You became a unicorn in that business.
• 2:12
  SPEAKER_02: And then you you started the business of being the provider to the Frontier AI labs on the data side, right?
• 2:21
  SPEAKER_02: How did this transition happen?
• 2:23
  SPEAKER_02: If you can walk us through the journey.
• 2:24
  SPEAKER_01: Great.
• 2:25
  SPEAKER_01: Maybe for the audience, I'll give a little bit of an overview of what Turing does, and then I'll walk through the journey for how we got here.
• 2:33
  SPEAKER_01: So our mission is to accelerate superintelligence to drive real economic progress.
• 2:40
  SPEAKER_01: And we do that in two specific ways.
• 2:43
  SPEAKER_01: First, we work with all the Frontier AI labs to advance the models in SUE, Enterprise, and Frontier STEM by providing high-quality data, evals, and oral environments.
• 2:56
  SPEAKER_01: In doing that, we discover the jagged intelligence of these Frontier models and agents.
• 3:03
  SPEAKER_01: We discover what different models and agents are good at.
• 3:06
  SPEAKER_01: We leverage those insights as a differentiated deployment partner in the enterprise.
• 3:13
  SPEAKER_01: And the enterprise, we've gone deep in financial services.
• 3:16
  SPEAKER_01: We work with some of the largest asset management firms in the world and build end-to-end AI systems for them.
• 3:23
  SPEAKER_01: And when we build these systems, we discover where these models and agents break.
• 3:29
  SPEAKER_01: We leverage those, uh, our knowledge of those capability gaps to create even more realistic data, evals, oral environments for the frontier AI labs.
• 3:40
  SPEAKER_01: So we so we can solve even bigger problems in enterprise, see where things break, generate even more realistic data, solve even bigger problems in enterprise, and we execute this loop.
• 3:51
  SPEAKER_01: So most data companies don't see deployment, and most deployment companies don't see data.
• 3:57
  SPEAKER_01: So Turing is the one company that works with uh the Frontier AI labs and Fortune 500 enterprises.
• 4:04
  SPEAKER_01: And we are a trusted bridge between research and deployment.
• 4:08
  SPEAKER_01: Um, on one hand, it's like humans making AI smarter, and on the other hand, it's AI making humans smarter.
• 4:15
  SPEAKER_01: Um, and these both uh compound.
• 4:18
  SPEAKER_01: That's what we do.
• 4:20
  SPEAKER_01: And you're right that the journey here um is super interesting.
• 4:24
  SPEAKER_01: So Turing started uh as this uh a platform that used AI to source and vet software engineers from all over the world.
• 4:34
  SPEAKER_01: We have the world's largest uh software engineering talent platform, uh, Silicon Valley Caliber software engineers from all over the planet.
• 4:43
  SPEAKER_01: So in um, and we were growing really fast.
• 4:47
  SPEAKER_01: Uh in um it the world kind of changed when uh OpenAI came to us uh when they were training GPT-3 and they wanted to teach GPT-3 to code.
• 4:59
  SPEAKER_01: Um, so we collaborated with them on teaching the models to code and to do tool use, function calling.
• 5:07
  SPEAKER_01: And then when ChatGPT launched and it took off, it was the birth of uh the AGI race.
• 5:15
  SPEAKER_01: Right.
• 5:16
  SPEAKER_01: And it became clear to us that this is this is going to be a huge shift in the way software engineering is done and how all knowledge work uh in front of a computer will be done.
• 5:30
  SPEAKER_01: So we leaned into it uh and we built a data platform on top of our talent platform and expanded beyond coding to enterprise knowledge work, frontier STEM.
• 5:43
  SPEAKER_01: And uh it's been incredible working with all the Frontier AI labs to advance the models in coding, knowledge work, and STEM.
• 5:53
  SPEAKER_02: And and how did OpenAI reach out to you?
• 5:56
  SPEAKER_02: Like what was your mode or positioning in the data space?
• 6:01
  SPEAKER_01: So we were not a data company when they reached out.
• 6:04
  SPEAKER_01: Like we, I think OpenAI at that time was looking for.
• 6:08
  SPEAKER_01: I mean, I don't know for sure, like, but they were looking for who has the best coders on the planet, and it is Turing.
• 6:17
  SPEAKER_01: We have the world's best platform for software engineers, and now we've expanded that to experts in every industry you can think of financial services, retail, healthcare, life sciences, every um function, software engineering, sales, marketing, finance, every role in the org chart in that function.
• 6:37
  SPEAKER_01: Um, we have, I mean, Turing's platform has the smartest humans in the world in every domain.
• 6:44
  SPEAKER_01: I think OpenAI at that time was just looking for um somebody who can generate really high quality data for coding models really quickly.
• 6:55
  SPEAKER_01: And the fact that we used AI to source talent, vet talent, match talent, manage talent meant we could scale up really fast while keeping quality high.
• 7:03
  SPEAKER_01: That was the key.
• 7:05
  SPEAKER_02: So they used your this engine uh uh to help them, or they they they used uh they said, Jonathan, uh this is what we need on the data side uh for coding.
• 7:18
  SPEAKER_02: Come and help us with whatever you have.
• 7:20
  SPEAKER_01: So I wouldn't be able to get into the specifics, but we but it it it was about generating coding data to improve the models.
• 7:30
  SPEAKER_01: Um for um at that time the models were at the level of writing, um like relatively simple Python code, like relatively the kind of um coding tasks that a human would do when interviewing for a role.
• 7:48
  SPEAKER_01: Now we've built up scaled up our engine for agentic coding, where I mean you're seeing what the coding models today are capable of, building end-to-end systems, like coding for days, yeah, uh sometimes a week.
• 8:02
  SPEAKER_01: Um, and we've uh we've uh evolved it.
• 8:05
  SPEAKER_01: Um but it started with just I think like they needed a partner that could help generate really high-quality coding data that could um improve the models, you're evaluating weaknesses in the models, figuring out failure modes, and generating data to uh address those, uh address those gaps.
• 8:27
  SPEAKER_02: And if you have to explain, let's say, coding data to our audience who many of them might not come from the AI background, how would you explain it?
• 8:35
  SPEAKER_01: Uh so there's I think it's um uh one way to think about data is just look at the LLM training pipeline today.
• 8:46
  SPEAKER_01: So there's pre-training data, uh SFT data, and reinforcement learning data.
• 8:53
  SPEAKER_01: All of this is slightly different.
• 8:55
  SPEAKER_01: Uh in pre-training, you're feeding the model um the entire internet.
• 9:01
  SPEAKER_03: Yeah.
• 9:02
  SPEAKER_01: Um, and you're also giving the model access to books, other sources of knowledge.
• 9:07
  SPEAKER_01: So pre-training is um largely unsupervised.
• 9:12
  SPEAKER_01: Uh, so you're just feeding the model data.
• 9:15
  SPEAKER_01: The model is building, I mean, you're you're basically building a base model that learns how to autocomplete.
• 9:20
  SPEAKER_01: Yeah.
• 9:21
  SPEAKER_01: In post-training, uh think of pre-training as creating like uh it's almost like brain mass.
• 9:29
  SPEAKER_01: There's this raw brain mass, but you haven't quite shaped it yet.
• 9:33
  SPEAKER_01: Um in that pre-training corpus, from a coding perspective, there's also data from GitHub, right?
• 9:39
  SPEAKER_01: Like and other sources of code.
• 9:41
  SPEAKER_01: If you've acquired code assets, you might also feed that into pre-training.
• 9:46
  SPEAKER_01: Um, but the base model is only doing um uh autocompletion of tokens, right?
• 9:53
  SPEAKER_01: It doesn't yet know how to follow instructions.
• 9:55
  SPEAKER_01: In supervised fine-tuning, there's this paper from OpenAI called Instruct GPT, which talks about the recipe for how to do this, where uh you're teaching the model to be an assistant.
• 10:10
  SPEAKER_01: So in supervised fine tuning, uh that's where um GPT-3 becomes Chat GPT, right?
• 10:17
  SPEAKER_01: So you're teaching the model to be helpful, honest, harmless, and you're teaching the model how to have a conversation.
• 10:23
  SPEAKER_01: So you the so humans are teaching the model to um you're having conversations with the model, the model knows um how to uh communicate with with a user.
• 10:35
  SPEAKER_01: Now then there is uh reinforcement learning.
• 10:39
  SPEAKER_01: And in reinforcement learning, you uh create uh oral environments uh where uh you're creating these environments where an agent is getting uh a reward when it does something correctly, and when it doesn't do something correctly, it doesn't get a reward.
• 10:59
  SPEAKER_01: And usually you also have to give the agent intermediate rewards as it's um uh as it's uh trying to get to a to a solution.
• 11:10
  SPEAKER_01: So in the context of coding, there is like a there is a benchmark called Suibench uh for your readers and encourage them to read it.
• 11:18
  SPEAKER_01: Sweetbench is a really good benchmark, which is uh a it's open, it's largely open source GitHub repos where you are um the models are being tested on their ability to merge pull requests correctly with the unit tests passing.
• 11:37
  SPEAKER_01: So one type of coding data that's helpful to people building models is, and Turing does this, where we are automatically crawling um open source GitHub repos at massive scale.
• 11:52
  SPEAKER_01: We are also acquiring code assets from startups and other companies.
• 11:57
  SPEAKER_01: And we are running those repos and figuring out um pull requests where the test cases uh don't pass for today's frontier models and agents.
• 12:12
  SPEAKER_01: Right.
• 12:12
  SPEAKER_01: So we have to uh and in some cases where the test cases don't exist, we would create test cases.
• 12:19
  SPEAKER_01: Um, but you'd want uh data sets like this that are realistic, like reflecting real-world software engineering, long horizon where the agent has to work for hours or days to complete a task, and calibrated at the right level of complexity.
• 12:38
  SPEAKER_01: Uh, if you have tasks that are too easy for the model, there is no learning signal.
• 12:44
  SPEAKER_01: If it is too difficult, there is no learning signal.
• 12:47
  SPEAKER_01: So there is a sweet spot of complexity that you'd want the RL environments to be at, usually like 20 to 40 percent, something in that zip code, like a Goldilocks zone.
• 12:58
  SPEAKER_01: And you'd have to spin these up at massive scale.
• 13:01
  SPEAKER_01: And uh and Turing does that.
• 13:04
  SPEAKER_01: So that's an example of like one type of coding data set where you're evaluating the agent's ability to merge uh pull requests with test cases that pass without false positives, false negatives.
• 13:20
  SPEAKER_01: There are other constructs like this.
• 13:22
  SPEAKER_01: Um there is a benchmark called terminal bench, uh, where you're evaluating um terminal bench is the kind of benchmark you'd need for like a command line coding agent that has access to a terminal, like ThinkClot Code, Codecs, Gemini CLI, Grok Build agents like that.
• 13:43
  SPEAKER_01: There it's an interesting problem where you give the agent access to a terminal and access to the internet, and it has you're evaluating its ability to do complex tasks in software engineering, machine learning, cybersecurity, scientific coding, etc.
• 14:00
  SPEAKER_01: In theory, if you have access to the terminal on your computer and the internet, you could do anything on your computer.
• 14:07
  SPEAKER_01: And there again, it's you're evaluating the uh agent's ability to this to do this type of knowledge work.
• 14:17
  SPEAKER_01: Where with Terminal Bench, you don't just evaluate the output, you evaluate the state of the system.
• 14:23
  SPEAKER_01: Maybe you might be uh you might be you might have to install something, you might have to write code, you might have to create some documents or PowerPoints or spreadsheets.
• 14:36
  SPEAKER_01: Um yeah, so that's another example.
• 14:39
  SPEAKER_01: Tunnel bench is like another benchmark.
• 14:41
  SPEAKER_01: Um, there are many others in coding.
• 14:43
  SPEAKER_01: There's MLE bench, which evaluates an agent's ability to write um machine learning code that satisfies some objective.
• 14:53
  SPEAKER_01: The way to think about reinforcement learning is um if you've um I I think the the example that everybody uh remembers well is um Alpha Go and Alpha Zero by DeepMind.
• 15:09
  SPEAKER_03: Okay.
• 15:09
  SPEAKER_01: Um, so Alpha Go was DeepMind's go-playing program that was originally trained by learning from expert humans.
• 15:18
  SPEAKER_01: Um Alpha Zero was their program where it was not trained from humans, it just played against itself.
• 15:26
  SPEAKER_01: And obviously, in a game, you get a reward based on whether you won the game or you lost the game.
• 15:31
  SPEAKER_01: And depending on whether you won the game or lost the game, you can um you can figure out like which moves led to you winning the game or losing the game.
• 15:41
  SPEAKER_01: You can have intermediate rewards.
• 15:44
  SPEAKER_01: Uh you can imagine a similar construct for coding or for knowledge work, where the reward can be whether you did the task well or not.
• 15:54
  SPEAKER_01: And but you have to be smart about intermediate process rewards.
• 15:58
  SPEAKER_01: How do you reward partial completion to get to the task?
• 16:01
  SPEAKER_01: And the task could be writing a program that can automatically vet a finance analyst.
• 16:09
  SPEAKER_01: Maybe that's a task.
• 16:11
  SPEAKER_01: Or it could be create a board deck.
• 16:15
  SPEAKER_01: Uh, or it could be evaluate this investment opportunity and write an investment memo.
• 16:22
  SPEAKER_01: It becomes a little more complex when the task is not binary, like with code or with math, it's relatively more binary, easy to verify.
• 16:31
  SPEAKER_01: But how do you verify the quality of a board deck?
• 16:34
  SPEAKER_03: Yeah.
• 16:34
  SPEAKER_01: It's uh you have to be, you have to have like a good rubric-based evaluator.
• 16:39
  SPEAKER_01: Um, so yeah, lots of interesting problems.
• 16:43
  SPEAKER_01: But reinforcement learning has been one of those fascinating areas that uh, like deep learning, has been like a very rich vein.
• 16:51
  SPEAKER_01: Uh the thing I find fascinating, Siddharth, is um Ilya Sutskiever, the chief scientist at OpenAI, like I saw in a podcast that apparently his research program for OpenAI in 2014 was um train a generative model on the world's knowledge and do RL on it.
• 17:16
  SPEAKER_01: Right.
• 17:16
  SPEAKER_01: And this was in 2024, and more than a decade later, it's still the formula that like everybody's largely following.
• 17:25
  SPEAKER_01: Um I remember this Sam Altman quote of um do more of what works.
• 17:31
  SPEAKER_01: And I feel like deep learning works and reinforcement learning works.
• 17:36
  SPEAKER_01: And I think all the labs are scaling this up, and we are constrained by algorithmic research, compute, and data, and companies like Turing advanced the data pillar, NVIDIA advances the compute pillar, and of course the labs advanced the algorithmic research pillar.
• 17:54
  SPEAKER_02: And uh from 2014, let's say OpenAI started roughly during that time frame.
• 17:59
  SPEAKER_02: So 2014 to 21, 22, uh the world never knew about how fast AI accelerating or uh but suddenly 22, 23, until now, every year, the acceleration is 10x more since then.
• 18:16
  SPEAKER_02: Why why is uh so much development happening at this space?
• 18:21
  SPEAKER_01: Um, so I think like I I feel like um up until 2012, we were in the pre-deep learning era.
• 18:31
  SPEAKER_01: We were in the machine learning era, right?
• 18:34
  SPEAKER_01: Where I mean I did machine learning at Stanford, like the where you would, there were all these different algorithms you would use.
• 18:41
  SPEAKER_01: Neural networks was one such algorithm.
• 18:45
  SPEAKER_01: We hadn't really discovered that deep neural networks trained with lots of data and lots of compute do magical things.
• 18:53
  SPEAKER_01: Yeah, we hadn't yet discovered that.
• 18:55
  SPEAKER_01: 2012 was a milestone when ImageNet came out, the ImageNet breakthrough came out, where these convolutional neural nets um exceeded human-level performance in certain tasks.
• 19:08
  SPEAKER_01: And after 2012, the world just doubled down on deep neural networks, lots of compute, lots of data.
• 19:15
  SPEAKER_01: And we started using GPUs uh for training.
• 19:20
  SPEAKER_01: Um and I think there was a small group of people that knew what was happening, and the there was that group that bet on deep neural networks.
• 19:30
  SPEAKER_01: I think the OpenAI team was one of them.
• 19:34
  SPEAKER_01: And I think it's one of those things where uh exponential growth is like exponential improvements are like hard to fathom.
• 19:44
  SPEAKER_01: And between GPT 2 to GPT-3 to GPT-4, it's like as you kept scaling up, these models just keep getting keep kept improving.
• 19:55
  SPEAKER_01: Like it is it was reported that the sorry the current series of frontier models are largely in the trillion parameter realm.
• 20:05
  SPEAKER_03: Yeah.
• 20:05
  SPEAKER_01: Now we are starting to see 10 trillion parameter models.
• 20:09
  SPEAKER_01: Um it's and I think when these models come out, I think we'll see it'll feel even more magical.
• 20:18
  SPEAKER_01: Um some people say like intelligence is log of compute, like you're just um and the scaling laws are continuing to hold, meaning bigger model with more data, with more compute means that the models smoothly keep getting better.
• 20:36
  SPEAKER_01: Um, it's like it's it's like the when the pre-training loss keeps coming down, it's like the model's performance on also all these a variety of different intelligence benchmarks seems to keep going up.
• 20:49
  SPEAKER_01: Um so I I think it's just people discovered the power of I think OpenAI like led the way, and uh everybody like has discovered that the scaling laws are holding, and I think it's the world's energy being more focused um on a recipe that works.
• 21:10
  SPEAKER_01: I think so we're seeing faster progress.
• 21:12
  SPEAKER_02: God, but but since now, uh, you know, and anthropics started in 21, 22, roughly, but in three years it's very astonishing how much they have been able to accomplish.
• 21:24
  SPEAKER_02: And today, across enterprises uh or even uh mid-market startups, they have become the default uh medium for coding.
• 21:34
  SPEAKER_02: How has such acceleration happened in in only three years?
• 21:39
  SPEAKER_01: I think with coding, um coding is one of those areas where because it's verifiable, I think that there is a good path to using reinforcement learning to improve coding models um quickly.
• 21:58
  SPEAKER_01: And I think like The labs, many of the labs see that coding is really important.
• 22:08
  SPEAKER_01: It's important for a few reasons.
• 22:11
  SPEAKER_01: One is many believe that the fastest way to get to AGI is to automate AI research itself.
• 22:19
  SPEAKER_01: And to automate AI research, you need a really strong coding model.
• 22:24
  SPEAKER_01: Since AI researchers are implementing research papers, trying out new ideas, you have to write a lot of code.
• 22:32
  SPEAKER_01: So if you can accelerate your ability to write high-quality code, ideally figure out what experiments to run, you'll of course accelerate.
• 22:39
  SPEAKER_01: So there was a lot of energy that went into improving coding.
• 22:44
  SPEAKER_01: Second, when the models improve in coding, they have out-of-domain performance.
• 22:50
  SPEAKER_01: They also improve in other tasks that have nothing to do with coding for reasons we don't fully understand.
• 22:56
  SPEAKER_01: My hunch is that when the models improve in coding, they also improve in tool use.
• 23:05
  SPEAKER_01: And that helps these generalized computer use agents like Codecs, Cloud Code, Gemini's CLI, etc.
• 23:13
  SPEAKER_01: So I think a lot of people prioritized coding.
• 23:16
  SPEAKER_01: And as a result, the coding models improved.
• 23:19
  SPEAKER_01: Like that's uh, I mean, uh we are probably the largest provider of coding data to all the labs.
• 23:27
  SPEAKER_01: And uh like before we expanded into other types of knowledge work, like we did, we really focused on coding.
• 23:33
  SPEAKER_01: Um, so part of it is the focus and the importance to the lab, so just improved more quickly.
• 23:38
  SPEAKER_03: Yeah.
• 23:39
  SPEAKER_01: And um, and I think like we some of these um benchmarks that we had, like I feel like there were a lot of benchmarks that were somewhat more academic, where you're testing intelligence rather than the ability to do real work.
• 23:59
  SPEAKER_01: Um, and it was needed at the time, like in the in the last few years, we needed to know how smart is AI.
• 24:05
  SPEAKER_01: So we were we were happy when AI wins like a gold medal in a math Olympiad or a physics Olympiad or wins a coding competition, stuff like that.
• 24:14
  SPEAKER_01: But I think there were also some really good benchmarks for coding.
• 24:17
  SPEAKER_01: I do think Sweetch was a good start.
• 24:20
  SPEAKER_01: Uh now Sweetbench is saturated, now you need more complex coding benchmarks.
• 24:24
  SPEAKER_01: Turing is creating many complex coding benchmarks.
• 24:27
  SPEAKER_01: Uh, and those also helped the labs hill climb on something that was more economically valuable.
• 24:33
  SPEAKER_02: So, one of the interesting things is that you mentioned everything that you give to a model is almost like a like a uh which you shared offline, is almost like a coding problem.
• 24:45
  SPEAKER_02: Can you give that example that you gave me offline?
• 24:49
  SPEAKER_01: Yes, I was saying like one reason coding is so important in AGI is a lot of problems are reducible to coding.
• 24:58
  SPEAKER_03: Yeah.
• 24:58
  SPEAKER_01: For example, uh, if you're asking an AI a question like um uh what's the difference between AI investing in Silicon Valley versus AI investing in uh uh India?
• 25:12
  SPEAKER_01: Um a good model would come up with a plan.
• 25:16
  SPEAKER_01: The plan might involve querying crunch base, querying pitchbook, downloading some data, figuring out, okay, let me break down AI investing by sector or by uh category.
• 25:28
  SPEAKER_01: Let me write some Python code to analyze the data, to visualize the data, and present it to you.
• 25:34
  SPEAKER_01: So it wrote code as part of its um execution.
• 25:39
  SPEAKER_01: So I we've we've uh we all know uh that you know a decade ago, Mark Andreessen spoke about um software is eating the world.
• 25:49
  SPEAKER_01: What he really meant was software is eating all types of knowledge work, and now AI is eating software.
• 25:57
  SPEAKER_01: So AI is eating all types of knowledge work.
• 26:00
  SPEAKER_01: It most tasks people do involve data analysis, involve some type of writing code.
• 26:08
  SPEAKER_01: For example, like I'm coming off of a leadership offsite with uh with my team.
• 26:14
  SPEAKER_01: And one of the things we do in the off-site is figure out exactly where to bet on.
• 26:20
  SPEAKER_01: And it's actually a data analysis problem.
• 26:23
  SPEAKER_01: Like you're looking at which segments are growing fastest, which segments are uh not growing as fast.
• 26:30
  SPEAKER_01: You might want to analyze data from your own internal systems.
• 26:34
  SPEAKER_01: And that's an example of um again where coding is kind of embedded in the loop.
• 26:39
  SPEAKER_01: If I ask my head of FPNA, what are some areas in Turing we should invest in, what are some areas we should reduce our investment in, that's again a coding problem, right?
• 26:48
  SPEAKER_01: Um a lot of problems in life are math problems framed the right way.
• 26:55
  SPEAKER_01: And code is a way to express and solve math problems.
• 27:00
  SPEAKER_01: And I feel like if you can solve um coding, tool use, reasoning, and multimodality, you have the keys to superintelligence.
• 27:10
  SPEAKER_01: Uh, those are the building blocks.
• 27:11
  SPEAKER_01: If you can solve those four, you can automate anything that a human does in front of a computer.
• 27:16
  SPEAKER_01: And it'll give humans superpowers.
• 27:18
  SPEAKER_01: For example, um, as an investor, uh, you might be interested in analyzing what are some trends in investing among top investors.
• 27:29
  SPEAKER_01: That's a coding problem.
• 27:31
  SPEAKER_01: You might be curious how is my portfolio at NEON different from Sequoia's?
• 27:36
  SPEAKER_01: That's a coding problem.
• 27:37
  SPEAKER_01: Coding and data analysis.
• 27:39
  SPEAKER_01: Now every human has the benefit of a giant software engineering team in the cloud that'll envision whatever they want.
• 27:48
  SPEAKER_01: It's actually even more than that.
• 27:50
  SPEAKER_01: Like when you think of you having access to a giant team of software engineers in the cloud, the onus is on you to also ask the right questions.
• 27:58
  SPEAKER_03: Yeah.
• 27:59
  SPEAKER_01: But now you, I mean, given we've also baked in um humanity's best knowledge from strategy consultants, analysts, data scientists, software engineers, product managers, CEOs, CTOs.
• 28:17
  SPEAKER_01: It can, I mean, you could literally ask your AI agent to come up with a strategy for how to build a portfolio that maximizes returns.
• 28:27
  SPEAKER_01: Yeah.
• 28:28
  SPEAKER_01: Of course, it may be wrong today, and it'll probably be wrong for a while, but that's where like you could exercise your human judgment to correct it.
• 28:37
  SPEAKER_01: Um basically, coding and coding is the key to superintelligence.
• 28:41
  SPEAKER_01: You solve coding, you've solved almost everything.
• 28:44
  SPEAKER_02: So then with this thesis, uh, the edge in most fields would the human edge in most fields or knowledge worker working fields, for example, investing is one of the fields, right?
• 28:57
  SPEAKER_02: Where judgment counts.
• 28:59
  SPEAKER_02: Uh right.
• 29:00
  SPEAKER_02: And in public markets, the data is more visible.
• 29:02
  SPEAKER_02: In private markets, uh you know, you don't discover or data is not visible mostly, uh right, especially at the earlier stages.
• 29:12
  SPEAKER_02: So according to that thesis, the the human judgment becomes less and less uh uh important because if AI superintelligence can make those judgments where to invest, then everybody has access to that information.
• 29:28
  SPEAKER_01: So I actually disagree.
• 29:29
  SPEAKER_01: I think human judgment can is gonna become even more important.
• 29:33
  SPEAKER_01: I think what's happening is the floor is going up.
• 29:36
  SPEAKER_01: The for example, if a um if a software engineer is working with a coding agent that has now written code for a week, um, you do, I mean, reviewing the code that get that is that that got written needs work, you have to know where to look, what to check.
• 29:59
  SPEAKER_01: Of course, AI will also help with that.
• 30:02
  SPEAKER_01: I um and in the in your point about investing too, I think I don't think these models are still at a point where um human input is not needed.
• 30:14
  SPEAKER_01: I think of these as productivity accelerators.
• 30:19
  SPEAKER_01: Like today there are things that you're using tools for to accelerate your ability to do to do work.
• 30:28
  SPEAKER_01: Um I think um these frontier models are the ultimate knowledge work accelerator.
• 30:37
  SPEAKER_01: So humans can solve problems at higher and higher levels of obstruction.
• 30:41
  SPEAKER_01: Um uh I do uh I think like, I mean, we used to think about humans as um having um in the Andy Grove high output management sense, maybe your span should be seven to eight people that you're managing, right?
• 30:55
  SPEAKER_01: Now I think in the limit over the next decade, every human might run seven to eight companies as these as the loop from idea to a I mean, as you could go from prompt to company in hopefully like a single step over the next decade.
• 31:10
  SPEAKER_03: Yeah.
• 31:11
  SPEAKER_01: Um I think it'll it's we still uh I think humans will ultimately still be key to knowing what questions to ask, knowing what problems to solve.
• 31:24
  SPEAKER_01: Um, it's just that now um going from idea to execution has been democratized.
• 31:32
  SPEAKER_01: Like in the past, if you were a plumber or a doctor or a lawyer, you might have an idea for a cool startup, but you're bottlenecked on I have to raise some venture capital, I have to hire some engineers, I have to hire a salesperson, a marketing person.
• 31:48
  SPEAKER_01: There was high friction.
• 31:50
  SPEAKER_01: Um I now it's like much lower friction.
• 31:54
  SPEAKER_01: I'm very bullish on what this means for humanity.
• 31:57
  SPEAKER_01: I think we'll see more entrepreneurs, lots and lots of companies getting started.
• 32:02
  SPEAKER_01: I think larger companies today will be, companies in the future will be a lot smaller.
• 32:06
  SPEAKER_01: Sure.
• 32:07
  SPEAKER_01: Um I I think humans, I think humans are just gonna be, I mean, we are more productive today after personal computers, after the World Wide Web, after Google.
• 32:21
  SPEAKER_01: And now I think this is the next step.
• 32:23
  SPEAKER_01: AGI is gonna just make humans 100x more productive.
• 32:28
  SPEAKER_02: So one limitation that I see right now is let's say if I have a high agency teammate in the in my investment team, yeah, on a daily basis, they'll think on what problems to solve.
• 32:43
  SPEAKER_02: Uh, I I think models are not there yet.
• 32:46
  SPEAKER_01: Let's say not yet, but they will be.
• 32:50
  SPEAKER_02: And you believe that that that is near term future.
• 32:53
  SPEAKER_01: Absolutely.
• 32:54
  SPEAKER_01: Absolutely.
• 32:55
  SPEAKER_01: It is even now, in theory, you could get that behavior by looping the model as an agent.
• 33:05
  SPEAKER_01: Um, that there are ways to get that from the models today with the right scaffolding, with the right harness.
• 33:11
  SPEAKER_01: Absolutely.
• 33:12
  SPEAKER_01: For example, I I uh have a project at Turing that I run.
• 33:17
  SPEAKER_01: I have a few agents running around to help me run Tiering.
• 33:21
  SPEAKER_01: The way I improve that code base is to also have an agent suggest to me how should I improve the code base and to do it.
• 33:30
  SPEAKER_01: So it's in a self-improvement loop by itself.
• 33:33
  SPEAKER_01: You can imagine applying a self-improvement loop to yourself, to your company, and letting it run.
• 33:40
  SPEAKER_01: Yeah, right.
• 33:41
  SPEAKER_01: Obviously, it may not be at a point where um you trust it enough to keep running it constantly.
• 33:48
  SPEAKER_01: But even that is huge.
• 33:49
  SPEAKER_01: I mean, even your human analyst might come to you for approval.
• 33:53
  SPEAKER_01: I have this new idea, I have this new business plan.
• 33:55
  SPEAKER_01: I think we should do this.
• 33:56
  SPEAKER_01: This will require this much investment, but it will give us this much in revenues.
• 34:02
  SPEAKER_01: You will you will do that, right?
• 34:04
  SPEAKER_01: So I would argue that it's already there with the right harness, and it's gonna get even better, like with especially with these bigger and bigger models.
• 34:15
  SPEAKER_01: Um I think of I think of what we have right now is intelligence as an API.
• 34:21
  SPEAKER_01: Um, and it's a very unique type of intelligence where it's already superhuman in many things, it's subhuman in certain things, and as long as you build around its jagged intelligence, like you're good.
• 34:35
  SPEAKER_02: Understood.
• 34:37
  SPEAKER_02: So today uh uh software engineering is one of the the works that is getting replaced.
• 34:43
  SPEAKER_02: Globally, companies, if we observe, uh right, people are replacing uh uh or reducing their team size in software engineering.
• 34:53
  SPEAKER_01: So I I think it's it's there's a few things happening.
• 34:57
  SPEAKER_01: I think so um I think these models have democratized coding to the point where every software engineer is now insanely more productive.
• 35:15
  SPEAKER_01: Now, if you're if it's a company that is that has product market fit that wants to scale, suddenly your team is now 10x more productive and you can grow faster than ever.
• 35:26
  SPEAKER_01: If you're a company that doesn't have product market fit, or you're operating, you're treating RD almost like a cost center, then yes, you need fewer engineers, and you can you can probably do that.
• 35:39
  SPEAKER_01: But it's also the case that engineers are now able to create a lot more than ever before.
• 35:46
  SPEAKER_01: Um it'll create some challenges with distribution.
• 35:50
  SPEAKER_01: Like if you can create a lot of apps, like how do you get users, how do you make money?
• 35:53
  SPEAKER_01: Those are still important problems.
• 35:55
  SPEAKER_01: Um but I feel like, like at Turing, for example, like we are scaling our software engineering team.
• 36:02
  SPEAKER_01: Like we want really strong engineers because we see this massive opportunity.
• 36:07
  SPEAKER_01: And in the past, like we would have been um like now for this, we are significantly more capital efficient in terms of how we can scale.
• 36:19
  SPEAKER_01: Um so I think the impact of it is dependent on uh the company and where it is in their life cycle.
• 36:27
  SPEAKER_01: You you are correct that if you're a junior and if you're an engineer that is not using AI, uh, not even an engineer, if you're in any type of knowledge work, if you're not using AI highly, like I'm I'm talking, you should actually be in the mindset of agent first, human second.
• 36:44
  SPEAKER_01: Turing operates this way.
• 36:46
  SPEAKER_02: Um what do you mean by that?
• 36:48
  SPEAKER_02: Can you share examples?
• 36:49
  SPEAKER_01: So um every knowledge worker at Turing, either in coding or any other domain, um, I want them to use AI to do the work first.
• 37:04
  SPEAKER_01: So agents should create, humans should steer, humans should create prompts, skill files, give the agent access to the right tools, access to the right knowledge sources.
• 37:15
  SPEAKER_01: Humans should create the scaffold, and AI should create.
• 37:19
  SPEAKER_01: For example, if you're creating a board deck, yeah, don't the human should not, the investor, let's say that uh my head of investor relations, he shouldn't create the board deck.
• 37:31
  SPEAKER_01: He should give the model access to the right sources of knowledge, access to the right tools, write a skills file to apply consistent formatting, um, give give the right instructions on how to create the board deck.
• 37:48
  SPEAKER_01: Uh, use codecs or cowork or Gemini or Grog to build it.
• 37:52
  SPEAKER_01: Um, and then when there are mistakes in it, prompt the model um to improve it.
• 37:59
  SPEAKER_03: Sure.
• 37:59
  SPEAKER_01: And then keep improving the skills file so that next time it's even faster.
• 38:04
  SPEAKER_01: Basically, um the agents create v1 of the work.
• 38:09
  SPEAKER_01: Humans are verifying and iterating, humans never create v1.
• 38:14
  SPEAKER_01: Um there's an interesting post from OpenAI on um how they used how they used codecs to uh to the for one particular software project, which was agent first, human second.
• 38:29
  SPEAKER_01: Uh humans were not allowed to write code directly.
• 38:33
  SPEAKER_01: They can only like prompt agent.
• 38:35
  SPEAKER_01: Yes.
• 38:37
  SPEAKER_01: Um so I think like knowledge work is that's what I meant when I said um agent first, human second in how you do knowledge work.
• 38:46
  SPEAKER_01: Um and uh I think it's a huge productivity unlock across every function and every workflow in an organization.
• 38:56
  SPEAKER_01: Think of every function software engineering, sales, marketing, finance, people, recruiting.
• 39:00
  SPEAKER_01: Think of every role in the org chart in that function.
• 39:04
  SPEAKER_02: Yeah.
• 39:05
  SPEAKER_01: And every role is a composite of workflows, right?
• 39:07
  SPEAKER_01: And every workflow has an artifact that you create.
• 39:10
  SPEAKER_01: There's a clear definition of done.
• 39:13
  SPEAKER_01: At Turing, we are in this unique position where we generate data to help the models improve in every workflow, in every role, in every function, in every industry.
• 39:23
  SPEAKER_01: We create data and oral environments to improve the models, and we also go build with it in the enterprise.
• 39:30
  SPEAKER_01: So we know exactly where these things break.
• 39:33
  SPEAKER_01: So that's why like I have the somewhat unique vantage point into seeing where this is going.
• 39:43
  SPEAKER_01: Um and it's super exciting.
• 39:46
  SPEAKER_02: And do you think the difference between today services and product company is blurring out?
• 39:53
  SPEAKER_01: Yes.
• 39:54
  SPEAKER_02: Can you share your perspective on that?
• 39:57
  SPEAKER_01: So I think what's happening is there's two trends that are happening right now.
• 40:05
  SPEAKER_01: One is the models are becoming agentic, where the models can accomplish long horizon workflows for every type of knowledge work, for every type of coding project.
• 40:17
  SPEAKER_01: The models are able to do it end-to-end, right?
• 40:19
  SPEAKER_01: And if you can control your computer, they can do everything you can in front of your computer, right?
• 40:24
  SPEAKER_01: With the humans verifying.
• 40:26
  SPEAKER_01: So, on one hand, the models are becoming agentic.
• 40:30
  SPEAKER_01: And in theory, if the models have access to the right sources of data, you kind of don't need software in the middle.
• 40:39
  SPEAKER_01: The models can do it end-to-end.
• 40:41
  SPEAKER_01: For example, imagine you want to create here is a fictitious conversation that I may have with an agent.
• 40:52
  SPEAKER_01: It's not quite fictitious because at Turing I'll soon be able to have this conversation.
• 40:56
  SPEAKER_01: But my conversation to the agent might be hey, I want to create a hiring rec for a researcher in reinforcement learning for code.
• 41:08
  SPEAKER_01: The agent will tell me, Oh, that's great.
• 41:11
  SPEAKER_01: Uh, here I picked up, I created V1 of a JD.
• 41:14
  SPEAKER_01: Does this sound good to you?
• 41:16
  SPEAKER_01: And I'd say, uh, you know, uh, this sounds good.
• 41:19
  SPEAKER_01: Um uh no prior experience needed.
• 41:21
  SPEAKER_03: Yeah.
• 41:22
  SPEAKER_01: So let's say, okay, done.
• 41:23
  SPEAKER_01: I made that change.
• 41:24
  SPEAKER_01: And then it says, the agent says, Hey, by the way, I looked up um comp benchmarking data and I came up with these compounds to the for the role.
• 41:33
  SPEAKER_01: Uh, I also checked with the finance agent, and this is in budget.
• 41:38
  SPEAKER_01: Uh, does it sound good to you?
• 41:40
  SPEAKER_01: I said, Yeah, that sounds good.
• 41:41
  SPEAKER_01: Let's go for it.
• 41:42
  SPEAKER_01: And add a geoconstraint for the Bay Area and Bangalore.
• 41:46
  SPEAKER_01: Um and then the agent asks me, hey, uh, can I update the website with this new role?
• 41:52
  SPEAKER_01: Yeah, go for it.
• 41:54
  SPEAKER_01: And by the way, in the future, it may not even ask me.
• 41:56
  SPEAKER_01: It remembers this once.
• 41:56
  SPEAKER_01: Next time it'll do it itself, right?
• 41:58
  SPEAKER_01: And it could say, um, I've also created, would you like me to create a LinkedIn ad for this role?
• 42:05
  SPEAKER_01: I say, yeah, uh, do it with a with, I don't know,$1,000 budget.
• 42:10
  SPEAKER_01: Um and then it could, it could then say, uh, I see that your calendar is uh, you have time on your calendar, you have like 10 hours a week in your calendar for recruiting.
• 42:20
  SPEAKER_01: Can I use those slots?
• 42:22
  SPEAKER_01: I say yes.
• 42:24
  SPEAKER_01: So now in, and then it uh when there when somebody emails me for this job, it could set up the meeting, it could even do the first assessment, it could assess talent on certain things.
• 42:36
  SPEAKER_01: Now, in this workflow that I mentioned, I did not talk to anybody in finance, anybody in HR, anybody in recruiting, anybody in the website, anybody in our engineering team.
• 42:47
  SPEAKER_01: I did not talk to our performance marketing team to run ads.
• 42:51
  SPEAKER_01: It was just the agent.
• 42:52
  SPEAKER_01: And no, there was no SaaS software involved.
• 42:55
  SPEAKER_01: There was no greenhouse, there was no Asana, there was no uh there was no separate vetting software.
• 43:04
  SPEAKER_01: The agent kind of whipped up whatever it needed on the fly.
• 43:08
  SPEAKER_01: There was literally no human involved, right?
• 43:12
  SPEAKER_01: It was so there's one there's one trend where the models are becoming agentic.
• 43:16
  SPEAKER_01: So legacy SaaS has is is going to have some challenges like as the models become more agentic.
• 43:25
  SPEAKER_01: I'll share uh think of this as a dual pincer.
• 43:28
  SPEAKER_01: It's a pincer movement.
• 43:29
  SPEAKER_01: That's the top-down pincer.
• 43:30
  SPEAKER_01: Yeah, the bottoms-up pincer is every human can now write software.
• 43:36
  SPEAKER_01: Yeah, my EA wrote software because I have a crazy schedule, like I work a lot, I travel a lot, like I'm always on, so it creates a lot of work.
• 43:47
  SPEAKER_01: So my EA wrote uh software to manage her EA team, and that software also automatically keeps track of all demands on my time.
• 44:00
  SPEAKER_01: Prioritizes it based on what's most important for me.
• 44:02
  SPEAKER_01: So nothing gets dropped.
• 44:05
  SPEAKER_01: Now, in the in in she could have used Trello or Jira or Asana, but she didn't.
• 44:14
  SPEAKER_01: She built her own task management system.
• 44:16
  SPEAKER_01: And she built something that was more custom for her than any of that software.
• 44:19
  SPEAKER_01: It's the era of infinite software, custom software.
• 44:24
  SPEAKER_01: And there is an alternate universe.
• 44:28
  SPEAKER_01: I've heard pitches for these types of software to help EAs serve their execs better, but probably wasn't venture backable, or for whatever reason it didn't, it didn't take hold.
• 44:39
  SPEAKER_01: But she just built it and you and used it.
• 44:41
  SPEAKER_01: So we're not buying it.
• 44:44
  SPEAKER_01: So you can see that superintelligence is eating SaaS and it's eating services, right?
• 44:52
  SPEAKER_01: It's as the models become agentic in this dual pincer motion.
• 44:55
  SPEAKER_01: Like one, every human now creates software that they wouldn't pay for, but it's just a fun weekend project now.
• 45:02
  SPEAKER_03: Yeah.
• 45:02
  SPEAKER_01: And the models are becoming agentic.
• 45:06
  SPEAKER_01: I think these two pose some unique challenges where I think the key is going to be who has access to the data layer.
• 45:13
  SPEAKER_01: I think the data is going to be key.
• 45:16
  SPEAKER_01: And in the enterprise, I think that'll be particularly important.
• 45:22
  SPEAKER_02: So according to this theory, then all workflow SaaS would, let's say, next five years would not be required.
• 45:31
  SPEAKER_02: Then only the system of records will be required.
• 45:36
  SPEAKER_01: I think it is a possibility.
• 46:24
  SPEAKER_02: So help me explain.
• 46:25
  SPEAKER_02: A company which is starting new and scales quickly, how does it build its own system of recording?
• 46:33
  SPEAKER_01: I'm not saying they may they'll build so they could uh I mean it's all of it is just software, right?
• 46:43
  SPEAKER_01: And if in the past, like writing enterprise grade secure software was hard and it would have taken a while, now it's become easier.
• 46:55
  SPEAKER_01: So they would just use the models to write what they need to write.
• 47:00
  SPEAKER_01: And the benefit for them is custom software, software that is uh fine-tuned for their specific workflows.
• 47:08
  SPEAKER_01: Like we are used to, because the cost of software RD was quite high, especially for ML AI software, it made sense to go outside and use something.
• 47:20
  SPEAKER_01: Now you can build it.
• 47:22
  SPEAKER_01: I mean, uh, there was a whole industry around um uh you know uh implement uh you know, like the I don't want to name specific software, but you're like implementing uh your own customization to some legacy SaaS software, right?
• 47:39
  SPEAKER_02: Now you don't have to the entire GSI industry is based on that.
• 47:42
  SPEAKER_01: Yeah.
• 47:45
  SPEAKER_01: There's an interesting uh thesis that's gonna be validated.
• 47:50
  SPEAKER_01: Like it's kind of unclear how it's gonna uh how it's gonna play out, which is on one hand, we could be in a world of, I'm gonna call it the no fine-tuning world, where you have a frontier model, like a trillion parameter or 10 trillion parameter from OpenAI, Anthropic, DeepMind, Meta, XAI, Amazon, Microsoft, somebody, a frontier model.
• 48:13
  SPEAKER_01: And all you need to do is manage context and memory, give it access to the right tools, and let the model cook.
• 48:23
  SPEAKER_01: So that's one way, the no-fine-tuning camp.
• 48:26
  SPEAKER_01: Uh, all you have to do is be smart about how you're managing context and memory.
• 48:30
  SPEAKER_01: You don't have to touch the weights.
• 48:32
  SPEAKER_01: That's one world.
• 48:33
  SPEAKER_01: Let's call it the no-fine-tuning camp.
• 48:36
  SPEAKER_01: There is another camp, which is hey, you don't need these giant models.
• 48:41
  SPEAKER_01: You could, with a half a billion to 10 billion parameter model, fine-tune it on your proprietary data, distill your proprietary human knowledge into the LLMs, automate your proprietary tool calls.
• 48:54
  SPEAKER_01: It'll be faster, cheaper, uh, more accurate.
• 48:59
  SPEAKER_01: Um that's one school of thought.
• 49:01
  SPEAKER_01: Let's call it the fine-tuning camp.
• 49:03
  SPEAKER_01: I think it'll be interesting to see how it plays out.
• 49:06
  SPEAKER_01: What I'm seeing more and more is there's a lot more people going into the no fine-tuning camp than a couple of years ago for very high value enterprise use cases.
• 49:19
  SPEAKER_01: Um, can you give examples?
• 49:21
  SPEAKER_01: The um, I mean, I have a system at Turing that um helps me run Turing.
• 49:30
  SPEAKER_01: And this is a system that I built on top of one of the frontier models.
• 49:35
  SPEAKER_01: And I didn't have to fine-tune it.
• 49:37
  SPEAKER_01: Like all I had to do was intelligent management of uh context, memory, access to the right tools, access to the right sources of knowledge.
• 49:49
  SPEAKER_01: Um and you can imagine, like in um in asset management, in financial services, there are workflows in um in uh in the in the day-to-day of a fund controller, uh, of um uh an asset manager, where the the the frontier models are actually doing very, very well, like without any fine-tuning.
• 50:18
  SPEAKER_01: Um I I I would start more specifically with what's the workflow that you're trying to automate, and um what's a verifier for the quality of the artifact that that workflow is creating.
• 50:35
  SPEAKER_01: Um and step one should be to like just do it with a frontier model and see how well it does.
• 50:43
  SPEAKER_01: And most times you'll see that you don't have to touch the weights.
• 50:47
  SPEAKER_01: But there are certain workflows, let's like customer support, um, like an invoice-to-pay system, some of these more basic stuff.
• 50:56
  SPEAKER_01: Where I think you know, a smaller open source model that's fine-tuned on proprietary data might be faster, cheaper, more accurate.
• 51:05
  SPEAKER_01: But for really high-value use cases like um discovering alpha in investment opportunities, you want the most powerful model.
• 51:15
  SPEAKER_01: And you'll probably see that that powerful model with good context and memory management will do very, very well.
• 51:25
  SPEAKER_01: And I I personally don't fall into any specific camp.
• 51:30
  SPEAKER_01: I think it's much more specific to the workflow that you're trying to automate.
• 51:36
  SPEAKER_01: Some require giant models that you don't have to bother fine-tuning, um, but have good context management.
• 51:44
  SPEAKER_01: Some require smaller models that you can um some you can make do with smaller models.
• 51:50
  SPEAKER_01: Yeah.
• 51:50
  SPEAKER_02: Uh and for our audience, if if you can share, let's say, in very simple terms, uh training and inference has been the key.
• 52:00
  SPEAKER_02: And and like say the last few years, there was huge investment in training part, but now inference is also coming up as well.
• 52:09
  SPEAKER_02: So probably if you can share what has been happening and where is it going in both these areas.
• 52:15
  SPEAKER_01: Yeah, yeah.
• 52:16
  SPEAKER_01: So um there is like um there is scale up in compute for a few different things.
• 52:27
  SPEAKER_01: Um there's pre-training compute.
• 52:30
  SPEAKER_01: So I would break training into two parts, uh, pre-training and post-training.
• 52:36
  SPEAKER_01: In the past, it was a lot of the compute went into pre-training.
• 52:39
  SPEAKER_01: Yeah, now a lot of compute goes into reinforcement learning in post-training as well.
• 52:45
  SPEAKER_01: Um, especially after O1 came out and Deep Seek came out.
• 52:49
  SPEAKER_01: So there's a lot of compute that's needed for still for training, right?
• 52:54
  SPEAKER_01: But what we're seeing is as these models have gotten so much more capable, uh, people need so much more.
• 53:02
  SPEAKER_01: Like I'm compute constrained, like for how much I want to create agents, run agents.
• 53:08
  SPEAKER_01: So it's um uh so so inference compute is skyrocketing because demand is skyrocketing.
• 53:16
  SPEAKER_02: And what are the use cases for that?
• 53:18
  SPEAKER_01: All types of knowledge work, all types of coding.
• 53:23
  SPEAKER_01: The um, for example, like um what's something in your workflow that you do most frequently in your workflow as uh I I think search search for company data.
• 53:35
  SPEAKER_02: Let's say, for example, if I'm evaluating a cybersecurity company.
• 53:39
  SPEAKER_01: Yeah.
• 53:40
  SPEAKER_02: Uh I I would try to find uh uh details about that company mostly.
• 53:48
  SPEAKER_02: Uh details about the competitors and where industry is heading.
• 53:52
  SPEAKER_02: Yeah.
• 53:52
  SPEAKER_02: And I would then try to find customer usage uh data to to verify uh how our customers are using.
• 53:59
  SPEAKER_02: Is are is the customer usage growing every month or not?
• 54:05
  SPEAKER_02: Uh because because for us, I'll give you a simple example.
• 54:08
  SPEAKER_02: We come at pre-seed at zero revenue.
• 54:11
  SPEAKER_02: So it's very important that after investment, is it two, three years, a company can reach 10 million ARR.
• 54:17
  SPEAKER_02: Yeah.
• 54:17
  SPEAKER_02: So all our effort is to validate and validate or disvalidate in that direction.
• 54:22
  SPEAKER_01: Yeah.
• 54:22
  SPEAKER_01: Can I I'm just brainstorming on the fly with you.
• 54:25
  SPEAKER_01: But I would can I give you a use case for you to be compute constrained, inference constraint?
• 54:30
  SPEAKER_01: It would be to run a deep research agent in a loop that is looking for cybersecurity companies all the time.
• 54:40
  SPEAKER_01: While we're talking, that agent is running, right?
• 54:42
  SPEAKER_01: Maybe on a Mac Mini somewhere safe.
• 54:44
  SPEAKER_01: It's running, it's pulling uh interesting companies that it thinks are good.
• 54:50
  SPEAKER_01: But then you've given it a budget.
• 54:53
  SPEAKER_01: So it goes and spends some money on GLG or like some expert network, it recruits some humans, the agent spins up Zoom and does a conversation with that human, does interviews with them, gives you a summary of the interview results.
• 55:10
  SPEAKER_01: And in some cases, it may play with the software itself.
• 55:15
  SPEAKER_01: Cybersecurity, I don't know if it is possible to do it well, but it might be.
• 55:19
  SPEAKER_01: It might test the software by itself, or it might recruit some other humans, give them access to the software, collect their feedback, maybe it recruits some CISOs.
• 55:30
  SPEAKER_01: Um, as long as you give it enough money to go hire people, research the web, go hire people, talk to people, run experiments, and come back.
• 55:40
  SPEAKER_01: This agent may come back to you uh every week and give you a summary.
• 55:45
  SPEAKER_01: Hey, Siddharth, like here are the top three companies in cybersecurity that I think you should pay attention to.
• 55:50
  SPEAKER_01: These are the most hyped, these are the most high, these are the most underrated, but where maybe there is more investment alpha.
• 55:58
  SPEAKER_01: How much would you pay for something like that?
• 56:00
  SPEAKER_01: It's literally helping you discover your next investment opportunity.
• 56:03
  SPEAKER_01: It's like your associate or your.
• 56:05
  SPEAKER_02: Yeah, absolutely.
• 56:06
  SPEAKER_02: I'll I'll pay the cost of an associate.
• 56:08
  SPEAKER_01: Yes.
• 56:08
  SPEAKER_01: And that's that's that's why you'd be inference constrained, right?
• 56:12
  SPEAKER_01: Because to do that, it's making a ton of LLM calls, it's like running in a loop, right?
• 56:17
  SPEAKER_01: Like that, I feel like if we kept talking, we can come up with even more.
• 56:21
  SPEAKER_01: Where and that's why like inference is skyrocketing.
• 56:24
  SPEAKER_01: It's because there's so much model capability overhang.
• 56:27
  SPEAKER_01: The models are capable of so much more, but humans are still not extracting the fullest value from these models, especially in enterprise.
• 56:36
  SPEAKER_01: Uh, in enterprise, I think first mile and last mile is still a problem.
• 56:40
  SPEAKER_01: Like the average enterprise is still not getting the best value.
• 56:45
  SPEAKER_01: And at Turing, that's one of the reasons why we focus on enterprise deployments.
• 56:50
  SPEAKER_01: Even if the models did not get any smarter, there's so much more value to be unlocked.
• 56:58
  SPEAKER_01: Uh, but of course the models are gonna get a lot smarter.
• 57:00
  SPEAKER_02: So it's gonna be uh I think all the money in the world is getting concentrated on the models.
• 57:06
  SPEAKER_01: Yes.
• 57:07
  SPEAKER_01: I used to, I used to like um a few years back, I used to think about why why do we need um so much compute?
• 57:17
  SPEAKER_01: Yeah.
• 57:18
  SPEAKER_01: Now it's clear to me.
• 57:19
  SPEAKER_01: Like, I think humanity is gonna be reasoning bound and compute bound to solve some of our biggest problems.
• 57:27
  SPEAKER_01: And especially if coding, enterprise, and STEM are so important.
• 57:32
  SPEAKER_01: If you accelerate coding, you'll accelerate AI research.
• 57:35
  SPEAKER_01: If you accelerate knowledge work, like we're gonna create so much more value in the economy, like we'll move the GDP of the world.
• 57:42
  SPEAKER_01: And if you accelerate scientific discovery to discover new materials, new um new sources of energy, um, more efficient uh robotics, there's just so much more um innovation that it'll unlock.
• 58:00
  SPEAKER_01: Uh so I think like it's it's gonna be an exciting decade or two ahead, more.
• 58:06
  SPEAKER_02: The the biggest fear right now among founders and and the community is that are models going to eat up all the work?
• 58:16
  SPEAKER_02: Then we discuss about cybersecurity coming.
• 58:18
  SPEAKER_02: If models eat up the domain of cybersecurity, then what is left for entrepreneurs to create?
• 58:26
  SPEAKER_01: I think we'll keep building on top of the models.
• 58:29
  SPEAKER_01: Like uh like uh so Sam Altman spoke at a Turing event uh a short while ago.
• 58:36
  SPEAKER_01: And one thing he would say is uh there are two schools of thought, there are two camps.
• 58:44
  SPEAKER_01: One camp that's betting on a world where the models are getting better.
• 58:48
  SPEAKER_01: There's another camp that's betting on a world where the models are not getting better.
• 58:51
  SPEAKER_01: You create all these other structures to fill in the gaps.
• 58:55
  SPEAKER_01: I think like I personally would bet on a world where the models keep getting better.
• 59:01
  SPEAKER_01: There are certain things you may do today thinking that the models are not going to get good at this.
• 59:06
  SPEAKER_01: I would not do that.
• 59:07
  SPEAKER_01: I would just think of intelligence as continuing to improve.
• 59:10
  SPEAKER_01: I think like humans, the best founders will discover big markets.
• 59:18
  SPEAKER_01: Like my advice to founders is uh pick a big market.
• 59:23
  SPEAKER_01: Um, markets that don't exist don't care how smart you are or how hard you work, pick a big market.
• 59:29
  SPEAKER_01: Pick a market with weak competition, like ideally not competition that's uh other big tech companies or other AI labs.
• 59:39
  SPEAKER_01: Like, don't don't pick a market with weak competition or analog competition.
• 59:45
  SPEAKER_01: For example, Uber competed with taxi companies, analog competition, not a fair fight.
• 59:50
  SPEAKER_01: Um, companies like um Airbnb competed with like hotels again, uh analog competition.
• 59:59
  SPEAKER_01: So pick analog competition or relatively weak competition.
• 1:00:02
  SPEAKER_01: Don't don't compete with the frontier labs or big tech companies, that's just not a good strategy.
• 1:00:08
  SPEAKER_01: Uh and third is um the um I think the basics of hire really smart people who are highly driven.
• 1:00:27
  SPEAKER_01: Um I think that always holds.
• 1:00:31
  SPEAKER_01: Um I would think about I I would think about coming up with ideas that are betting on a world where intelligence will be abundant and the models keep getting smarter.
• 1:00:46
  SPEAKER_01: Um I think the hard problems to solve will still be how do you uh get distribution, uh, how do you generate revenue, not short-term revenue, but long-term revenue.
• 1:00:59
  SPEAKER_01: I think those problems will still will still remain.
• 1:01:03
  SPEAKER_01: Um and I think I would pick on problems that require inherently human um a human touch, like maybe problems more in the physical world rather than in the purely digital world.
• 1:01:17
  SPEAKER_01: Um maybe problems where human trust, human connection um are important.
• 1:01:28
  SPEAKER_02: Um can can you give examples of some of the areas where let's say if you have to invest personally, which was which are the kind of companies that you would invest?
• 1:01:40
  SPEAKER_01: So uh I'll give you a set that I think is uh necessary but not sufficient.
• 1:01:45
  SPEAKER_01: It's a subset.
• 1:01:46
  SPEAKER_01: I haven't thought deeply about expanding it.
• 1:01:49
  SPEAKER_01: Um I would bet on companies that are an input to superintelligence.
• 1:01:54
  SPEAKER_01: Like uh Turing's an input to superintelligence, like date the models need data.
• 1:01:58
  SPEAKER_01: They're always gonna need data for a long, a very, very long horizon.
• 1:02:03
  SPEAKER_01: Uh I would bet on companies that are either an input, like I mean, what are the key inputs?
• 1:02:09
  SPEAKER_01: Compute, energy, data.
• 1:02:11
  SPEAKER_01: You need high-quality research talent, but the labs would hire that themselves.
• 1:02:15
  SPEAKER_01: Those are inputs, they are fuel to superintelligence.
• 1:02:18
  SPEAKER_01: I would bet on companies there.
• 1:02:21
  SPEAKER_01: Um, I would also bet on companies that are betting on a future where intelligence is widely distributed.
• 1:02:28
  SPEAKER_01: Turing's enterprise business is a bet on deployment that intelligence is already there.
• 1:02:33
  SPEAKER_01: Let's just build the right systems around it so enterprises can unlock value.
• 1:02:38
  SPEAKER_01: Um, for example, if Neon were a Turing customer, we'd build that agent for you so that you have like a um an associate GPT or an associate cloud that can do these.
• 1:02:50
  SPEAKER_01: So I'd bet on companies that are inputs to superintelligence or an outcome of superintelligence.
• 1:02:56
  SPEAKER_01: In a world where superintelligence exists, you'd need it.
• 1:02:58
  SPEAKER_01: Um I might also like bet on robotics clearly will be huge.
• 1:03:10
  SPEAKER_01: And it's I think it's a matter of time.
• 1:03:14
  SPEAKER_02: Uh would you invest in domains like cybersecurity, or would you think models would take up that domain?
• 1:03:20
  SPEAKER_01: I would invest in domains in cybersecurity.
• 1:03:22
  SPEAKER_01: Turing is investing in cybersecurity to make sure that um the um yeah, to make sure that agentic coding is safe, both for offense and defense.
• 1:03:36
  SPEAKER_02: But like with anthropic launching models like Mythos, there's a huge debate that would they try to take up whatever is going on on vulnerability management, and then slowly start getting into endpoint management and everything else.
• 1:03:50
  SPEAKER_01: Yeah, yeah.
• 1:03:51
  SPEAKER_01: I think it's it uh uh I would not build a I I I when I said I would invest in cybersecurity more in, I mean, we are investing in cybersecurity at Turin to um help these models keep our systems safe, help these frontier models detect um vulnerabilities in a safe way, um, and help the coding models write code that is more secure.
• 1:04:22
  SPEAKER_01: I meant in that sense.
• 1:04:24
  SPEAKER_01: Uh, but you're you're right in this the same reason I gave against uh legacy software, yeah.
• 1:04:31
  SPEAKER_01: Uh legacy SaaS software also applies to legacy cybersecurity products too.
• 1:04:37
  SPEAKER_02: Or I I would say uh entrepreneurs not stop thinking in terms of products, but start thinking in terms of problems.
• 1:04:45
  SPEAKER_02: Yeah.
• 1:04:47
  SPEAKER_01: Yeah.
• 1:04:47
  SPEAKER_02: What that's what you are hinting at, right?
• 1:04:49
  SPEAKER_02: Uh what whatever uh problem horizon that feeds into model or becomes an outcome of model.
• 1:04:56
  SPEAKER_02: But to to contest your, let's say the work that you are doing for an enterprise, yeah, the model are coming or will come in that kind of a work.
• 1:05:06
  SPEAKER_02: So, how how do you make that defensible and growing?
• 1:05:10
  SPEAKER_01: Yeah, yeah.
• 1:05:10
  SPEAKER_01: We are uh we are building on top of the models.
• 1:05:14
  SPEAKER_01: Oftentimes with intelligent context management, memory management, we for the enterprises oftentimes they want a system that is relatively model agnostic.
• 1:05:27
  SPEAKER_01: Uh, you'd want to make sure that you have the right auditability, governance, traceability, verifiability.
• 1:05:33
  SPEAKER_01: You want a human in the loop.
• 1:05:35
  SPEAKER_01: You need a workflow that's constructed on top of the models.
• 1:05:38
  SPEAKER_01: We would we would not build our own model.
• 1:05:40
  SPEAKER_01: We don't want to compete with our customers, but we are building around the models so that enterprises can unlock the fullest value.
• 1:05:48
  SPEAKER_01: And in the real world enterprise, there is still a lot of last mile messiness where it is there is a gap between what the model is capable of.
• 1:06:00
  SPEAKER_01: And what an enterprise is actually able to extract uh value out of.
• 1:06:05
  SPEAKER_01: And it's it's non-trivial.
• 1:06:08
  SPEAKER_01: Like the and even at Turing, it it requires work to make sure that you're operating in the right agent first, human, second way.
• 1:06:17
  SPEAKER_01: Um yeah.
• 1:06:19
  SPEAKER_02: So you you so your bet is the the model will not come into the last bit of stitching together everything.
• 1:06:30
  SPEAKER_01: Yes, and part of that bet is you're seeing like uh I mean Turing is hiring an insane amount of forward-deployed engineers.
• 1:06:40
  SPEAKER_01: You can see OpenAI, Anthropic, and all the Frontier Labs also doing that.
• 1:06:44
  SPEAKER_01: Yeah.
• 1:06:45
  SPEAKER_01: Because it's hard.
• 1:06:46
  SPEAKER_01: It's hard, and that last mile is also not fully digital.
• 1:06:51
  SPEAKER_01: In some cases, you have to you have to um be in the right rooms, talk to the right people, observe the right workflows to make sure that you're doing something that helps helps people.
• 1:07:06
  SPEAKER_01: You have to manage change inside those organizations, you have to train people in new workflows.
• 1:07:12
  SPEAKER_01: Um it is complex.
• 1:07:15
  SPEAKER_01: And you wanna, you're you're also not vibe coding in the enterprise, right?
• 1:07:19
  SPEAKER_01: Like you have to make sure that systems are secure, maintainable.
• 1:07:25
  SPEAKER_01: Um, they are built with the same coding standards as the rest of the code base in that enterprise.
• 1:07:33
  SPEAKER_01: Uh, so that is that is quite a bit of last mile engineering to do.
• 1:07:39
  SPEAKER_01: So, what we are building on the enterprise side is a palantir for AGI.
• 1:07:43
  SPEAKER_01: We've built our platform that makes it very easy for us to build and deploy agents, uh, which is highly model and agent agnostic.
• 1:07:52
  SPEAKER_01: It's actually agnostic at every layer of the stack.
• 1:07:55
  SPEAKER_01: It's agnostic at the layer of the cloud, the model provider, agent tick orchestration layer, memory.
• 1:08:02
  SPEAKER_01: It's very, very modular, which is which is helpful for the enterprises so that they can use the right um models and agents for the right task.
• 1:08:13
  SPEAKER_02: Well, no, thank you so much, Jonathan.
• 1:08:15
  SPEAKER_02: You know, it's been an amazing conversation.
• 1:08:18
  SPEAKER_02: I learned a lot in this process of uh discussing brainstorming with you.
• 1:08:23
  SPEAKER_02: Uh my last uh question is uh if you see the world different in 10 years, uh right, what would your prediction be for 2035 or 2036?
• 1:08:39
  SPEAKER_01: The name companies will look very different, jobs will look very different, education will look very different.
• 1:08:46
  SPEAKER_01: I think it'll look almost unrecognizable from what exists today.
• 1:08:53
  SPEAKER_01: Um and I think like we are it's gonna be a golden age of productivity.
• 1:09:03
  SPEAKER_01: I feel like most of humanity's most challenging problems are intelligence constrained in terms of us being able to throw more intelligence at it to solve the problem.
• 1:09:15
  SPEAKER_01: But imagine a world where intelligence is so abundantly available so that we find cures to diseases faster.
• 1:09:23
  SPEAKER_01: We've mastered, maybe, maybe we've mastered some crazy interstellar travel.
• 1:09:30
  SPEAKER_01: We've uh we've discovered new materials.
• 1:09:33
  SPEAKER_01: Um I think it's I think it's gonna be, I think we'll it'll it'll be an exciting future.
• 1:09:40
  SPEAKER_01: Like when we look back 100 years ago, 200 years ago, sometimes you may wonder, like, how did people live without computers uh and all of these tools and gadgets we take for granted?
• 1:09:52
  SPEAKER_01: I think people will look at this this this will look archaic.
• 1:09:56
  SPEAKER_01: Um so I think the so those three things will look very different.
• 1:10:00
  SPEAKER_01: Enterprises will look different, jobs will look different, and the education system will look totally different.
• 1:10:07
  SPEAKER_01: And I hope we've uh we've also maybe we've also extended human lifespan, right?
• 1:10:14
  SPEAKER_01: Like with uh with better knowledge of human biology and better drugs and better elimination of um sort of common reasons people uh people die.
• 1:10:29
  SPEAKER_02: Thank you so much, Jonathan.
• 1:10:30
  SPEAKER_02: It's been an amazing conversation with you.
• 1:10:32
  SPEAKER_03: Great.
• 1:10:32
  SPEAKER_03: Thank you for having me.