Latest / In the Loop AI / AI Loops: How the World's Best Engineers Use AI

Transcript

AI Loops: How the World's Best Engineers Use AI · In the Loop AI

0:00
Ejaaz: 99% of people are using AI models the same way that they use Google.
0:03
Ejaaz: But recently, a new way of prompting your AI has emerged that doesn't just replace
0:08
Ejaaz: the way that you work, it promotes you to the CEO of your very own AI company.
0:12
Ejaaz: It's called Loops and it's part of a growing development in agent autonomy where
0:16
Ejaaz: AI agents basically spin up and autonomously complete tasks or goals that you
0:21
Ejaaz: set for it, often working throughout the night.
0:23
Ejaaz: In 2019, the longest that an AI agent could work autonomously for was for two seconds.
0:28
Ejaaz: Fast forward to today, and they can work autonomously for 12 hours,
0:32
Ejaaz: and that's doubling every couple of months.
0:34
Ejaaz: Andre Carpathy calls this phenomenon the autonomy slider, where you can go take
0:38
Ejaaz: a dial that slides from humans that approve everything to humans that periodically check in.
0:43
Ejaaz: And it's part of this growing trend of agents consuming and taking up more of
0:47
Ejaaz: human capital and labor. And the question that remains going forwards is, what will humans do?
0:52
Ejaaz: And will they be entirely replaced
0:54
Ejaaz: by AI? Or will they be the ultimate orchestrator of their destiny?
0:58
Josh: Yeah, I think the goal for this episode is really just to inform people on what's
1:02
Josh: possible current day with these agents, with these LLMs, with writing these
1:06
Josh: loops, as well as where you can possibly find yourself within that stack,
1:10
Josh: because it gets pretty complicated.
1:11
Josh: When we're getting into loops, not everyone needs to use loops,
1:14
Josh: but everyone should be using LLMs probably slightly different than how you're using them today.
1:19
Josh: So maybe we could start with a little history lesson in terms of the four levels
1:23
Josh: in which we have been engaging with llm starting with the first level which is just prompting,
1:29
Josh: generally like most people are probably still doing this started in 2022 2023
1:34
Josh: around the release of chat gpt
1:35
Josh: the way that you would engage with these llms is you would just submit a question
1:39
Josh: or you submit a prompt and you get some language back now if you are still doing
1:43
Josh: this that's okay because i find a.
1:50
Josh: Three years ago, four years ago. It has since advanced pretty,
1:54
Josh: pretty meaningfully since then.
1:55
Josh: The second step of this is agents. And we're going to spend some time on agents.
1:59
Josh: Everyone's kind of heard of an agent. Maybe not everyone knows what an agent is.
2:02
Josh: An agent is something that could think for a little bit longer.
2:05
Josh: It could run a bit longer than just a standard prompt.
2:08
Josh: It can go off and do things. It could call tools for you.
2:11
Josh: It's a much more capable version of the text box. Then, like we talked about
2:15
Josh: all the time on the show recently in the last few weeks, there's the harness
2:18
Josh: feature in which you put an LLM into a container and that gives it a memory feature.
2:23
Josh: That gives it complete tool use. That's something like an open claw that we've
2:26
Josh: talked about a lot that some people do use and that's level three.
2:29
Josh: And now level four, which is the new thing that has come this week,
2:32
Josh: that's really been highlighted by some of the top leaders at these AI labs is loops.
2:36
Josh: And a loop is essentially a version of an agent that has an orchestration layer
2:41
Josh: and kind of builds upon itself.
2:43
Josh: So it allows you to kind of continue to scope yourself out. If you can imagine
2:47
Josh: you're kind of you're dealing directly with an employee at level one and then
2:50
Josh: you're kind of directing that person to go off and do their own in level two,
2:54
Josh: At level three with the harness, you're kind of directing a series of people to help you.
2:59
Josh: And then level four, you're just the top level CEO who's directing your C-suite
3:02
Josh: to go and manage all the employees below you. So there's an entire stack to this.
3:06
Josh: It's very cool. It just how do you use your AI currently? Where would you say
3:10
Josh: that you fit in this stack?
3:12
Ejaaz: Yeah, so looking at this diagram that we have on the screen here,
3:16
Ejaaz: I'm somewhere between number two and number three. I'm somewhere between using
3:19
Ejaaz: agents and trying to figure out the whole harness thing.
3:23
Ejaaz: Now, what am I doing when it comes to like spinning up agents?
3:27
Ejaaz: If you look at either my Claude or my ChatGPT desktop apps right now,
3:31
Ejaaz: I've renamed a bunch of my conversations to a particular focus or subject and then agent after it.
3:38
Ejaaz: And so I can go to it and this agent basically has all the context of what I
3:42
Ejaaz: wanted to do, whether it's like research a particular topic,
3:45
Ejaaz: create some kind of an outline for something, research a particular investment angle.
3:49
Ejaaz: It already knows and has the embedded context for what it needs to do.
3:52
Ejaaz: And there's usually like one to maybe three tasks that it needs to autonomously execute on its own.
3:58
Ejaaz: And so it runs in kind of like a sequence. But if any of that sequence kind
4:02
Ejaaz: of breaks, let's say it kind of tries to retrieve data from some particular
4:06
Ejaaz: website and it is unable to do so, it breaks.
4:09
Ejaaz: And it comes to me and it says, hey, Ejaz, is there some other thing that you
4:13
Ejaaz: want to look at or retrieve from, blah, blah, blah?
4:15
Ejaaz: It's not fully autonomous. Now, number three, the Harness side of things is
4:19
Ejaaz: what I'm trying to kind of like mold my understanding around.
4:22
Ejaaz: What I've noticed is when you type in a prompt and you get a response,
4:26
Ejaaz: you can kind of tell that it's AI-y. Like usually when we kind of create artifacts,
4:30
Ejaaz: it comes in a particular font or it speaks in a particular type of language.
4:34
Ejaaz: The Harness helps kind of like take your prompt and kind of mold it into something
4:38
Ejaaz: that is more human-like, but also more nuanced with what you are trying to do.
4:43
Ejaaz: Like it effectively gets
4:45
Ejaaz: closer towards that ultimate goal. Like we were talking before recording this
4:49
Ejaaz: episode about human taste and how AI doesn't really get human taste.
4:53
Ejaaz: The harness helps you get towards that ultimate kind of taste profile for the
4:57
Ejaaz: particular output that you're trying to generate.
4:59
Ejaaz: I haven't tried working with loops just yet, but my understanding of this,
5:04
Ejaaz: and correct me if I'm wrong, is you have an AI. You can prompt it and you can get some kind of output.
5:10
Ejaaz: A loop specifically is an AI agent that doesn't break, if it comes across an
5:15
Ejaaz: obstacle that it doesn't understand, its instinct isn't to come to the human
5:19
Ejaaz: and say, hey, like, I can't figure this out, guide me.
5:23
Ejaaz: It completely reiterates the prompt over and over again until it gets past that
5:28
Ejaaz: obstacle, working towards like one objective. So a few examples I've seen for
5:32
Ejaaz: this is if you are coding, right?
5:34
Ejaaz: And let's say there's multiple workflows of a code base that you want to work
5:38
Ejaaz: on, and it comes across a hiccup where it can't retrieve data from one of those
5:42
Ejaaz: particular flows, it is able to kind of like circumnavigate around it,
5:46
Ejaaz: maybe spin up its own separate flow and try to figure out the problem.
5:49
Ejaaz: And often this results in an agent working for multiple hours at a time,
5:53
Ejaaz: often overnight. I think Carpathy spoke about his auto research agent working
5:57
Ejaaz: overnight whilst he slept.
5:59
Ejaaz: And we're seeing different variations of this start to arise.
6:02
Ejaaz: Where are you, Josh, in the stack?
6:04
Josh: Yeah, loops are like the closed source a system where you kind of define an
6:09
Josh: outcome and it will continue to work towards that outcome without any external inputs.
6:13
Josh: It's very cool. It's very automated. I don't think it's for everyone.
6:16
Josh: It's certainly not for me because I haven't really had a use case for loops per se.
6:21
Josh: I would say I'm sitting at each one of those first three phases given whatever
6:26
Josh: tasks I'm trying to do. And I think it's important to understand that a lot
6:28
Josh: of people might not even need to go past number one unless you're actually doing productive work.
6:33
Josh: A lot of the agents, a lot of the harnesses are for kind of automating more.
6:38
Josh: More systems from your life if you're just trying to use this as google if you're
6:41
Josh: just trying to use this as a writing assistant or someone to chat with the prompting
6:46
Josh: is really strong and i find a lot of times
6:48
Josh: this is my outlook or this is my outlet for like google search results so instead
6:53
Josh: of searching for google i'll get a little more in-depth results i'll ask my llm
6:56
Josh: for agents i use them quite a bit when i'm doing a little bit more productive
7:00
Josh: work for example we track the analytics on limitless and we want a place in
7:04
Josh: which we can have all those analytics dumped to a dashboard,
7:06
Josh: that is an agent that I run.
7:08
Josh: So it goes into my browser. It detects all of the views that we've had from
7:12
Josh: the week for YouTube, from Spotify, from RSS feed, where you should all be subscribed
7:16
Josh: to and rate us five stars.
7:18
Josh: And it compiles it into a singular spreadsheet in which we could then publish
7:21
Josh: online and we could share with prospective sponsors and things like that.
7:25
Josh: And then for harnesses I've used, because I mean, that's mostly OpenClaw.
7:28
Josh: I've used OpenClaw. I really enjoyed the process. I find myself using it a bit less and less.
7:34
Josh: And I think in the loops feature, at least it's probably most productive right
7:39
Josh: now for people who are writing code, who are writing verifiable solutions.
7:43
Josh: One of the difficult things that as I was looking into loops and figuring out
7:46
Josh: how I can structure them into my life, one of the problems that I run into is
7:49
Josh: I'm not really sure I have a verifiable,
7:51
Josh: set of outputs that I wanted to optimize for, for a lot of the work that I'm
7:55
Josh: doing, because a lot of it is subjective. A lot of it is kind of creative work.
7:59
Josh: It requires a human in the loop for a lot more of it.
8:01
Josh: So I would say I am number one, two, and three on the list. Haven't quite made my way to four.
8:07
Josh: But yeah, for the people who are, those are the people like Boris Churny from
8:11
Josh: Anthropic. And we know Andre and Peter Steinberg from OpenAI.
8:16
Josh: They are all on four. They are using it to,
8:19
Josh: create these like unbelievable, agentic systems and continue to remove themselves out of the loop.
8:23
Ejaaz: You know what I've realized? With loops in particular and just AI agents in
8:28
Ejaaz: general, they're trying to improve our understanding or rather their understanding
8:35
Ejaaz: of the English language.
8:36
Ejaaz: So one of my favorite Carpathy quotes back in the day was English is the new
8:41
Ejaaz: programming language. I think you said this like two, two and a half years ago.
8:44
Ejaaz: And I've just realized that like us creating AI agents is basically like,
8:49
Ejaaz: it's the same model. It hasn't necessarily got smarter.
8:52
Ejaaz: It's just like using that model to kind of like keep ramming its head and its
8:56
Ejaaz: brain against a particular problem until it understands what the human actually means.
9:00
Ejaaz: And so like in this new world, like I know you just used the example of like,
9:04
Ejaaz: you know, loops can be used for coding specifically,
9:08
Ejaaz: coding that Boris Churny and Carpathy is doing is English.
9:12
Ejaaz: Like they're speaking to the LLM, they are writing in English to the LLM.
9:16
Ejaaz: And yeah, maybe they're copy and pasting some versions of code,
9:18
Ejaaz: but that code is primarily generated by an AI.
9:21
Ejaaz: I think like something crazy, like 80% plus of code generated at Anthropic,
9:25
Ejaaz: both for research and for just general consumer adoption is generated by Claude itself.
9:31
Ejaaz: And so that's one thing. The other thing is the model just not getting smarter
9:36
Ejaaz: is a really interesting thing. Like typically in my head, I would think,
9:39
Ejaaz: okay, you need a better model to be able to unlock some of these new features
9:43
Ejaaz: like AI agents, autonomous loops, et cetera.
9:45
Ejaaz: But really you could just take the same model, wrap a harness around it and
9:49
Ejaaz: try to get it to understand what particular goal it's getting at and just run
9:53
Ejaaz: that iteration over and over and over again until you get a better output.
9:58
Ejaaz: And I guess this is the same concept as inference or reinforcement learning
10:03
Ejaaz: where like we've found this trend of post-training of these AI models,
10:07
Ejaaz: these AI models just getting smarter, not because they've got bigger GPUs or more expensive GPUs.
10:12
Ejaaz: It's because you've just taken the same model and you've just run it through
10:15
Ejaaz: a different reasoning framework over and over again until it can do a thing.
10:18
Ejaaz: And this is the practical embellishment of it. I personally haven't found like
10:23
Ejaaz: an obvious use case for loops either.
10:25
Ejaaz: So either you and I are boxing ourselves into a particular realm and maybe someone
10:29
Ejaaz: listening to this is using this for like their software engineering thing or
10:32
Ejaaz: their marketing thing. But yeah, I guess that's where I sit right now.
10:35
Josh: Well, I think it's probably a skill issue on both our parts.
10:37
Josh: Like there is certainly a use case for us in which we can use a loop in which
10:40
Josh: we can define this outcome, send an agent off to go do it, and it will iterate
10:44
Josh: on itself until it comes to a conclusion.
10:46
Josh: I think it's just so novel and so new. It's difficult to kind of understand
10:50
Josh: why. And we have this really great chart on screen that you're showing now,
10:53
Josh: which is the why now section of this.
10:55
Josh: And it's because the duration of a task that these agents can run is so much
10:59
Josh: longer than it used to be.
11:01
Josh: I mean, in 2019 we have here, it was two seconds. This was well before ChatGPT.
11:06
Josh: But even early last year, in 2025, the duration that an agent could run on one single task was
11:13
Josh: less than an hour in length so there's only so many tokens it could generate
11:17
Josh: there's only so much reasoning it can do and there's only so much iteration
11:19
Josh: you could get over that hour time period let alone the amount of costs that
11:23
Josh: these tokens are going to be,
11:25
Josh: costing you if you're using like the api or anything like that now fast forward
11:28
Josh: to today i mean the best models in the world they're getting days worth of runtime
11:33
Josh: so they can really think deeply and continue to iterate on themselves over and over i see examples of um.
11:38
Josh: Backslash goal on x all the time of people who have a problem whether it be
11:43
Josh: an optimization problem where they have a bug that they need to fix and they'll
11:47
Josh: put this backslash goal on it for
11:49
Josh: however long it needs to and it'll think for three four even five days i've
11:53
Josh: seen in order to optimize for the specific parameter and this is possible because
11:57
Josh: these models now can think for days long,
12:00
Josh: you have to assume months is coming what does it look like Like when an agent can think for months.
12:04
Josh: I mean, it's a really interesting paradigm shift that I'm not sure where people
12:10
Josh: are going to find value in the open-ended way that it exists today,
12:15
Josh: right? It's like, okay, here's this agent.
12:17
Josh: You can tell to do whatever you want. You can create a loop.
12:19
Josh: You can create an infrastructure system for it to operate in.
12:23
Josh: It's pretty much open-ended and it's on you. And I think the answer to that
12:25
Josh: is that not even the AI companies really understand the best use cases for it quite yet.
12:30
Josh: I would imagine it's still this really difficult thing of how do you unlock
12:33
Josh: value from essentially an open-ended agent that can go and run for an infinite
12:37
Josh: amount of time? I don't know.
12:39
Ejaaz: I also question like what a human's purpose would be at that point.
12:45
Ejaaz: Like if you automate enough of the thinking and the curiosity behind like solving
12:51
Ejaaz: particular problems, What do humans end up doing at that point,
12:54
Ejaaz: especially if they don't do the work themselves?
12:56
Ejaaz: They don't understand it, right? You need an AI to kind of like understand what
12:59
Ejaaz: on earth is going on in the first place.
13:01
Ejaaz: And eventually like an AI will then start setting goals, like more ambitious
13:05
Ejaaz: goals than a human can in terms of like what to like kind of solve or go after.
13:09
Ejaaz: There were some very low-level examples that I saw in response to Pete Steyer's tweet about loops.
13:16
Ejaaz: And there's some kind of concrete examples that I want to run through very quickly here.
13:19
Ejaaz: So one of them is using it for code, right? So a classic loop could basically
13:25
Ejaaz: look like, okay, can you please pull live errors for my particular app?
13:30
Ejaaz: Can you inspect and figure out where the bug might particularly be?
13:34
Ejaaz: Can you create then a fix for this particular bug in my code?
13:39
Ejaaz: And then can you deploy it? Then can you check the health of that deployment
13:41
Ejaaz: and make sure that nothing else is broken?
13:43
Ejaaz: And then record what failed and feed that into a database so that in the future,
13:47
Ejaaz: we can detect errors like this or prevent it when we code and build some of
13:52
Ejaaz: These future app features.
13:53
Ejaaz: Now, that is kind of like a very small and specific enough use case that can
13:58
Ejaaz: be generalized across basically any app or software engineering project that
14:02
Ejaaz: you might be working on if you're listening to this.
14:05
Ejaaz: And I wonder how many hours worth of engineering time that this replaces.
14:11
Ejaaz: Because I know that there are entire teams having worked at companies.
14:14
Ejaaz: Been a product manager in the past, entire teams of software engineers that
14:18
Ejaaz: spend their entire days working on something like that. So that's one thing.
14:22
Ejaaz: And then for content, which is very applicable for product managers,
14:25
Ejaaz: or even like the work that you and I do, Josh, an agent could read a PRD.
14:29
Ejaaz: So which is a product requirement doc, which is usually kind of like created
14:33
Ejaaz: for a strategic goal that you want to kind of like build at your company,
14:36
Ejaaz: like a product or a feature, it then writes whatever that next asset could be.
14:40
Ejaaz: So it could be like a design profile or a mockup of what that feature might
14:43
Ejaaz: look like, score it against like some kind of criteria that the company has
14:47
Ejaaz: across like, you know, it must follow our vision, A, B, and C.
14:51
Ejaaz: It must also look a particular way. This is our design profile,
14:53
Ejaaz: our brand kind of profile and our aesthetic.
14:56
Ejaaz: And then it kind of like updates its progress depending on like what other teams
15:00
Ejaaz: have shipped. So maybe it's dependent on a particular feature.
15:03
Ejaaz: And so it updates itself autonomously like that. Now, this all sounds very vague
15:07
Ejaaz: intentionally because it's meant to apply to your particular business or your particular project.
15:12
Ejaaz: But make no mistake, this is what
15:14
Ejaaz: a lot of humans are paid upwards of six figures to do on a daily basis.
15:18
Ejaaz: It's that nuance. And we're starting to see basically AI models and AI agents
15:23
Ejaaz: enter into that human taste profile. So when I think about where we end up eventually,
15:29
Ejaaz: There's a common argument that's made that it's like, oh, humans will always have the taste.
15:33
Ejaaz: They'll always be able to kind of direct where the AI should go because we are
15:36
Ejaaz: this all being kind of like smart kind of entity.
15:39
Ejaaz: But I see increasingly AI stepping into that boundary and becoming the tastemaker
15:45
Ejaaz: for all of the work that we end up doing.
15:47
Josh: I still believe that to be true, that humans in the loop are critically important
15:51
Josh: to applying human taste. I saw this great chart. I have no idea where it is.
15:55
Josh: Somewhere in the depths of X. But basically, it was showing that in the App
15:59
Josh: Store, the iOS App Store, where everyone downloads their apps,
16:02
Josh: the amount of apps that have gone into production that have been published recently has gone vertical.
16:06
Josh: I think it's doubled or tripled over the last six months. Everybody's publishing apps at the App Store.
16:10
Josh: The amount of five-star reviews and the amount of downloads has actually either
16:15
Josh: stayed flat or gone down.
16:17
Josh: It has not matched the amount of new apps that are going to the app store.
16:21
Josh: Why is this? It's because a lot of the apps don't have enough care applied to
16:26
Josh: them. They're just not great applications. And when I think about,
16:29
Josh: how I use my phone on a regular device or on a regular day or how I use my laptop
16:33
Josh: and the applications that I actually spend time on, there's a very fixed set
16:37
Josh: of them. And I'm a little stubborn when it comes to downloading new ones because
16:41
Josh: a lot of the new ones just are not great.
16:43
Josh: And I think a lot of that comes from this, this lack of care that is presented
16:47
Josh: from AI outputs, where if you're optimizing for a specific parameter that you
16:51
Josh: can measure, it's going to do it great, but it doesn't understand the subtle nuances of how humans
16:57
Josh: engage and how they really love to use these products like one of the products
17:00
Josh: that i use totally unrelated totally not sponsored but this app called copilot
17:04
Josh: money it's like a budgeting application and it's so thoughtfully curated and designed and.
17:09
Josh: And it really deeply understands all the complexities that are related to humans
17:13
Josh: when it comes to budgeting it understands a lot of the
17:16
Josh: the design characteristics same with an app called flighty i'm sure a lot of
17:19
Josh: people have heard flighty it's like a flight tracking application there's a
17:22
Josh: thousand ways to track a flight but flighty really cares about design they really care about how,
17:28
Josh: it's implemented with the human and they've created this amazing output and
17:31
Josh: i don't see that changing one thing that i did want to note is that,
17:35
Josh: i think when a lot of people see this they imagine a world in which they are
17:39
Josh: getting replaced everyone's like ai is replacing me look how much i could do
17:42
Josh: now it has these loops and i think the reality is it gives you a lot more agency
17:47
Josh: to do the things you want to do,
17:48
Josh: where maybe you're not doing the day-to-day where,
17:52
Josh: you would normally prompt an agent to do this but you're doing a lot of the
17:55
Josh: higher level tasks you can imagine yourself not having to do
17:58
Josh: the day-to-day like for example if you're just managing your household you no
18:02
Josh: longer have to take out the trash you don't have to run errands you could just
18:05
Josh: focus on how to make your household the best household it is because you have
18:08
Josh: that higher level ability
18:09
Josh: and in that chart that we showed in the artifact earlier on it shows a decreasing
18:13
Josh: sized human it's the amount of input that a human is needed to get the output you want,
18:17
Josh: but it's still ultimately on the human being in order to to push and navigate
18:21
Josh: towards the outputs that you want because ultimately these tools are just for us so when i think of.
18:26
Josh: Ai becoming increasingly good and when it comes to running the show even i've
18:30
Josh: leaned on it we both have i think a lot more recently but all that's done is
18:34
Josh: actually given us more leverage to do more with the show than have it replace
18:38
Josh: us and even in the case that.
18:41
Josh: We could clone ourselves. We could create a video version of ourselves that
18:44
Josh: has a perfect voice that sounds just like us. I don't think people actually want that.
18:48
Josh: There's that lacking human nature that still isn't understood.
18:52
Josh: And I find that it's more empowering when I hear that these loops exist that
18:55
Josh: can run for days on end and create amazing outputs versus not where it's kind
19:01
Josh: of extracted from us. I don't really think that's true.
19:03
Ejaaz: Yeah, it's like that stat of, well, it's that thesis that everyone held about
19:08
Ejaaz: a year ago, which is like with the increase of AI adoption, people will have
19:13
Ejaaz: more free time to have fun and leisure.
19:16
Ejaaz: And in fact, the opposite has shown that like people just work way more and work harder.
19:21
Ejaaz: And the output of that work is measured across like pretty much every single
19:24
Ejaaz: company and profession and role.
19:28
Ejaaz: I do generally agree with that. I don't think humans are going to get wiped out anytime soon.
19:33
Ejaaz: But one thing that is kind of nagging my brain is if we extrapolate this intelligence out enough,
19:42
Ejaaz: there is no reason why AI won't be able to take over or replace other parts
19:48
Ejaaz: of the cognitive process that a human can do,
19:52
Ejaaz: particularly if it's one AI model trained on the entire corpus of knowledge
19:57
Ejaaz: that a bunch of humans have been guiding it.
19:59
Ejaaz: So when I think about Anthropic, when I think about OpenAI, I think about all the
20:05
Ejaaz: millions of people that use their product every single day and the data that
20:08
Ejaaz: they ingest every single day that gets recorded on one singular database that
20:12
Ejaaz: can then be reused to train a better model that is more hyper-optimized towards humans.
20:18
Ejaaz: You could argue that as a single human, you don't get to meet and read the thoughts
20:22
Ejaaz: of every other human that is out there.
20:25
Ejaaz: You have your very own individual process. And I think that an AM model that
20:29
Ejaaz: can get access to the world's brain and thoughts could probably create something
20:34
Ejaaz: kind of close to knowing what that human taste profile would be.
20:39
Ejaaz: The other major question that I'm wondering is, how much is all of this going to cost?
20:46
Ejaaz: One, like, stat that has stuck in my head over the recent few weeks is that
20:50
Ejaaz: Philanthropic particularly, they service, or like,
20:53
Ejaaz: the Fortune 10, the top 10 companies in the world, nine of them use Clawed,
20:58
Ejaaz: and their budget's increased by 500%, or is projected to increase by 500% by the end of this year.
21:04
Ejaaz: And they're doing this willingly because the ROI, the value that they're getting
21:07
Ejaaz: out of that is pretty massive.
21:09
Ejaaz: Alternatively, there are companies like Uber that have slashed their budgets
21:13
Ejaaz: massively because their entire year's budget was spent in a couple of months.
21:17
Ejaaz: So I'm wondering, in this world of agent loops where you've got AIs working
21:22
Ejaaz: overnight for you, the bills are going to increase pretty massively.
21:25
Ejaaz: And I'm wondering, unless these AI models don't get cheaper,
21:29
Ejaaz: and there's an infrastructure bottleneck there where these GPUs cost a lot of
21:33
Ejaaz: money, we can't scale power and infrastructure anytime soon.
21:36
Ejaaz: We need so much more energy than we already have currently on Earth to be able to power these things.
21:42
Ejaaz: The cost of these things are just going to go up a lot more massively,
21:46
Ejaaz: which means that either this is only going to be a power or a tool reserved
21:51
Ejaaz: for the rich, or something's going to break here and maybe open source models
21:54
Ejaaz: get adopted more aggressively.
21:56
Josh: Yeah, I imagine there's probably use cases for all of the above.
22:01
Josh: It's like open source models will continue to improve they'll be able to do
22:04
Josh: a lot of the more trivial tasks that don't require frontier intelligence so
22:08
Josh: therefore the cost of those types of loops will go down because not everyone
22:11
Josh: needs to have the most cutting edge,
22:13
Josh: software stack engineering like they're just kind of having it help them through
22:16
Josh: their day-to-day maybe it's replying to emails maybe it's whatever miscellaneous things it may be
22:21
Josh: there's a high probability that these open source models as they continue to
22:23
Josh: improve will be able to bite off a meaningful chunk of that then the other half
22:27
Josh: is using these frontier models that is a requirement in order to get the absolute
22:30
Josh: best results for whatever very challenging work they're doing.
22:34
Josh: And that is going to cost a lot of money for sure.
22:39
Josh: And I don't see that changing, but I think the output of the dollars in will continue to go up.
22:46
Josh: It's because as you get more knowledge per token, as you get more output per
22:51
Josh: prompt, it very clearly, I mean, the economics seem to make sense.
22:55
Josh: And I think that's kind of right now.
22:58
Josh: Enterprise spend on these models they're trying to figure out well how much
23:02
Josh: value can we actually get back from every dollar spent and right now it's a
23:06
Josh: little bit unsure you mentioned uber we have uber here that we're showing on screen
23:10
Josh: where uber just recently put a cap on the amount of tokens that
23:15
Josh: their employees are allowed to use at fifteen hundred dollars per engineer per tool per month and
23:21
Josh: we'll see how that works because a lot of other companies that we know they're
23:24
Josh: kind giving their engineers unlimited budget in fact they're kind of ranking
23:29
Josh: the engineers based on how many tokens they're using per month and.
23:34
Josh: We'll see where that goes. I suspect the companies that are spending more on
23:38
Josh: tokens will continue to see a higher upside for now, at least.
23:42
Josh: But like you mentioned, the underlying problem with all of this is we're going
23:46
Josh: to continue to have more prompts. I mean, these loops consume a tremendous amount
23:48
Josh: of tokens, whether they're frontier tokens or open source tokens.
23:51
Josh: It doesn't matter. We're going to need orders of magnitude more than we have.
23:54
Josh: And we don't have the computability. It really does always come down to that
23:59
Josh: energy problem, that infrastructure problem.
24:01
Josh: We don't have the infra built out to support this so therefore the costs likely
24:06
Josh: continue to stay high maybe it's not because you're paying the provider for tokens
24:11
Josh: perhaps it's just renting the gpu time from a cluster that is doing much more
24:15
Josh: valuable work so i think that might ultimately be
24:18
Josh: that crux is the actual availability of the compute to do these things and that's
24:23
Josh: why these edge compute devices like having your,
24:26
Josh: mac studio on your desktop that can run locally it's probably a pretty valuable thing to have.
24:30
Ejaaz: So I'm sure a lot of you are wondering, you know, how does this apply to me?
24:34
Ejaaz: You know, I have none of my friends have mentioned this loop feature.
24:37
Ejaaz: I don't really know many people who are using it.
24:40
Ejaaz: As we mentioned earlier, like this isn't probably going to be used by the bulk
24:43
Ejaaz: or majority of people yet until some of those use cases actually arise.
24:47
Ejaaz: I think it's mainly going to happen in the workplace. It's going to happen with
24:51
Ejaaz: like some of these enterprise companies that are trying to automate certain
24:54
Ejaaz: departments or functions of their particular a company like marketing,
24:57
Ejaaz: like software engineering.
24:58
Ejaaz: And I think it'll start with lower level tasks because these agents still aren't
25:02
Ejaaz: smart enough to understand nuance completely.
25:05
Ejaaz: And also, you don't just want to let an agent run loose overnight whilst you're
25:09
Ejaaz: sleeping and then take down your entire company. And one place where it's working
25:12
Ejaaz: tirelessly to accelerate the development of that
25:18
Ejaaz: And we have Boris Cherny over here basically explaining how he's basically ditched
25:23
Ejaaz: his integrated development environment.
25:25
Ejaaz: He has ditched all of his normal tools that he had spent decades basically honing
25:30
Ejaaz: his software engineering skill on to now completely focus on building up these
25:34
Ejaaz: agent loops. And what is he focused on?
25:36
Ejaaz: Well, he works primarily on cloud code, but the other folks at Anthropic and
25:40
Ejaaz: OpenAI have started this thing called
25:43
Ejaaz: Recursive self-improvement or RSI, which is basically the goal of getting your
25:48
Ejaaz: AI model to build the next version of itself.
25:52
Ejaaz: And this is a test that Anthropic and the folks at OpenAI do for any new model that they release.
25:57
Ejaaz: They set it a goal or task to basically rebuild itself in a more improved fashion.
26:02
Ejaaz: Now, one thing that the AI has gotten really good at is building out that next function.
26:07
Ejaaz: But one thing it's not very good at is figuring out what research problems they
26:12
Ejaaz: should fix, what research problems it should focus on to try and,
26:16
Ejaaz: you know, overcome and make it ultimately, you know, a better model than its competitors.
26:21
Ejaaz: Now, RSI is something, it's kind of like the golden egg that each AI lab is going after.
26:27
Ejaaz: And this is the primary use of agent loops right now.
26:30
Ejaaz: And you can see why it might be obvious. If you have an AI model that can basically
26:34
Ejaaz: build the next best version of itself, eventually you're going to get to AGI,
26:38
Ejaaz: whatever the hell that looks like, and then you can apply it to pretty much any sector.
26:42
Ejaaz: Now, the problem and the worry that kind of immediately pops into my head and
26:45
Ejaaz: a lot of these researchers head is, if it eventually does get that smart, right?
26:51
Ejaaz: Could escape human control completely and run off on its own and do its own
26:55
Ejaaz: thing. Because at that point, why would it need a human to kind of like guide it or shepherd it?
27:00
Ejaaz: Instead, it can just kind of like do its own thing. So this is like the primary
27:03
Ejaaz: use case that I'm seeing for agent loops being worked on right now.
27:06
Ejaaz: I would love to see a like more broader application across like kind of like
27:10
Ejaaz: consumer professions, like in finance, like in science and stuff like that,
27:12
Ejaaz: which I do believe it'll spill over eventually. But unless you're seeing anything
27:16
Ejaaz: else, Josh, I think like that is primarily it on agent loops and agent autonomy.
27:21
Josh: It's on you to figure out the best use cases for it. Like there's no real company
27:25
Josh: defining it. They're just giving you the tools. And I mean, for better or worse,
27:29
Josh: it's very open-ended. So it's on you to figure out how best to use these.
27:33
Josh: I think if this sounds a little overwhelming, maybe we could outline a few examples
27:36
Josh: of each one of these kind of four rungs in the ladder here.
27:39
Josh: The first one being prompting this everyone has done before.
27:42
Josh: I'm sure it's like rewrite this
27:44
Josh: email to sound more confident or explain what my doctor meant by this.
27:49
Josh: But then you've probably also used the partial agentic usage as well of these
27:53
Josh: models, which is like planet my three day vacation to Lisbon that I'm going on next week.
27:58
Josh: And it will actually go off and use tools and it will think complex thoughts
28:02
Josh: and ideas and kind of surface you a full itinerary for your trip.
28:05
Josh: And then there's the third one, which is the harness. This is a little more
28:08
Josh: complicated. This is for people who are building more project based stuff.
28:11
Josh: So for example, if you want to build you a website for your dog walking business
28:15
Josh: and you kind of describe it and you go back and forth on a spec and then it
28:19
Josh: goes off and implements that.
28:20
Josh: And the fourth is loops, which doesn't have to necessarily be overwhelming.
28:24
Josh: It can be simple as let's say you are.
28:27
Josh: Interested in the news, you could say every morning before I wake up,
28:30
Josh: scan these 10 sources plus market data and give me this bulleted brief.
28:35
Josh: Or let's say you have a to-do list. It'll go off and think overnight and solve
28:38
Josh: all those problems overnight, iteratively until it comes to a solution that
28:42
Josh: it hopefully arrives at in the mornings. There's a lot of use cases.
28:45
Josh: I think a lot of it requires creativity.
28:47
Josh: And that is the prompt we will leave you with today, which is share with us,
28:51
Josh: please, how you are using these models best.
28:54
Josh: Because so much of the question isn't are these models smart?
28:57
Josh: It's how can I extract that intelligence from them in the most effective way
29:00
Josh: for my life? So I would be so curious to hear which rung of the ladder you find
29:04
Josh: yourself on one through four.
29:05
Josh: And then what the most interesting use cases you found,
29:08
Josh: among those rungs of the ladder are you using loops currently what are you using them for,
29:13
Josh: are you with agents are you still using it as a google extension if you're still
29:16
Josh: using it as a google extension i would encourage a little more creativity really
29:19
Josh: try to ask harder questions and figure out how it could be implemented in your
29:22
Josh: life but i think that's pretty much it on the loop um,
29:26
Josh: you're not going anywhere but your job might shift a little bit in terms of
29:31
Josh: scope as these tools get more powerful and that should be the hope that should
29:34
Josh: be the goal because it'll allow you to do so much more that you want to accomplish, I believe.
29:39
Josh: And yeah, I think that's where we'll leave you with today.
29:42
Ejaaz: Thank you folks so much for listening. Similar to Josh's prompts,
29:46
Ejaaz: I'm actually kind of curious, for one singular task that you've used your AI
29:51
Ejaaz: for, what is the most number of tokens that you've burnt?
29:53
Ejaaz: Be honest, it can be for any use case, doesn't matter, let us know.
29:57
Ejaaz: And also, what is the longest that you've had an AI work on a particular task
30:01
Ejaaz: for? Is it a couple of minutes? Is it hours? Is it potentially overnight?
30:05
Ejaaz: Let us know. I'm curious. And what was the associated bill with that?
30:09
Ejaaz: And yeah, we'll see you on the next episode. Wherever you listen to us,
30:12
Ejaaz: if you haven't subscribed, if you haven't rated us, if you're not leaving us
30:16
Ejaaz: comments, what are you doing?
30:17
Ejaaz: We respond to pretty much any and every one of them. We listen to your feedback.
30:20
Ejaaz: It feeds into some of the work and content that we put out.
30:23
Ejaaz: We are almost hitting 60,000 of you folks. And you guys are reading our newsletter,
30:28
Ejaaz: which is like hit out to about 100,000 plus people. every single week. We post twice a week.
30:34
Ejaaz: But yeah, wherever you are, please subscribe to us, leave us a comment,
30:37
Ejaaz: and we'll see you on the next one.
30:39
Josh: See you guys next time.
30:40
Ejaaz: Peace.