Latest / Agentic Conversations (formally mlops.community) / Omnigent: Composition, Control, and Collaboration for AI Agents

Transcript

Omnigent: Composition, Control, and Collaboration for AI Agents · Agentic Conversations (formally mlops.community)

0:00
That's exactly what's going on right now, which is basically
0:03
that service based architecture SOA, you know, we're we've
0:08
switched from this idea that we had to go ahead and originally
0:13
it was 1 central command to understand what the budget was.
0:16
And now that got pushed down to developers because the
0:19
developers could actually then care, understand and help
0:23
control that token. Novice is the exact same thing
0:26
100%. I will have you know I'm still
0:34
waiting for the coffee from Taiwan that you promised you
0:37
were going to send me after that blog post you.
0:39
Wrote and I'm and I'm glad you're recording it to basically
0:42
make sure that I'm held. Accountability one O 1 Yeah,
0:45
this is. No, no, you're right, I'm wrong.
0:47
The problem is that I'm actually not going this summer,
0:50
unfortunately. But if when I do, yes, there is
0:55
a specific set of farmers and a specific set of places that I
0:58
want to go to. And it's actually interesting
1:00
enough related to agents and we will get into that when you're
1:03
ready to go. But it's actually related to
1:05
agents, my friend. Are are there some of your
1:08
agentic workflows that you've got going on finding different
1:10
micro fincas around the world? Not just that, but also which
1:16
regions in Taiwan versus which regions of Japan or Korea
1:19
specifically for macho and shit like that.
1:22
What? How?
1:25
How are you getting agents to do this research?
1:28
Is it just like exploring the? Yeah, it's exploring.
1:31
So so basically, OK, so we're probably jumping ahead of her,
1:34
but like I'm using omnigents, Polly.
1:37
Sorry, Debbie, excuse me. That's the one where basically
1:40
they debate each other, right? And so this case is basically a
1:43
my ChatGPT, you know, 55 versus my Claude Opus 47, right?
1:48
And they basically debate each other.
1:49
And so basically I start, I, I already have enough like agentic
1:54
memory and skills, MD's and all this other shit that provide
1:58
like a lot of huge background on what I'm specifically looking
2:01
for anyways, right? So if you go to my blog either
2:04
like the daily jeely.com blog, the personal one, you'll
2:07
remember that there's this one asinine long Taiwanese blog
2:11
about Taiwanese coffee, right? It was supposed to be Yeah,
2:14
Yeah, exactly. Yeah.
2:15
It's supposed to be small. And then it turned out to this
2:18
basically be this long diatribe that went from like, here, all
2:21
the great coffee shops in Taiwan, to like, let me go visit
2:24
a farmer in Taiwan, in Nanto Province, in Nanto County,
2:27
specifically to go ahead and understand how he's going to
2:29
beans and how like. Yeah, like, and how used to be
2:32
Bidlon leaves, right? And then how the Taiwanese
2:35
government was switching over to convince people not to do grow
2:38
bin Laden leaves because bin Laden is a it's like a like
2:41
cigarettes, people get super addicted to it.
2:43
But also the leaves are huge, so the water actually degrades the
2:47
land underneath it. So the Taiwanese government's
2:48
trying to convince people to grow coffee.
2:50
Well the this is already too long, but the TLDR version of
2:55
this is that Nanto County is actually famous for growing tea
2:59
and 1st before growing coffee. So I've been actually using a
3:03
homogene to find me specific regions in Nanto to grow matcha.
3:12
Wait with the intention of you going there and then growing
3:17
matcha or? Partnering or working with the
3:18
farmers, working with the farmers in that area to grow the
3:21
matcha. And so some interesting tidbits
3:24
is that the oxidase when you, when you working, when you pick
3:29
the leaves, right it, it could oxidize really fast.
3:32
So you actually have to steam and dry them immediately.
3:34
So, so the problem like Japan and Korea, they both have the
3:38
infrastructure set up that next to where the leaf picking is
3:42
actually happening, they can immediately deal with the
3:45
steaming and drying OK and of the leaves.
3:49
For example, this is a good setup for coffee because coffee
3:53
in the coffee regions, they know there's like bourbon style,
3:57
honey style, whatever else, but it's all about drying the coffee
3:59
beans after you pick them and how do you do it correctly,
4:01
right? And so the for oolong and other
4:05
types of teas in Taiwan, they're all set.
4:06
That's nonto's famous for that, for that producing amazing like
4:10
for example, Juan tea, like they're, they're great at that
4:13
stuff. But for teas that oxidize fast,
4:16
like they, they don't have the infrastructure yet.
4:21
And so the idea is to actually ensure that they have the, the
4:24
steaming and the drying technology there right away #1
4:28
but then is there an area in Taiwan that actually can
4:31
actually handle the how to deal with oxidized tea?
4:36
Basically, like once you've tried, how do you process it?
4:39
There's only one area in Taiwan that actually knows how to do
4:42
it. It's a it's a, it's a region
4:43
called Sung Chao, which is near Banchao.
4:47
Banchao is like a, a major city in Taiwan.
4:49
Yeah, OK. And so ironically, that's an
4:52
area that I've been trying to convince my family to move to
4:54
like in other like to stay in Taiwan just because I love the
4:57
area a lot. It's got, it's got a lot of
5:00
great cycling, dude. Like the, the cycling there is
5:02
gorgeous. There's a lot of really, really
5:04
painfully painful hills. It's amazing.
5:06
So, so, so I'm basically getting the agents to debate with each
5:10
other to get me to dive in. And so it's showing me which
5:13
areas of social I can work with specifically, which with
5:18
farmers, which small distributors I can work with to
5:20
basically figure out how to go ahead and have them process the
5:24
oxidized tea as well. So it's like, if I'm actually
5:28
ever decide that I want to go into that business, then the
5:31
idea is like, yeah, like obviously it's a side business,
5:34
but as you can tell, I'm just a tiny bit OCD stuff.
5:37
Like that? Just a little bit, yeah, yeah,
5:40
just just a little bit. So so because of that, then the
5:43
idea is like the one of my fun projects when if I'm not cycling
5:45
would be like, OK, let's figure out how to actually like grow
5:49
high quality matcha. So obviously Japan has the lead.
5:52
Korea's infrastructure is amazing in typical fashion, the
5:55
Korea does. And so for me, I'm like, huh,
5:58
can I do this in Taiwan? Because the the final tip is
6:01
Taiwan Nanto elevation is typically isn't high enough to
6:06
grow some of the very interesting coffee beans.
6:10
Yeah, it does. So for some reason it works even
6:15
though it shouldn't exactly. You have the theory that that
6:18
also can translate to matcha, matcha.
6:23
And now with these agents debating each other, that's just
6:27
giving you all the context that you would need.
6:29
If you want to pull the trigger and say, OK, I'm going to go do
6:32
it now, I've got it, all the information I need, I can point
6:37
another agent at it and say read through this context and then
6:41
give me what I need. Yeah, so I save obviously all
6:46
the all the like, it's the old like build skills or build an
6:49
agentic memory to store all the stuff.
6:51
So I have a reference point because there's only so much
6:53
context you can compress before the thing blows up, right?
6:56
But don't forget it's like things like, OK, cool, now I
6:59
have a general idea of what's going on.
7:01
But then how do I for seek argument?
7:04
My mandarin's horrible. OK, let's be very clear.
7:06
It is disgustingly bad. OK, my at the time 5 year old
7:12
when we're in Taiwan makes fun of my Chinese accent and she's
7:15
it's fair. It's fair game.
7:17
It's not even like no, that's true, right?
7:19
Because my my Mandarin sounds like I'm speaking like
7:22
Cantonese, like I have a Cantonese accent Mandarin.
7:25
And mainly because I've loaded most of my Mandarin by watching
7:28
Jackie Chan movies that were dumped into Mandarin.
7:30
And so they had a Cantonese speaker speaking Mandarin.
7:32
And that's how I actually learned a lot of my Mandarin.
7:35
I wish I was joking. This is completely freaking
7:37
true. OK, so, so in other words, no,
7:41
I'm, I'm bad. OK.
7:44
And so now I need to go ahead and mess around with agents to
7:48
go ahead and read Mandarin properly.
7:51
And a lot of it's handwritten, not digitized, because we're
7:54
talking small farmers, small distributors, right?
7:56
So this is when I start switching over to the the
8:00
DeepSeek models, except now here we get into this, the DeepSeek
8:04
models and all the Chinese open source models are great, but
8:07
they're for simplified Chinese, not for traditional Chinese and
8:10
traditional Chinese because there's actually there's
8:13
actually more context. So you actually need more tokens
8:17
in order to process the same amount of Chinese characters for
8:21
traditional Chinese versus simplified Chinese.
8:23
OK, I forget the name of it. Those are really good.
8:26
Cool researcher, she had done all this research about creating
8:30
Taiwanese specific models. All right, LMS and how that it
8:35
was actually required more context required more tokens to
8:38
process this in essence the same amount of words.
8:41
And so there was some, I forgot who did it, but there was some
8:44
person was actually trying to do like, no, no, we'll translate
8:47
the, the traditional Chinese to simplify Chinese run against the
8:52
LMS that way and then translate it back.
8:55
And because, because then they'll be less tokens required.
8:58
And the translation actually is relatively straightforward.
9:01
It's just a direct mapping in a lot of cases you don't like need
9:05
thinking. In that case, it's like, no,
9:07
here's a simple vibe version. Here's the traditional version.
9:09
So I'm actually still testing that out.
9:11
I, I screwed up my selections of those models.
9:14
Why? Because my Mandarin sucks.
9:16
I think that's the thing here. But the idea is that I'm, I'm
9:19
trying out different things like that because then that's the
9:21
whole point. Like how do I then work with the
9:27
traditional farmer, the traditional distributor, with
9:31
the ones that actually have real longevity and real knowledge of
9:37
the topic, right. And so, so part of the reason
9:42
I'm bringing this up, by the way, is just because a lot of
9:44
people talk about how AI agents are removing jobs, right?
9:48
And, and there's obviously validity of that statement by
9:51
every stretch of imagination. But I keep trying to remind
9:53
people like, yeah, but it's like every other technology we have,
9:55
it also introduces new jobs. It also introduces a way for us
9:59
to rediscover things that we forgot using this small example
10:03
of like this, my obsession with matcha or my obsession with tea
10:06
or coffee in general, right? No, just any obsession I have,
10:09
right? Now you can just ask an agent to
10:11
go do these things that you probably would have never gotten
10:14
around to doing. No, because think about how much
10:16
research you can think of how horrible my Mandarin is, right?
10:19
And so instead I'm having fun being a geek again, But then I'm
10:22
also connecting with old farmers.
10:26
I'm connecting with old distributors.
10:27
Like this is fucking sweet. Like, so in other words, I get
10:32
to go ahead and work with people that normally I wouldn't even
10:35
have a clue to where to even start now, right?
10:38
Which is. Which is to say that's.
10:40
That's cool. Yeah, that is very cool that.
10:43
And so you've got these you mentioned there's the setup
10:49
where the agents will debate each other and they debate each
10:52
other just because you're saying, hey, look, give me a
10:57
reason not to do this, like try and.
10:59
Find. No, it's not even that.
11:00
No, no, no, it's it's actually better than that.
11:03
It's not like, because most of the time, at least with the
11:06
prompts I use or the context I'm using, it's not about doing or
11:09
not doing. No, you, you've known me.
11:11
Not unknown Demetrius, I'm always going to go do it right,
11:17
right. But my problem isn't am I going
11:18
to do it or not. My problem is like, am I going
11:21
to basically slam into a wall or how thick that wall is when I
11:24
slam into it right? That that's usually my problem.
11:26
So so you can tell like basic how I look at the tech is
11:31
actually you can still look at it from an optimistic
11:34
perspective, right? And so for me, it's not about
11:37
like, should I do it or not should I do it?
11:38
It's more like what's the right way to do it?
11:42
Like. And so in other words, as you
11:44
can probably hear or tell from the way I'm describing
11:46
everything, my obsession is high quality going back to the roots
11:51
of actual like tea making, right?
11:54
I'm not an expert in that at all, but I respect the quality.
11:59
I respect the art. I respect the cultural
12:02
significance of of it. That that I can say.
12:05
Yeah, I can vibe with that. Right.
12:07
And so because of that, then the idea is that what I'm asking the
12:09
agents is basically look, I'm giving the agents that context.
12:14
So then why they're debating with each other, like, oh, well
12:19
then what region of Taiwan am I looking at?
12:22
What's the elevation, what's the pH, what's the soil quality?
12:26
Am I dealing with lots of, for example, like the theory is if
12:31
we were to grow matcha in Taiwan, it would be more floral
12:35
based, especially in Nanto County.
12:37
Why? Because that's what Nanto's
12:40
famous for. Like their teas have a lot of
12:42
floral, lot of lychee like. So for example, like coffee,
12:45
that coffee block that I, I sent you, right?
12:48
That one, his coffee beans were notorious, were famous,
12:53
notorious, the wrong word famous partially because there was very
12:59
strong, very, very strong hints of lychee in the coffee beans
13:02
themselves. Right.
13:03
And the reason why is because literally the lychees were
13:06
growing wild in his coffee farm. So literally like no joke, we
13:11
when we got there to try out the coffee, the grandma was
13:15
literally just giving us lychees for free.
13:17
And just like, don't you want us to pay for them?
13:19
No, no, they grow everywhere. They're literally a weed at this
13:22
point. Like I'm like, I've never heard
13:24
lychees referred to as weeds, but OK, like right, but that's
13:28
literally the context. It's like 05 be able to find
13:31
cool things like that. And so that's the that's the
13:35
attempt to discover. That's the attempt to like try
13:37
to like somebody like myself who has the obsession of it or the
13:42
OCD Ness, but not necessarily the knowledge.
13:45
Letting the two systems debate with each other provide me the
13:49
additional information to say to know what other questions to
13:52
ask. Yeah, because when you read
13:55
through that debate, then you go, oh, I didn't even think
13:58
about that. That is such a great trick that
14:03
I recognize and you know what? I one of the actually toys that
14:07
I've been making, and I don't know why this isn't a common UX
14:13
for dealing with agents is I now have created a little widget
14:21
that you can highlight any piece of text, whether you're inside
14:26
of the chat window with the agents or you're on a website.
14:31
And it will pop up a little prompt box and it will read the
14:35
context of whatever the either the paragraph or the whole page
14:38
that you're on. And you can prompt whatever ask
14:42
a question to it just about. Because a lot of times what I
14:45
recognize with myself is that I don't want to continue this
14:50
conversation in the actual chat that I'm having.
14:53
I want to shoot off and do this on a different thread because
14:57
it's just like something that isn't necessarily needed to
15:02
pollute this conversation with the agent that I'm having.
15:05
But I still want to know like, hey, that you see, you reference
15:08
this thing here. What does that mean?
15:11
Go deeper. Give me more context on this
15:13
specific thing, but not in the main chat.
15:16
So it's like little sidebar conversations and I've got this.
15:22
I, I use this tool called clearance to read like markdown
15:25
files. And so now I created a little
15:27
right hand sidebar where you can have a side chat with an agent.
15:33
OK, I I know this is definitely wasn't done on purpose for
15:36
everybody that's listening to us right now, but you you wonderful
15:39
Segway to Omniton by the way, because OK, so for example, I
15:45
did this already. So I when I first started asking
15:48
the question about Machos and things like that, like what
15:51
regions, I learned a lot about more about the regions of Japan
15:56
for starters, and then the regions of Korea.
15:58
All right. But then where my end goal was
16:03
to figure out if I could do it in Nonto County in Taiwan.
16:06
All right. But of course, I, as I'm to your
16:10
point, I'm learning more about these reasons to Japan and these
16:13
reasons in Korea. I'm like oh this is really cool
16:17
so I want to know more but now. So I really wish there was a way
16:20
to fork my conversation. Yes, that's it.
16:24
So homage, it has the ability for me to fork.
16:28
So that's what I did. I basically had the
16:30
conversation, I forked it off that I allowed myself to
16:33
continue one conversation specifically about the Japan and
16:36
Korea areas, while the other one the the main line, I just
16:41
redirected the Nanto County. And so they both had the
16:44
original context by which we we had the conversation with.
16:47
But now I can have two very separate conversations so I can
16:50
learn more. Why?
16:51
Because I'm fortunate enough for it, because I live in Seattle.
16:56
No, I'm not being paid for this, but there's direct on Alaska
17:00
Airlines from Seattle to Tokyo, Seattle to Seoul now.
17:05
So I'm going. So yeah, so I hopefully maybe
17:08
they'll give me something for that.
17:09
But in that the reason I called that I was like.
17:14
Sponsored by Sponsored. Right.
17:16
But no, it's not I wish I'm not that we're we're both not big.
17:22
We're both not big enough for that right.
17:23
But. Exactly.
17:24
We need to become the true travel bloggers.
17:27
Exactly. Yeah, we're.
17:28
Just we're not we're not travel influencers.
17:30
So unfortunately that's not going to work.
17:31
But not yet. There you go.
17:33
True. Point taken.
17:34
Yeah, we we have goals, right? But nevertheless, the idea being
17:38
now I'll go directly to Korea, I'll go directly to Japan so I
17:43
can explore the regions myself. Yeah.
17:46
So that's why I started going down.
17:48
So I forked the conversation specifically so I could go do
17:50
that, yeah. But I don't understand why that
17:52
isn't a common UX pattern with these agents.
17:55
And specifically for me it's like I have this one sentence or
18:00
these 5 words that appear together in my context and I
18:04
want to fork them out and have a separate thread on that specific
18:08
piece. Like explain more but I haven't
18:12
seen any of the labs create this as a integral part of how we
18:20
interact with agents and so it it makes me think like maybe I'm
18:25
the only one who cares about. That no, no, no, no, no, no, no,
18:28
no, no, no, no. You're, you're look, when we
18:31
started all this, this entire AI community, right?
18:35
Think about how chat windows are still the even after all this
18:40
time, chat windows are still the way we work with things even our
18:44
sticking when we decide to use CLI and terminals, it's still
18:47
basically ACLI chat window That's basically all this for
18:51
all intensive purposes, right? Just that we might have some
18:53
Jason stuff shoved in there. Now that's that's the extent of
18:56
the difference, right? The reason why is because, and
19:02
you actually see harnesses like being created right now, like a
19:05
bunch of start-ups, a bunch, even the major lab, they all
19:07
create harnesses themselves, right?
19:09
And the reason why is because it's like look up to this point.
19:13
And this is like our definitely our learnings within database
19:16
itself, right? We all had like 5 terminal
19:20
windows open, 6 browser windows open.
19:23
Yeah, we're all going back and forth, back and forth.
19:27
You deep dive on a particular topic.
19:30
I'm coding something for Delta Rust, right?
19:33
Like so I can get off of T for a second, right.
19:36
So I'm, I'm coding something for Delta Rust.
19:38
I'm trying to understand the latest version of arrow or and
19:45
data fusion because they keep changing.
19:48
Hats off to the convenience. This is not a complaint.
19:49
This is just, but they do they do change things.
19:52
So we're trying to keep Delta Rust up to date, right?
19:54
And so when we're trying to make those changes, then I need to
19:57
run some terminal windows to I need to figure out.
20:02
What's the current version? Another terminal set is creating
20:07
the tests harnesses that I want to go create another browser
20:10
window is giving me oh, what's the latest security contacts?
20:14
What's the latest like, you know, security alerts associated
20:18
to each and everyone, the 20 different dependencies we have
20:23
that, that we have to go through, right?
20:25
And, and what are you doing? You're opening multiple windows
20:28
and you're losing context, no pun intended, right?
20:32
And you're, you're losing all of the connections because you've
20:39
got so many windows, so many terminal windows open, right?
20:41
So you know how Amazon made the idea of like the each team
20:48
should only have like 2 beats of boxes, right?
20:51
All right, the whole reason that was Crave was because it was the
20:56
idea that each team should only have five to seven people
20:59
reporting to it. OK, so each manager should only
21:02
have 5:00 to 7:00 because anything greater than that, they
21:04
basically start losing context and again, pun intended.
21:07
OK, well, this was determined, not Amazon might have made it
21:11
famous, but it was actually a military thing.
21:13
Like military, military command. In terms of units, teams
21:17
understood that each commander could only logistically handle
21:23
five to seven people reporting to them in order to be
21:26
effective. Right.
21:27
And so I'm not military, but I I am.
21:31
I used to be King County OPS, the search and rescue OPS.
21:39
Excuse me. I'm sorry.
21:40
I'm trying to remember SNR look, this is how tired I am.
21:43
OK, dude. Search and rescue operations.
21:46
So we learned that from so SNR OPS learned that from obviously
21:50
military. And then the idea is that this
21:51
is a a time value tradition, like an understanding and so so
21:56
in the tech world, everybody thought about two pizza boxes,
21:59
but basically this is a well understood logistical structure
22:04
within the military. OK.
22:05
And so cool. The reason I bring that up is
22:09
because, well, then in a lot of ways, if you consider an agent,
22:14
I'm almost like a junior developer that you're managing
22:20
once you go beyond 5. Yeah, yeah.
22:22
Once you go with seven, you're losing track of everything right
22:26
And so more than anything else, the things like to your point
22:31
exactly, but I want to go off and do that it's actually going
22:36
to make you lose context even more so right.
22:39
Part of the reason why we we've had basically built this idea of
22:44
forking was actually not originally that idea you talked
22:47
about, by the way, like to be very clear, like it was more
22:49
about literally we're coding. So you fork.
22:53
So that way you can literally go ahead and try out different
22:56
ideas, but had the initial same context.
23:00
But to your point exactly, coding and actually having a
23:03
real conversation isn't actually that much different.
23:05
Coding happens to be just around code, but it's still an
23:08
abstraction layer, right? And so conversations are just
23:10
another abstraction layer. Like everyone's like, well,
23:12
Cody, no, that's real code. I'm like, no, are you writing
23:14
this thing in binary or assembly?
23:16
No. OK, well then stop talking to
23:18
me. It's an abstraction layer, OK.
23:20
And so the idea is that if you've got all these different
23:24
things floating around right, you will lose context.
23:27
You will not remember why you asked in the first place.
23:31
So there's so there's a lot of research being done by the labs
23:35
and to an extent even a lot of the harnesses, which is like how
23:38
do you basically have now an agent fleet and the fleet manage
23:44
itself. Right.
23:45
And there's various like by the time this podcast comes out,
23:49
which is only a week or two, I'm sure everything will change.
23:52
So I'm not going to bother with the pretense of knowing what's
23:54
going to happen with fleets. But irrelevant, whether you you
23:57
call the fleet or you talk about an agent or there's a, some
24:00
other layer, the the whole reason we built what we built
24:04
was because we understood that this was going to be a problem
24:06
either way. And just as importantly, or if
24:11
not in many cases more importantly, the models keep
24:14
changing. The hardest keep changing.
24:18
I want to be able to go ahead and move all that context, all
24:21
the agentic memory, all that configuration.
24:23
I don't want to rebuild it so now I can use the latest open AI
24:27
model or the latest DeepSeek model or the latest anthropic
24:30
model or whatever it is I'm using.
24:32
I want this one layer that says whatever harness you're using,
24:36
whatever model you're using, it'll flow through irrelevant.
24:41
Where I get a little bit confused there is the harnesses
24:45
are optimized for certain models, right?
24:48
I guess if you're building a harness, you're building it for
24:52
a model and there's certain things that the labs are going
24:55
to be doing on their own that's like the inner harness and then
24:59
you have the outer harness and the inner harness.
25:02
They're RL ING that model so that it works really well with
25:07
their harness. 100%. How do you like envision these
25:12
hot swaps in a world where you know that?
25:16
Oh, because that that's actually why we value Omnigen as an open
25:20
source project to begin with, right?
25:21
Because the idea is that like for example, even just building
25:24
it, we notice for example, what deems to be simple but actually
25:30
changes people's workflows is like approvals, like the
25:34
approval process. I'm not going to get this quite
25:37
right. So I'm going to stay vague.
25:38
So that way I can, I can not make it sound like 1's better
25:41
than the other because that's not true at all.
25:42
OK, But the process is slightly different between harness A and
25:48
harness B and the process is slightly different between model
25:51
A and model B, just to be very vague.
25:53
So that way, again, I'm not calling anybody out, right?
25:56
And again, it's not even calling because like I said, it's not
25:58
like 1's better or one's worse. It's just purely they're
26:01
different. That's all this is OK.
26:03
So if if you look in the Omni code base, that's we actually
26:07
reflect that, right? And so then that way we're
26:10
actually leveraging what model A or harness A is really good at
26:18
because like the creation object isn't about going and saying
26:22
we're a better harness than everybody else.
26:25
We're not. No, that's why we call ourselves
26:27
a meta harness because we're saying no, no, we're on top of
26:30
this. So that way you, the individual,
26:34
the person that's using these things, all that configuration,
26:38
all that context, all that agent of memory that is portable, that
26:45
you have control over, that you can flow, right?
26:48
You can compose these different ideas where basically I want to
26:52
go ahead and run like just be a little bit more specific now,
26:56
forsake argument. I was using Opus 4.8 and
27:02
actually 4.7 of them implicate a real example to generate.
27:05
The code Owen better to be truthful.
27:08
Also, I'm glad I glad I have a kindred spirit here.
27:13
Right. Way more. 4.8 is not.
27:15
I'm not. Stoked.
27:16
Yeah, yeah, yeah, yeah, yeah. That's a whole other thing.
27:18
But let's not debate that, OK? I don't want to throw a kick in
27:21
my ass, OK, Right. So right.
27:23
But I'm using global force of it.
27:24
And then, but then I was using GBD 5.4 at the time to do the
27:29
test cases and fuck yeah, like it was wonderful because then I
27:35
got the 2 using Poly to debate each other in terms of high
27:38
quality coding. And then I and then I got one to
27:40
actually go ahead and and generate the test cases for me
27:43
as well, which is amazing, right?
27:46
And then maybe a week later I would have switched over to say,
27:48
you know what chat GVD 5.5. I like their code base a little
27:52
better. So I'm using that now.
27:53
I'll stick to Alpos 47 for the of the debate it I'll use maybe
27:59
sonnet to to go ahead and do the the test case because those are
28:02
relatively easier to once you put the original template.
28:05
I don't actually need something as powerful to generate the test
28:07
cases. I just need to sort of update
28:09
them. So this is perfectly fine.
28:11
And then when you know when Fable and all that stuff came
28:14
out that rewrote everything and then Fable got removed and then
28:17
we wrote everything right. The whole point is that I'm just
28:20
using Omni jets. I didn't care.
28:22
I just redirected it and like hell, I'm using π for a bunch of
28:28
my test cases now because it's cheap, like really dirt cheap.
28:31
So I'm like, why not? It's really good.
28:33
So just use that in some cases, right?
28:35
So that's most the point. Like it just gives you that
28:38
flexibility to say like, OK, you start off with everybody telling
28:42
you you're supposed to use the most advanced 1.
28:44
And I get it that you're tempted to do that.
28:48
And if you if you're allowed toka maxing, you will do that,
28:50
right? But the reality is this stuff
28:52
starting to get expensive. Right.
28:55
Prices are going up or discounts are going down or both, right.
28:58
And so my take on the whole thing's like, OK, well then try
29:02
to find the right model or the right harness for the job.
29:06
And so maybe even if you are initially using the more
29:09
advanced models to build it, a lot of the maintenance doesn't
29:12
require the most advanced model at all.
29:14
In fact, the most advanced models more times than not
29:17
actually make things literally more complicated.
29:19
Like you're going, I only needed simple test cases.
29:22
I didn't need this gargantuan thing that is impossible for me
29:26
to read. Right.
29:26
That's a feature, not a bug. That's why Anthropic's worth
29:29
it's whatever trillion dollars they are.
29:33
Very no, but see, that's what it's like in the end, you get my
29:35
trip, right? Like it's, it's about going
29:37
ahead. And then because after all, like
29:40
I'm definitely not mean to get in the debate just right now,
29:42
but like during, for example, DNA summit during the open lake
29:45
house contributors meeting, right?
29:49
One of the sessions that I actually end up helping to lead
29:52
was the deal with coding AI slump, right?
29:56
And one of the call outs we had, because the way we run open lake
30:01
house was very much in the thing of like, yeah, I might have led
30:04
the discussion, but it was meant for everybody to chat.
30:06
I'm not interested in like me telling people what to do.
30:09
That's not interesting at all. I'm we were getting all these
30:13
people in the, the open lake house and the AI community
30:15
specifically to go to debate. Like, like we're marking on
30:19
competing projects, whatever, I don't care.
30:20
But like, we all are suffering from same version of, of
30:23
insanity in one form or another. So can we help each other out on
30:26
that part at least, right. And so for example, there was a
30:31
whole debate like on the on various distribution lists,
30:36
including like Apache Iceberg on the delta list, whatever I was
30:38
like created by versus assisted by.
30:40
So created by is the person who did it assisted by is like it's
30:43
almost 47 or Sonnet or whatever it is that you were using at the
30:46
time, right? And the call out more than
30:48
anything else unequivocally was that you needed to.
30:55
In the end, everybody agrees with the idea that it is still
30:58
the individual, the user, that is the one that submitted the
31:03
code that there still is from a TLDR perspective, who's the neck
31:07
you're going to choke? It's not, I'm not choking the
31:09
neck of an agent. I'm not choking the neck of
31:11
Anthropic. I'm choking the neck of the
31:12
person that's contributing the code, right.
31:15
And so ultimately, like whatever we're phrasing as in terms of
31:19
creative biases by or anything else, the point is that me, the
31:23
developer, whether it's developing code or developing a
31:27
business project or whatever it is, you, it's still me, it's
31:31
still you, it's still the person that's making that decision that
31:33
how ultimately succeeds or fails.
31:36
So whether it's the the code for Delta rust or whether it's, you
31:40
know, figuring out which region of non telecom county that I'm I
31:44
want to grow macho leave, right. Ultimately it's like it's me,
31:48
right. The onus has to be on me alone,
31:51
right. And so knowing how to follow a
31:56
thread, knowing what these harness can do, knowing that you
31:59
can move your context, your thought processes across the
32:03
different models so that way you yourself can succeed.
32:07
Ultimately, that's it. And and I think you, Dimitra,
32:09
you and I have actually had this conversation on the side.
32:11
You've also did this as part of your own set of podcasts where
32:15
ultimately the main reason people fail when they're using
32:19
agents is because they they zero shot it and they have no context
32:24
applied at all. They there's no free thinking.
32:26
So people are like, well, of course it's garbage that came
32:29
out of it. Like, yeah, so that means these
32:30
ancients suck. I'm like, well, it's the old
32:33
garbage in, garbage out. Like if you, if you're not
32:35
putting any effort into this, it's it's going to be garbage
32:38
coming out. It doesn't matter how good the
32:39
agent is. It doesn't matter how good the
32:41
model is. Have you been playing with the
32:46
different? Because basically what I
32:48
understand is, hey, you bring your own model and you can bring
32:51
your own harness and then you have this meta harness above it
32:58
that is unopinionated. And it's like, cool, you want to
33:02
come with your open claw over here and then your Hermes.
33:05
You want to come with your Hermes?
33:06
You want to toss an integration with a llama?
33:09
Have fun at it by all means, yeah?
33:11
How have you and how have you been seeing that working in
33:15
like? Is there a world where you're
33:18
recognizing that now? What is important is
33:22
understanding what each one of these is best at.
33:28
We built homogene in about two weeks by the way, to give you
33:31
some context. All right, again, pun intended.
33:35
We literally used homogene to help build homogene, OK, like so
33:39
everything you see, we we basically used homogene itself
33:42
to basically build it, improve it the whole kin Kaboodle.
33:46
The reason I bring that up is because look we have a ton of
33:50
benchmarks right but before the two weeks we data bricks
33:56
engineering product we we were we were all suffering from the
33:59
same thing right We were suffering from like which wall
34:02
should we use this week? Should I use Kimi now?
34:04
Should I use right? Should I be looking more into
34:08
the open source ones right should like so if you like, if
34:12
you look at data bricks itself right, you'll notice that
34:14
there's a whole bunch of different models like that all
34:16
natively run within data bricks. They're all governed that's
34:19
great. I love that fact.
34:21
But then exactly to your point, I don't know which one's better.
34:25
And so, yeah, if you even go to the database website, we
34:28
actually have this thing where we update monthly, forget the
34:30
exact URL, but basically say like, here's the best ones based
34:33
on what we see this month, right?
34:35
And then we'll obviously for documentation, for debate, for
34:40
financial, like whatever, we just, we actually write down all
34:43
the steps because we're like, this is what we think in real
34:45
world scenarios. And you'll notice that the real
34:47
world scenarios often do not match benchmarks, right?
34:53
And so that's what got us to build Omni in the 1st place
34:57
because we were basically, we actually had a version of Omni
35:00
internally. We obviously can't share that
35:02
one. But the basically use that as
35:05
our starting point, mainly because we were recognizing that
35:09
every single engineer needed the ability to have that flexibility
35:12
to switch, right? Because they realized that maybe
35:17
that particular Co base that they were working on was better
35:20
with X while different team, same org, hell, same team, like
35:26
same division, everything realizing Oh no, but I'm doing
35:30
why? And it's better with this.
35:34
And so the idea of us trying to dictate you're supposed to only
35:38
use one like we were never, we're going to do that in the
35:41
1st place just because of the nature of data bricks.
35:43
But like it just the whole thing fell on his face almost
35:49
immediately. And So what we did.
35:51
The model provider could be down, which we see all the time.
35:55
Oh my gosh, yeah, yeah, yeah. Claude was just down yesterday.
35:58
So I was when I was just trying to run my stuff.
36:00
So fortunately, not a big deal. Flip it automatically flipped
36:03
over to codecs and all right, I'm back up and running.
36:05
Yippee, KA, barely, barely a blip of the whole thing, right.
36:08
But so ultimately it was more like, OK, do I have the
36:13
flexibility to change? Do I have the flexibility to, to
36:16
all, but also not to change but control my costs?
36:20
So like with the knowledge that we actually have policies put
36:23
right in there, would you say OK policies as simple as like, OK,
36:26
can I run this command? OK, that's one of them.
36:29
But like a like a shell command, right?
36:31
For example, you and I were joking about how just starting
36:33
off, you had run a skill that basically shut off your redirect
36:38
your browser, right? Well, guess what?
36:40
Like Aldrin has it built in where you have a policy that
36:42
says, no, you actually have to prove a such thing.
36:44
So it can't do that, right? But also even budgets, which is
36:47
like, OK, if this thing's going to cost you more than X dollars
36:50
or X cents or whatever it is, guess what you have to prove
36:53
before you're allowed running it, right?
36:55
So just things like that, which allow you have control over the
36:58
situation. And then the final little tidbit
37:00
is that inside of all the conversations you and I've just
37:03
had, right? The one thing is that it's a,
37:07
I'm still very like the OR a developer centric, right?
37:11
But in order to be successful more times than not, what we
37:14
have to do is we have to work with your team.
37:18
You want to share your learnings for your team.
37:20
And so that's the the coup de grace for on that, which is
37:23
like, OK, it's like a Google Doc type interface where basically
37:27
now, hey, I ran this agent or had this Polly do this
37:31
particular debate. Why don't you take a look, you
37:34
know, team leader X. And then not only can they see
37:39
what's being generated, they can then themselves run on that
37:44
session themselves because you shared it with them and they can
37:46
run the session, say, wait, can you run this particular test?
37:49
Did you take an account of X? Did you take an account of like
37:51
the new updated data fusion or whatever else that you need to
37:55
do? Right.
37:55
And then basically now that process continues on.
37:59
So you're actually sharing the workload together so the teams
38:01
can actually work together really well.
38:03
Right. Yeah, that is so cool.
38:05
And actually funny thing that you were mentioning, another one
38:09
of my little side projects, toy projects that I'm creating for
38:12
myself is a widget that goes up into the corner of your screen
38:18
and it just calculates how much money you've spent.
38:21
There you go. By day or by week.
38:24
And every time you hit another $100 increment, it goes cha
38:28
ching and confetti goes everywhere.
38:31
And this. Dude you should so do APR into
38:35
Omnagent to do that. Seriously because we already
38:37
have the basis for calculation. Do APR baby like I will use use
38:42
Omnagent to create the PR for you because that's all I've
38:44
done. Yeah, you know what else I did
38:47
though? I put a little GIF so all of a
38:50
sudden a gift pops up of like someone making it rain.
38:56
Make it so it's like 5 different random gifts that you can get
38:58
and it's like making it rain. And so you recognize, and I'm
39:02
created this with because I was at Fin OPS X which is this whole
39:07
conference around budgets and all they care about are
39:11
tokenomics. That's the whole thing now.
39:14
Nobody cares about dashboards for your clouds.
39:16
Oh God, yeah, yeah, 100. Percent, dude, we got that
39:19
figured out right. All they care about are
39:21
tokenomics and how people are spending their tokens.
39:24
And what I realized is that a lot of folks are creating
39:28
dashboards so that their CF OS or whoever in the finance
39:33
department can see how much the engineers themselves are
39:36
spending. But what I didn't see a lot of
39:39
is pushing that visibility onto the end users.
39:44
And I'm like, if I knew that I was spending 1000 bucks a day, I
39:48
might be a little bit more conscientious about the cost.
39:54
So wonderful call out like so I still remember because I'm a
39:59
long time Microsoft guy, OK, before all this.
40:01
And then when we switched to the cloud, the whole premise is that
40:06
like this idea of CapEx to OpEx, right, like we're changing cable
40:10
expenditures topics. In that process of OpEx, it was
40:13
the into DevOps, meaning that the developers actually had to
40:17
understand how much money they were spending on the
40:19
infrastructure that they're building.
40:21
Right. So if there weren't like
40:23
needlessly running a small query and spinning up a A50 node
40:27
Hadoop cluster when their laptop could have solved the problem,
40:31
right? So that's exactly what's going
40:35
on right now, which is basically that service based architecture,
40:39
SOA, you know where we've switched from this idea that we
40:44
had to go ahead and originally it was 1 central command to
40:48
understand what the budget was. And now that got pushed down to
40:52
developers because the developers could actually then
40:55
care, understand and help control that token.
40:58
Novice is the exact same thing 100.
41:00
Percent that's so true it's the same motion and you're just
41:04
we're reliving it, but it's a little bit different now because
41:08
what is funny and I think one of the hardest questions that I saw
41:12
people struggling with at the event was how do you know what
41:18
token spend is valuable versus wasteful?
41:22
Yeah and so there's all kinds of tricks that you can do on these
41:27
like these details and one of them being hey, choose a
41:32
different model. If it's not that big of or
41:34
complex of a problem, just use a better model, use an open source
41:38
model, use whatever. But still you have certain
41:41
engineers that are burning a ton of tokens and for some sub set
41:48
of those engineers that burn is worth it.
41:53
And then for another subset of those engineers that burn is not
41:58
worth it. And it's like what are they
42:00
getting done? No.
42:01
No, it's 100%. So this goes back to the idea
42:03
where it's actually just like with the switch from CapEx
42:06
topics, right? You actually need both.
42:09
You need to give the developers the tools to understand what
42:12
they're doing and what the spend is, but you also need to have
42:16
some central governance story so you can understand what's going
42:19
on right? Like so like you've, I've come
42:22
to you, we've actually had this conversations like why do we
42:24
have, you know, for example, uni catalog and why did we database
42:28
introduce uni a gateway. That's the whole point.
42:32
It's two sides of the same coin. Like homogen is there for the
42:35
developer so that they themselves understand they
42:37
themselves can apply policies. But then we have Uni a gateway
42:41
to to for centralized form, which says a look, yeah, you can
42:45
use Opus 4748, whichever one you're doing 47, right?
42:51
But wait, like for example, if you were to look at my
42:55
questions, especially in terms of when I was still beginning
42:57
with Ongen, when I was testing it out, there will be an absurd
43:00
number of which Bagel is better, Montreal versus New York, right.
43:04
And so if I was a central person, they'd be like, no, this
43:07
Denny guy, we we got. A token.
43:10
This is a ridiculous amount of tokens being spent on this
43:13
particular question. OK?
43:14
Especially because everybody knows Montreal bagels are
43:16
better. Yes, I see it.
43:17
All right, bye. But you get my drift.
43:20
Like, obviously there was a reason for what I did it, but if
43:23
I still do it today, oh, my goodness.
43:25
Yeah. They would just lock into Uniaga
43:27
anyway. No, we feel like no, you're not
43:30
allowed doing this. Shifting budget over.
43:32
Yeah, yeah. They're shifting budgets.
43:33
They're like, OK, like, no, daddy, you're allowed using pie.
43:37
You're allowed using like you're allowed.
43:40
Maybe if we're, if, if they're, if you're lucky, we'll let you
43:44
use Kimi just for the sake of doing it.
43:46
But not even a later version of Kimi, like an older version,
43:49
right? But like Opus, Oh hell no, we're
43:51
never going to let you use that, right?
43:53
So. You know what, on this idea of
43:56
choosing the models and choosing the harnesses and all of that,
44:00
where I see the biggest friction is when I have to manually make
44:07
the decision because I am a little bit lazy and I just kind
44:12
of go with whatever I've got on, you know?
44:15
And so I'm not going to, especially if it's my company's
44:19
money in a way, I'm going to be like, whatever, let's see if it.
44:22
Works. I can just blow the money and
44:24
ask the bagel. Question.
44:25
It's going to be like let's use the smallest model.
44:28
It doesn't work and then I'm going to go back and try and go
44:30
to the next smallest and cheapest and then it still
44:33
doesn't work. And then I've wasted time, which
44:35
is the whole reason I thought I was using these coding agents
44:39
was to save myself time. And so it's like I see that I
44:44
will overcompensate with bigger models just for the fear of a
44:50
smaller model. It's probably not going to be
44:52
able to get this, even though it totally will get it.
44:54
Like it's like, oh, you just changed some names in the code
44:57
base and it went through and you know, like control FT, you did
45:01
not need a 4.7 for that. Right.
45:04
So I don't think that's completely solved yet because
45:09
this is sort of like the same idea like the journey from CapEx
45:12
topics like we don't know when, when one should be applied where
45:16
like like in other words, preventing you from using it in
45:19
the first place is very limiting.
45:20
We don't want to do that, but we also don't want to get to a
45:23
point where you've blown 1,000,000 tokens for which color
45:29
of mauve is the best, OK, right. And so we're at a point where
45:36
especially with meta harnesses and especially with things like
45:38
in the AI Gateway, that now we at least have visibility and
45:43
some form of control. One of the things that we've
45:46
been experimenting on, and you're going to see some of it
45:48
show up actually in object itself because we've been
45:52
experimenting internally first because we want to see how many
45:54
engineers are going to kill us, right?
45:57
Is the idea of where do we auto choose for you based on the
46:02
questions that's being asked right?
46:04
But that's itself is hard and that itself may require us to
46:07
use an LM to make the choice. So do we just shift the token
46:10
usage that way? Right?
46:12
But this is where like a good old fashioned machine learning
46:16
may kick in for us. You know, like you know how.
46:19
RL is back in vogue again. Which is it?
46:21
Never should have not been invoked, by the way.
46:23
But you get my trip, right? RL is back in vogue.
46:25
It's like, oh, that's cheaper, That's faster, that's better.
46:29
Yeah, we've been saying this pretty much the entire time
46:32
guys. But yeah, so goes back to the
46:34
same idea. Machine learning is back in
46:35
vocal. I'm like, no, a lot of these
46:37
types of questions though, like in terms of the complexity of
46:39
the question, you can literally use good old fashioned machine
46:42
learning to make the choices. So that's what I'm saying.
46:44
We can. We're definitely seeing trends
46:45
like that to help with that process.
46:47
And ultimately, however it's done, the key thing is to your
46:52
point is like is that for example, with homogene there
46:55
isn't changing the code base to choose.
46:59
It's literally click and it automatically chooses for and
47:03
you have the the drop down list of which one to go with in the
47:06
1st place for starters. And So what we're hoping for by
47:10
the time we're done with everything is that once we get
47:14
into some good patterns around that, we actually tell you
47:18
upfront we're going to switch to Model X.
47:21
Here's the reason why and let it run.
47:25
As long as it if it fails it will retry with a better model
47:30
then I'm stoked because I don't have to spend my time being like
47:33
this. Doesn't really look that good.
47:35
Right, right, right, right. But then it goes back to like,
47:37
This is why are we like Polly so much?
47:39
And and Debbie? Because there's different
47:41
versions of what it means to say it doesn't work, if you get my
47:45
drift. Right.
47:46
So that's why we like the idea of the the agents debating each
47:50
other, because then you can walk away, let the two agents fight
47:54
each other out. And basically, yeah, because
47:58
they have enough context, they have enough memory, they can
48:01
actually figure out, OK, there's there's a reasonable assertion
48:05
here that actually because they're debating each other, you
48:08
will get a result that actually works.
48:10
Or at least even if it doesn't work, at least now you have the
48:14
basis of why it doesn't work. So it left because in the end,
48:17
especially with how powerful are these open source models are
48:20
these days? For me, a lot of time it's.
48:22
It's less about which model I'm using nowadays, and it's a lot
48:26
more about did I provide the data, the context, the
48:30
background, the configuration, the memory.
48:34
The big thing is, did I set up the correct loops?
48:37
Did I give it what it needed to just be able to run for an hour
48:42
so that I don't have to come back?
48:44
And that is one of them. I think that's a great point is
48:47
being able to debate and recognize what good looks like
48:51
so that you can trust, OK, it's a smaller, cheaper model, but I
48:55
know that the debate is going to happen and if it doesn't pass
49:01
the sniff test, then it will go back and it will loop again.
49:04
Right. And then this goes and why do we
49:07
see yet another resurgence of evals all over again for.
49:11
Precisely that we're still talking about evals.
49:13
We have to, because in the end there has to be some numerical,
49:17
There has to be a number that allows us to state if what was
49:21
created actually per evaluation actually is.
49:25
It's the old definition, what is good, it goes back to that
49:28
definition over and over again, right?
49:30
And so in the end, like I said, it's, it's becoming more and
49:33
more of an obvious that it's less about the models.
49:35
This is not a knock on any of the models.
49:37
They're all really good. Now at the point that's what I'm
49:39
just trying to get AT. And it's much more about, OK,
49:42
now that I know the models good, what else is missing?
49:46
And then the evals will actually help us figure that out.
49:49
Yeah, yeah, yeah, that. How can I make this harness top
49:54
notch exactly, 100%. I think the auto model choosing
49:58
is the way to go because if it is left on to the developer,
50:03
they're never going to choose a shitty model.
50:05
Or no, No, they're always going to choose the best model, yeah.
50:08
It's a psychological thing. You're like, I need to do this
50:13
because what if it encounter something in this run that it
50:18
needs the power? Let's just let's do it just in
50:20
case it's a few extra cents or dollars.
50:24
Depending on what, yeah, I mean, if a few extra cents, we're
50:27
probably OK with that. It's the few extra dollars $100.
50:30
And unequivocally, no, you're not wrong.
50:31
You're absolutely right. And I think like this is part of
50:36
that story for us. Like it's, that's part of the
50:38
story why we made it open source because we're going, look, we're
50:41
not pretending we have all the answers.
50:43
We're just pretending. We're just stating the fact that
50:45
this is the problem we faced internally.
50:48
We want to share this with the world.
50:49
So that way y'all can like we can all work together to get to
50:54
that solution that you're talking about, which is like,
50:56
how do we do a proper auto model, right?
51:00
How do we, how do we do a proper thing?
51:02
So that is absolutely 100% vendor agnostic.
51:06
How you know what I kind of question with because certain
51:10
harnesses have certain ways of expressing their harness, right?
51:14
And I'm thinking specifically like in Hermes and Open Claw,
51:18
you have the soul, but then in Cloud code you don't have that.
51:23
How does this meta harness of Omni agent act when you're
51:28
running these different shapes? Of harnesses.
51:32
So the the TLDR is the root you, you ruined the DS Π project
51:36
right? Yeah, right.
51:38
The root of a lot of our learnings is basic from DSPI in
51:41
the 1st place, which is the idea of how do you, how do I go ahead
51:44
and allow myself to functionally run these queries against all
51:48
these different models that actually instinct that actually
51:50
have different ways of running in the 1st place, right?
51:53
Different parameters, different weights, different everything,
51:55
right. And so with that, the TLDR is we
52:01
started looking at the systems that we're integrating with and
52:04
we tried our best to basically take an account of each one.
52:10
And so that way any integration we have basically takes an
52:13
account of like what each one is strong at at the code level,
52:17
right at the conflict level. But I would be remiss with if I
52:21
didn't also basically mention the facts.
52:23
Like that's actually why we're reaching out to a lot of these,
52:26
both the model makers and also the harness of makers, because
52:28
we're going to look, we're not pretending we got it right.
52:30
We're pretending we got a good start.
52:32
That's all we're saying, right? We'd love to work with you to
52:35
make sure we have configured it the best to represent, you know,
52:41
your particular model, your particular harness the best.
52:44
Because, yeah, we're, we we're not pretending we understand at
52:47
all. We're just simply saying that as
52:49
you can probably guess, we've used a lot of them.
52:52
Yeah. So we we've incorporated as much
52:54
of that knowledge as we possibly can.
52:56
But that still doesn't replace the fact that there are people
52:58
in these other who crave the harnesses themselves or crave
53:01
the models themselves. They would know better than us.
53:03
Fair. And again, why is an open source
53:06
project? So you get to look at the code,
53:08
tell us what we screwed up on. We want to go ahead and work
53:12
with you to figure out how to make it better.
53:13
Simple as that. You know what gives me this like
53:17
visual in my head of almost like you have your different
53:23
harnesses. And so the outer harness we
53:26
could consider like the MCP servers that I set up, maybe I
53:30
do memory in a certain way, etcetera, etcetera.
53:33
And then you've got the meta harness, which is the it's
53:39
outside of all these outer harnesses.
53:43
And that is where you have this way to almost like be an
53:48
overlord of all your agents. So like if you're looking at a a
53:55
circular thing like the inner harness and the outer.
53:57
Right. Basically the outside layer of
53:59
the onion. Yeah, yeah, yeah.
54:00
Yeah, exactly. It's it's the onion or it's the
54:03
earth with the different like rocks.
54:06
That core mantle shelf. Yeah, the magma, all of that.
54:10
So that's kind of how I I visualize it.
54:13
Is that the way that you would? Is that a good way of thinking
54:18
of it? Oh.
54:19
Absolutely. I mean, look, everything we do
54:25
business wise, development wise, whatever else, we're always
54:29
trying to figure out ways to abstract it, right?
54:32
The reason for the abstraction is because it gives us
54:35
flexibility. It gives U.S.
54:37
Open standards, it gives us the ability to go ahead and like
54:43
simplify said business logic, right?
54:45
So that way you're able to focus on the real problem, not on it
54:50
was a little problem. I mean, if if you and I were
54:52
having discussion five years ago when ChatGPT just came out
54:56
right, you and I would have definitely been debating on
55:02
things like what were the weights?
55:05
What were the hyperparameters used?
55:07
What's the? Prompt right, right.
55:10
Are we still having those debates now?
55:12
Some damn good AI researchers, of course they are, because
55:15
they're trying to improve the models, right.
55:17
But you and I, as users of these systems, we're not having that
55:21
debate anymore. We were barely qualified to have
55:23
that debate then. We're certainly not qualified to
55:25
have that debate now, right? And so, but what, what can we
55:28
do? We can talk about like how to
55:31
effectively use agents, right? Well, remember what I just said,
55:36
effectively use agents, not effectively use LMS, right?
55:38
Because we've already accepted the fact that like whether it
55:41
was called compound AI systems before or whatever else
55:44
afterwards, the reality is like the agent is basically not just
55:47
the LM, it's actually a lot of the things that are around it as
55:50
well, right? And then us saying harnesses,
55:52
yes, exactly to your point, like whether it's an MCP server, it's
55:55
ACLI, it's reading adjacent from somewhere and XML from
55:59
somewhere, hitting a database, whatever, whatever it's doing
56:02
doesn't matter, right? The point is that we're trying
56:05
to figure out how to make these things portable and stateful.
56:09
Well, guess what? That is an old fashioned
56:13
business database problem that we've been doing since the 60s,
56:18
right? Like, why are databases back?
56:22
Like why? Why do we talk about like
56:24
Serverless Postcrest? Why did Databricks talk about
56:26
Lake Base and Neon? Why?
56:28
Because literally we have to have cheap stateful memory for
56:33
these agents to run, right? As simple as that, right?
56:35
Well then if I have cheap stateful, then I'm like, OK,
56:38
what's stateful? The best thing is a database.
56:41
Cool. Why is poster sequel?
56:43
Why is there resurgence? I would argue there never was a
56:45
drop in the 1st place. That's what I would argue
56:48
actually. But irrelevant of like you want
56:50
to call resurgence or not, the premise is that I need something
56:53
that's trustworthy, that can keep state.
56:55
That's it, that, that. And so that's never left us.
56:58
And I mean, like, remember I was like former sequel server.
57:01
So like I'm like, yeah, yeah, like the database is still
57:03
there. It's no matter what we've done,
57:05
no matter how much we've talked about lake houses, which
57:09
obviously I'm a big fan of, and we're talking about agents,
57:12
which is the to the shore these days, we still need something
57:16
called state. Well, that state is databases.
57:18
And then guess what? That's if you can make it
57:21
cheaper, you're more successful. Well, that's how these agents
57:25
have run. That's how all these systems
57:27
work where we're just extracting the way and then storing what
57:31
needs to be stored at IE state. So that way you can be
57:35
effective. And so and everything when it
57:38
comes to like how we started with databases before B i.e., TL
57:44
pipelines, it's the same action plan, right?
57:48
With agents right now, which is like, how do I modularize this?
57:53
How do I make it portable? And then for a bunch of us, how
57:57
do I make it open so that way it's easy for you to move from
58:02
one to the other?