Latest / Agentic Conversations (formally mlops.community) / The Control-vs-Magic Spectrum Building Agents

Transcript

The Control-vs-Magic Spectrum Building Agents · Agentic Conversations (formally mlops.community)

0:18
You're working on iPhone Pago, which is the fintech arm of I
0:23
Food. Exactly.
0:24
So a lot of people know I food, especially in Brazil, most of
0:28
Brazilians know I food, it's a food delivery app in the
0:32
marketplace, the the market leader in Brazil.
0:37
But a few people know that we have all these other business.
0:40
One of those is our fintech business.
0:43
So our fintech business is how can we help and enable our
0:47
restaurants to grow and have more support to grow inside our
0:52
platform. So it's, it's not a bank that
0:55
lives alone in his space, but it's a bank that serves ifood in
1:01
the ifood ecosystem. So success for our bank, our
1:04
fintech is seeing our customers, our restaurants grow inside the
1:09
platform. And I'm assuming you know a lot
1:12
about the restaurants, so you can help them in different ways
1:16
by giving them loans, but also understanding their credit and
1:21
how risky the loans are. We know how the restaurants
1:24
operate. So we can we can give loans to
1:27
restaurants that probably wouldn't get a loan outside our
1:32
fintech, but we can give this loan because we know that he's a
1:35
good restaurant, that he's serving the clients, that he's
1:38
actually delivering the food and so on.
1:41
But. Why wouldn't they be able to get
1:42
it outside? Because around 80% of our
1:45
restaurants are actually small restaurants in Brazil.
1:48
So sometimes you see a single person that takes care of the
1:52
restaurant, the packaging, preparing the meal and then
1:56
taking care of the finance of the business as well.
1:58
So it's a very small business and sometimes they don't have
2:02
access to credit in in regular banks.
2:05
So what we do is since we know this is a good restaurant that
2:09
is really operating inside eye food, we can give a loan to that
2:13
person. And you can know what their
2:16
order volume is, and you can understand like, yeah, this is
2:20
sustainable. Exactly, because having working
2:23
capital is very important for a restaurant.
2:25
So they cannot grow if they don't have access to some
2:29
capital to start growing the business.
2:32
So this is very important for them.
2:35
And since we know how it operates and how it's actually
2:41
working in the platform, we can choose the right amount, the the
2:45
right size of loan for this restaurant.
2:48
Yeah. And how are you bringing the
2:50
agentic experience into that whole life cycle?
2:54
Yeah, we have a lot of different agents inside Pago in that life
3:00
cycle. Specifically, we have chat bank,
3:03
which, which is like the bank manager for the, for those
3:07
restaurants. And it's a a different kind of
3:10
bank manager because it's not only saying things about like
3:16
your, your balance or general information about your account,
3:23
but it actually helps the entrepreneur to run the
3:26
restaurant. So it says, OK, you need to pay
3:28
your employees. The, the, the entrepreneur can
3:32
say, I want to play my employees.
3:34
I have these five employees and those are the accounts.
3:39
And then the, the bank manager can do the, the bank transfers.
3:44
It also can and. The bank.
3:46
So me as a restaurant owner, I keep all my money in the bank
3:50
and I'm using ifood as my bank as opposed to whatever like ING
3:57
or N 26 type of thing. Different people choose
4:01
different ways of working, but a lot of our restaurants use our
4:05
account because of all these different features that we have
4:09
for. Them it makes it a lot easier I
4:10
can imagine. Yes, exactly.
4:12
So this chat bank assistant the the bank manager, it helps the
4:18
the the restaurant owner with the day-to-day life of the
4:22
restaurant. So making payments, knowing that
4:25
he has an outstanding invoice that he needs to pay.
4:29
And this also is an opportunity for helping him with the working
4:33
capital as well. So for example, we can say we
4:36
see that you have an outstanding payment, but you don't have the
4:39
money in your account right now. But I, I see that you have
4:43
access to credit like you have this three different sources of
4:46
credit you can use. But I'm seeing that the cheapest
4:49
1 is this one. So you can access the cheapest
4:53
possible credit just to solve your problem right now in the in
4:59
the quickest and cheapest way possible.
5:01
And this is all done through a chat interface.
5:03
Yes, exactly. WhatsApp is very popular in
5:06
Brazil and the restaurant owners are usually operating in
5:11
WhatsApp as well. So he's talking to suppliers,
5:14
he's talking to customers in WhatsApp.
5:16
So helping him in WhatsApp works really well because it's it's
5:21
very comfortable for him. So they, I imagine a world where
5:24
they can just forward invoices to chat, Bing, WhatsApp, and
5:29
then it's automatically uploaded, Yeah.
5:31
Exactly. He actually does that.
5:32
Yeah. Yeah.
5:33
He can even take a picture of something and say I want to make
5:37
a pix for this number. Pix is the Brazilian Trans Money
5:41
Transfer Protocol, one of the the protocols.
5:44
So I can say I just want to transfer money for this account
5:47
and take a picture and then the Chet Bank takes care of the
5:50
rest. Yeah.
5:51
It's like pay this shit, I don't want to deal with it Exactly and
5:56
it will transfer the funds. It's just I send a WhatsApp
6:01
message. Yes, and then you make the
6:04
transfer. It's, of course, it's a secure
6:07
procedure, so the the restaurant owner has to tap his password,
6:11
but everything is very well integrated, like the user
6:14
experience is very seamless for him.
6:18
So he just types the password and then the transfer is made.
6:21
Wow, yeah. Behind the scenes, what's it
6:24
look like? Behind the scenes we have an
6:27
agent there that for every message the user sends it reacts
6:32
to it. So it it receives the message
6:35
creates an execution plan. So what should I do for the this
6:39
message? So for example, I need to start
6:44
a money transfer operation. So he calls this function and
6:49
then this function becomes UI element inside WhatsApp for the
6:56
user to confirm the password. And then the after the
6:59
password's confirmed, then the transfer is created in the back
7:02
end. So it's the agent works like a
7:06
front end, like a web front end. And you have all these tools in
7:10
the back end that interface with the different APIs we have
7:13
inside the the fintech. So we have APIs for the money
7:19
transfer, we have APIs for getting the account balance for
7:24
authentication. We need to know if a WhatsApp
7:28
number is attached to a fintech account, what account is that
7:32
account. So these are all the different
7:35
back end functions that we need and and we have other agents and
7:41
other business as well. So inside our Fintech we have
7:45
beneficious which is a corporate meal voucher that is very common
7:49
in Brazil. It's like a credit card but only
7:52
for meals. And we have an agent that helps
7:58
small companies hire this the service and also operate the
8:02
service in a day-to-day basis. And hire the service?
8:07
You mean like go and get you a credit card?
8:12
Not for the person, but the company's HR.
8:16
So I'm from a company HR and I go there and I want to hire this
8:20
beneficious for all my employees and have this credit card for
8:24
all my employees. And the thing with the agents
8:26
here is that we can scale a very custom experience for this
8:33
person. So we can say, OK, what's the
8:35
size of your company? What are your needs for
8:40
beneficious for the the meal voucher?
8:43
What are you worried about? And sometimes people ask what's
8:47
the difference between this product here and this other
8:50
product? And then we can help the user
8:54
guide his decision and and make the best decision.
8:57
So this is one of the ways we use these agents.
8:59
Yeah, talk to me more about this idea of scaling personalization.
9:04
Yeah, because I think that's something fascinating that you
9:07
all are doing and it is mainly happening right now through
9:13
chat, right? Yeah.
9:15
But what are your thoughts around it?
9:18
Yeah. So for me, when I started using
9:22
AI, when I started working with AI, the the interesting thing
9:27
for me is that you can bring a very customized experience to a
9:33
whole new level. You can like scale that to a
9:37
huge population. So for example, when you look at
9:41
ifood, not only the fintech part, but but the whole ifood
9:46
company, you have a lot of opportunities that you want to
9:48
be more tailored to the customers.
9:53
So when you're ordering food, I want to offer you what's good
9:56
for you. When I'm trying to help you find
10:01
and hire beneficials, I want to clarify for you and make easier
10:06
for you to understand how beneficious can help you in your
10:09
company, not someone else's company.
10:12
And with agents, we can bring that to everyone so you can talk
10:18
to the person in their own context.
10:21
So it's way easier to do that. And today this is, this mostly
10:25
happens by chat because it's a very open-ended interface.
10:30
It's, it's kind of easy to put everywhere, but what I'm seeing
10:34
right now is more and more we're starting to creating like custom
10:40
UIS that are more tailored to the specific cases.
10:43
So for example, we, we have a case in ILO, which is our food
10:49
recommendation agent that having the the right UI for the user
10:55
not only saves time because he can just tap the dish that he
10:59
wants, but it also saves a lot of processing in the background
11:02
because reduce like LLM calls. Yeah.
11:05
So it's not only about having the right experience, which is
11:09
easier and and and nicer to use, but it's also about efficiency
11:15
as well. Dude that is so true.
11:17
The tightrope that you have to walk to make sure that you're
11:23
not just sending everything to the LLM, It's that fine line and
11:27
you get to be almost an artist about where you throw in the LLM
11:34
and where you don't. Yes, as the models improves,
11:38
it's getting easier because it's not that sensitive to your,
11:44
your, the, the way you write your prompt and everything.
11:47
But there's still some magic to it, like maybe it's one word
11:51
that you put that changes how the agent works.
11:55
So how you evaluate that and and how you choose what you put
12:00
inside the context is it's very important.
12:04
Yeah. And just thinking about where
12:06
the human plays the role, where do you actually have the human
12:13
be part of the process versus not, is also fascinating in my
12:18
mind. Because I imagine you're
12:21
thinking a lot about how to get the merchant the least friction
12:28
way possible to do something. So for you, anywhere that the
12:33
human has to do something is it's like it better be a must
12:39
that they have to actually do it, because otherwise you're not
12:43
saving them as much time. But but at the same time, you
12:47
have to build trust as well. So when you use an agent and it
12:52
starts doing everything for you, maybe you can get uncomfortable
12:55
like I don't want you to do that or just wait.
12:58
You need my approval to do that. So it's it's a fine line you
13:02
have to walk between giving the automation and making the person
13:07
comfortable with the decisions that are being taken.
13:11
Well, especially with money. Yeah, exactly.
13:14
So it's it's a very sensitive topic, Yeah.
13:16
Yeah, you're, I, I mean, if the wrong pizza shows up at my door,
13:21
that's one thing, But if you send my employees an extra 2
13:25
grand, I'm gonna. That's a totally different
13:29
problem to have. And if it's the agent's mistake
13:31
on both of those parts, like 1 is a little excusable.
13:35
The other is like, hey, I want that 2 grand back, but I'm not
13:39
going to not give it back to my like it's just a messy situation
13:45
that you're in. Yeah, exactly.
13:47
The the harness we build for different types of products and
13:51
applications is completely different.
13:53
Like I, I, I like to put agents and AI in a spectrum, like in in
13:59
one end, you have a lot of control and in on the other end,
14:04
you have a lot of magic. So you have to choose where
14:07
you're going to put your product in this spectrum because
14:10
sometimes it will feel a bit more stiff, but you want that
14:14
because you need more control. And sometimes the the risks are
14:18
low. So you want to see more magic.
14:20
You want things to happen and and people to feel that wow
14:23
moment when using your product. Are there moments where you've
14:27
just let it go completely to the magic side of that spectrum?
14:31
Yeah. I don't think that in fintech
14:33
business we have that, but in I food in ILO, for example, for
14:40
recommending food, yes, it we want it to be a magical
14:44
experience. So instead of saying I just want
14:47
something that I like or I, I just want the pizza here, we
14:51
want people to ask what should I get for a romantic dinner with
14:56
my wife? So this is a different level of
15:02
openness that we want from our users.
15:04
You talked a little bit about how the chat interface is
15:08
changing and how we're starting to create these new components.
15:12
I'm also fascinated by that. I've seen there's a, there's a
15:17
project that I saw on Figma that was, I can't, I think it's
15:22
called Language GUI and it's a bunch of Figma components that
15:27
you can use inside the chat to help render different things
15:33
that you wouldn't necessarily get in a chat.
15:35
Maybe that's an embedded tweet or it's a voice note or it's a
15:39
snippet of a podcast, so that instead of always getting back
15:45
text, I get different, more enriched data points.
15:51
And now we're seeing that a ton with the MCPUI, which is that
15:56
taking to the extreme. And I'm excited to see what
15:58
people build with it. I know that some people in the
16:00
MO OPS community, Slack, we're talking about how hard it was to
16:04
work with MCPUI right now, just because it's so nascent and you
16:08
kind of expect that. But are you thinking through
16:13
different components that you're now rendering?
16:17
You talked about how you tap for authorization, so that's one.
16:22
What else is there that you're like trying to bring into the
16:26
chat that you don't have to have the user leave for any reason?
16:29
Those are usually very specific and tailor made for each case.
16:35
Of course you have like a generic, a genetic UI toolkit
16:39
that can be used. But I think the the real
16:45
experience it, it becomes really good when you tailor the
16:49
components exactly for the use case that we have and what the
16:52
user needs. So it's, it's not about the only
16:56
about the agentic part, the AI, but it's you, it's the how the
17:02
the customer is using your product in a day-to-day basis in
17:06
the experience that he has. So those special components are
17:10
very important. I think we have this for
17:13
passwords, for confirming information with the user to
17:18
make sure that he's aware of what the agent is doing and
17:21
what's happening. So this is something we are
17:23
experimenting about. But when I think of agentic UI
17:29
components and so on, I also see that spectrum from control to
17:33
magic. Like on one side you have all
17:36
these agentic UI toolkits popping up right now with
17:40
different CSSHTML or React components already packed that
17:45
the agent can choose. And on the other side of the
17:48
spectrum you have like free HTML with JavaScript and CSS with
17:54
which also is working really well nowadays.
17:58
So for for some internal tools we are actually doing that a
18:02
lot. So generate on the fly the HTML
18:06
with this visualization for this specific case.
18:10
Yeah, that's kind of like the playgrounds that I was showing
18:13
you. Yeah.
18:14
Exactly, Yeah, like the playgrounds you were showing,
18:18
for example, we had one one problem in a in a, in a data set
18:25
that we had and we wanted to investigate in the book what was
18:29
happening. And we created a simple tool
18:31
like called code created this tool.
18:33
I want to investigate this, this and that.
18:36
I want to see the lineage of the data set and why this happened.
18:40
And it just generates the HTML with the visualization, the
18:44
graph with the data sets being built.
18:46
And it was very easy to see why the problem happened.
18:49
Yeah, I, I've, that's one of my primary use cases for this
18:54
playground skill is debugging, Yeah.
18:57
And getting visual representations and then being
19:00
able to hone in on what the problem is and where it's
19:04
happening. And then you can just copy over
19:06
the prompt and be like alright fix that for me please.
19:09
Yeah, exactly. It's a really cool application.
19:12
When you're talking about these different components that you're
19:16
serving up to customers, do you feel like that is also
19:19
personalization at scales? So now I'm going to get on the
19:23
app and it knows the top four things that I normally do when I
19:27
talk to it on WhatsApp or whatever.
19:30
So it instantly says, hey, welcome back.
19:32
Here's do you want to do any of these, like quick buttons, you
19:36
know, so I don't even have to think.
19:37
I'm talking more about context in general.
19:41
That affects everything. So for example, I know that you
19:44
have two employees. I know that you're not opening
19:48
your restaurant tomorrow. Those are the things in context
19:51
that start summing up to give a nice experience for the user.
19:56
So today I'm not going to say for I'm not going to send you a
19:59
message for you to pay your employees because I know you're
20:03
not opening today or something like that.
20:05
So this context enriches every experience.
20:10
And and one being the the buttons I'm going to show in
20:14
what order and what's the user going to use first?
20:17
Like what's the most frequently used?
20:20
Yeah. Right now, behind the scenes to
20:24
create that context. I know that isn't the easiest
20:28
problem. How are you going about making
20:31
sure that you have all of the right context at the right time?
20:36
Yeah. I think it's something that is
20:37
more of a emergent behavior because you have different
20:42
systems. Some are pretty simple but bring
20:45
this feeling and some are pretty complex.
20:49
In in one end of the spectrum, the complex part, we have LCM,
20:52
which is our own custom LLM tailored for e-commerce data
20:58
that has the profiles that can build profiles for the user and
21:02
say Thiago really likes Japanese food and during the weekends he
21:08
likes to eat pizza as well and so on.
21:10
So this is LCM that brings a whole new level of
21:14
personalization for our customers in I food.
21:18
But on the other end of the spectrum, maybe just counting
21:22
how frequently you you use every feature and use that to sort the
21:27
the menu is enough. So I think this is more of a
21:31
emergent behavior that when we start assembling all the pieces
21:35
and putting everything together, it seems like it's pretty
21:39
complex, but some components are complex, some are very simple
21:44
and they when they are all aligned, they bring this this
21:47
feeling. Is there stuff with like your
21:53
context engineering that is the low hanging fruit?
21:57
Usually it's the the basic profile that you have on the
22:01
restaurant or the person and so on.
22:04
So for example, I'll give an example with the the customer
22:08
facing part because I think it's easier to visualize.
22:11
But if I have my last 10 orders inside my context, it's pretty
22:16
easy to see what I like eating. So for the restaurants, this can
22:21
be the, the, the way it operates.
22:24
So where it is located, what's the opening hours and what's the
22:31
cuisine, all these things can help creating this experience
22:37
for the restaurant. So for example, we have a
22:39
product which is I food shop that sells supplies for
22:43
restaurants. And knowing the cuisine is and
22:46
the dishes that the restaurant serves is the information we use
22:51
to recommend what products he can buy.
22:54
So for example, if it's a pizza restaurant, I'm going to sell,
22:58
for example, cheese because I know it's something that he uses
23:01
a lot. Yeah, you're not going to
23:02
recommend chopsticks. Yes, exactly.
23:05
Yeah, it feels like there is a law of diminishing returns with
23:11
the context that you give it versus what actually is the most
23:15
valuable in that output and what your your end user is
23:19
experiencing. Depends a lot on the task you're
23:22
actually trying to solve. So when when I'm building
23:28
agents, I see that if the technology was perfect and the
23:32
models were, were perfect, we would we would have only one
23:38
single agent agentic prompt that says everything the agent has to
23:43
do with all the guides and all the information and all the
23:47
tools. And it would have access to like
23:49
100 tools and everything would be perfect.
23:52
But the reality is not like that.
23:53
As you increase the prompt size, as you increase the number of
23:56
different instructions the agents has to follow, as you add
24:00
more tools, it gets harder and harder for the agent to to have
24:05
a high accuracy to do the right thing at the right time.
24:08
So what we see is that as the complexity grows, you start
24:13
breaking this agent in sub graphs of agents that that
24:20
solves a specific problem. So for example, this agent
24:23
starts really complex. I see that when I'm talking
24:27
about payments for the employees, I, I should have a
24:31
special care when dealing with this part.
24:34
So I've created a different prompt of a different context
24:38
with different tools for the agent to handle that specific
24:41
case. So when we look at the context,
24:44
for me this is very important, what we're trying to solve.
24:46
And this context is going to change given the task we're
24:51
trying to solve. Do you feel like there is ever a
24:54
point where you're just going to have sub agent bloat because
24:59
you've created so many sub agents now that the at some
25:04
point there's got to be that bottleneck, right?
25:08
And so if Or is it just layers of abstraction?
25:13
I see more as layers of abstraction.
25:16
It's like breaking a complex problem in smaller problems.
25:20
So it's usually easier to handle that way.
25:23
So I haven't reached a point of a sub agent bloat or something
25:27
like that. And I don't even like to think a
25:29
lot as sub agents because when we're building an agent, I see
25:35
more like a graph like I have these nodes and this node
25:39
execute a specific operation and it can go to other nodes and so
25:43
on. So a sub agent can be a sub
25:47
graph can be a set of different nodes that solve a task.
25:51
And it's a very seamless thing is it's not something like you,
25:57
you say, oh, I'm going to have those 3 sub agents.
26:00
It's more something like grows into 3 sub agents.
26:04
So when I'm building a single product, I think more of an
26:10
agent that starts to be assembled by different pieces
26:14
and start to being created by different pieces.
26:18
When I think of sub agents, I I think in something more similar
26:22
to like micro services. Like I have two different teams.
26:25
One team builds one specific agent to solve a task.
26:28
I have another team building an agent that handles another part
26:32
of the business and sometimes one will need to call the other
26:35
and then I have this extra change of information.
26:39
Tell me more about how it grows. If we start with this single
26:43
node, that solves everything. This is the way I think all
26:47
agents should start. You always start with the
26:49
simplest approach possible, which is a single node.
26:54
We have access to tools, and then it can choose what to do.
26:59
And you have this set of evaluations that for every case
27:03
that you have, we evaluate what's the accuracy of your
27:06
agent. As you progress and as you get
27:10
more data and you start expanding that evaluation set
27:14
and you start having real users with your agent, then things
27:20
start not working so well. And you start to detect which
27:23
points are not working really well.
27:25
So when the user tries to do action A, it's usually the, the,
27:30
the agent usually messes up what he's trying to do.
27:34
So here's something that we should break.
27:36
So you go to your evals, you're going to see that in your eval,
27:40
this specific case is not performing really well.
27:43
And then you break that node in two nodes.
27:46
And now I have one node that takes care of only this specific
27:49
case. And then you go back to your
27:51
evals and see how it's working. Now everything's fine.
27:54
We, we keep working with this agent.
27:58
As you progress, as you get more data, people use it more than
28:01
you're going to identify another use case that's not working
28:04
well. And then you do the same thing.
28:06
Now I have to break this specific node in two nodes and
28:10
so on. So it's more of a iterative
28:12
process. It's not something that you, I,
28:15
well, I believe it's not something you can plan in
28:18
advance. So I'm going to have this agent,
28:20
I'm going to have all this 10 nodes and it's going to be the
28:24
best architecture possible. I think the architecture is
28:27
being discovered as you gain knowledge of the problem.
28:31
Yeah, you don't try and boil the ocean with that first agent.
28:34
Exactly. You see what it's doing right,
28:37
what it's doing wrong, and then you try and create those edge
28:40
cases and continue to have that feedback loop.
28:43
I I like that vision. Now I understand why you say it
28:48
grows because it's that iterative loop and you water
28:52
your little plant by giving it more evals and looking at it
28:57
again, seeing where we can break it up.
28:59
OK, now there's a new branch. Now it's sprouted and and this
29:04
side of the tree is flowering. Cool, let's go there.
29:08
Let's water that a little bit more and give it more eval sets.
29:11
Yeah, in in my opinion, complexity should never be
29:15
planned. It it should be a result of of
29:20
time. It's not something that you
29:22
prepare. You're going to start complex.
29:24
You have to reach that point after a lot of time of work.
29:29
And and for me, what makes a very effective team at building
29:34
agents is actually running this feedback loop really quick.
29:39
So if you can get new instance for your evals, run your evals,
29:44
detect what's what's wrong, grow in your agent and doing that
29:50
loop really quick, those are the teams that are winners and those
29:54
are the agents that are winners as well.
29:56
I've never met an engineer that told me I wish this system was
30:02
more complex. If anything you try and go the
30:06
other way and like you said, you want to try and keep it as
30:09
simple as possible. But that feedback loop, you
30:12
allow it to enrich the experience and complexity comes
30:17
from that. But that's only when you're
30:20
faced with the reality of the world.
30:22
Exactly. It guides you when to add
30:25
complexity and where. So I know you had hot takes
30:28
yesterday on Claudebot, which is Moldbot, which is open Claude
30:33
now. I really like open claw.
30:36
It's something that it's, it's a game changer.
30:41
It, it doesn't bring a lot of new techniques or anything like
30:46
that, but it's a very good interface.
30:50
Like you, you have it 24/7 by your side, answering your
30:55
questions using the tools. And the, the, the thing that is
30:59
different from open Claw is that it actually develops itself.
31:03
So for me, this is the the biggest thing for open Claw
31:08
because I can say I wanted to create a new SKU to search the
31:12
Internet for me and do this. So for I'll give you an example
31:17
for my open claw, I asked it to create a new SKU that researches
31:23
the Internet and Twitter and see what are the trending topics for
31:27
the day. And then it reflects in the art
31:33
image to create and it has its consistent style.
31:36
So it has like a book of styles that it can use.
31:40
And then he chooses for this reflection he's creating for
31:43
this specific day, what is the style he's going to use.
31:48
And then he generates the description for an image and
31:50
then generates an image with this reflection that represents
31:54
this reflection. So this is something I created
31:57
there. And, and this is not trivial to
31:59
do because he had to create a specific script to use the X API
32:05
to fetch training data. He had to get data from the
32:08
Internet as well, then summarize this, and then he had to create
32:13
a script to call the image generation API and so on.
32:18
And it's all nicely packaged and I only had to talk to him to
32:22
create that. So this self development process
32:25
is what makes open class so interesting in my opinion.
32:30
But at the same time, all this complexity and all this freedom
32:36
for it to create also brings a lot of security risks as well.
32:40
So Open Claw expands a lot on the attack surface that we have
32:47
not only by installing software and downloading software and and
32:52
doing getting skills from somewhere on the Internet and so
32:56
on, which is 1 attack surface. But also when you look at prompt
33:01
injection and other problems that can arise from processing
33:05
the the textual information as well.
33:07
And for the prompt injection, it's not enough to control the
33:13
entry point. Like for example, if you don't
33:15
talk to my open cloud, then I'm safe.
33:17
This is not true because maybe you can get a calendar invite
33:21
that has a prompt injection, or maybe someone sends you a
33:23
message in WhatsApp with a prompt injection, or you receive
33:26
an e-mail with a prompt injection.
33:28
Yeah, because it's monitoring all of it, Yes, so.
33:32
The the amount of excess you give, it's also the amount of
33:38
risk you're taking. You had mentioned too before the
33:41
idea of how much your job has changed.
33:45
As you were talking about what you've created with Open Cloud,
33:49
I was like, oh, so basically what you were doing in 2009, you
33:53
basically can put together in 20 minutes now with Open Cloud and
33:58
probably more. Can you talk me through now like
34:02
how you think about working through the all those changes
34:07
that you've had over the years? Yeah, I think that the the cost
34:10
of software of writing code, it's already a fraction of what
34:16
it was a few years ago. We're still in the adaption
34:20
period in in which people are actually understand how to use
34:24
and how to make it work. But one example for my personal
34:28
experience as well, it's been easier sometimes to write a tool
34:33
to solve a problem than to find a suitable candidate in in
34:36
Google. So for example, I have one
34:39
website that I put all the tools, small tools that I code.
34:44
One example, I sometimes will create a presentation deck, some
34:49
slides and I want to put a video converted as a GIF.
34:53
So I want to get a video convert to GIF and put in my
34:56
presentation and I always used to search for a website and I
35:00
always was thinking if this website is storing my video
35:04
there or not and everything which it is.
35:06
Yeah. And what I did was I opened 1A
35:11
genetic programming tool and I prompted to create.
35:16
I want this small tool only using HTML and JavaScript.
35:20
So it runs on the browser and you're going to use FFM peg in
35:25
web assembly to convert this video to GIF and then it created
35:29
like one shot the the application.
35:32
So I have this ripple that I put all those tools and I have it
35:36
online and for the specific case, it's easier to do that
35:41
than to find a suitable candidate.
35:44
So right now this is getting very common and as the models
35:48
progress and get better, it's going to be more and more common
35:54
to see that happen. Actually, you are bringing up
35:56
such a good point. Yesterday I saw a tweet from
35:59
Andre Caparti. Then he said that he trained a G
36:04
PT2 model. It was capable the same stuff
36:10
for under 100 bucks. Yeah, that is so crazy because
36:14
you think about how much it cost to train back in whatever 2019
36:18
when they first did 2017 when they were doing it till now.
36:22
And so not only is software going trending towards being 0
36:28
price, but the models themselves are trending towards 0.
36:33
It is 7 years later, but still man, if you can train it for
36:38
under 100 bucks, in seven years from now, you're telling me that
36:42
we're going to be able to train Kodak's 5.2 for under 100 bucks?
36:46
You have access to a huge amount of intelligence for a very small
36:51
price. And when you look at code in
36:53
this scenario, it's it's so cheap that you're going to get
36:58
software for free. And what happens when we live in
37:01
the world that software is free. So my, my personal take is that
37:07
it's going to increase a lot of demand for software because it,
37:11
it's now easier to build. So we want to build more stuff.
37:17
So for example, you said, I'm going to create a small tool
37:22
here just to debug a problem that I have or just to
37:25
experiment with a different character I want to put in my
37:29
game. And and this is not something
37:31
that you used to do. Even the game itself.
37:34
I'm making a game for my daughter, and I would have never
37:37
thought I'm going to code a game so that my daughter can play.
37:42
I would have gone to Google and I would have found a game and it
37:45
probably would fit a little bit. But it goes back to that
37:48
personalization at scale, and now I can have our own game that
37:54
I choose the character names, I choose how they look and what
37:56
they're doing and what it's related to.
37:59
This one specifically around geometry.
38:01
And it has horses because my daughter loves that, right?
38:04
That's the personalization for me and of one, well, and my
38:08
daughter. So I guess I have two.
38:10
Yeah. And, and I believe that everyone
38:12
will have his or hers 2 belt of programs that you're going to
38:18
build and sometimes those will be disposable.
38:20
So I create a, a program just to solve this specific case.
38:25
And it's so cheap and fast that OK, I can create, I solve my
38:28
problem and I'll never touch the software again.
38:30
Yeah. Yeah, and you'll want the
38:33
novelty, like if I had this horsey game for my daughter, if
38:38
it was faster and easier to create.
38:41
Because right now I'm spending whatever a weekend on it or a
38:45
week. If it was easier to create, then
38:51
I would probably make more games I like.
38:54
Today I'm playing the horsey game that's revolving around
38:57
geometry. Tomorrow I'm going to play the
39:01
dog game that revolves around whatever vocabulary expansion or
39:05
something like that to try and help my daughter's learning
39:09
journey in that regard. And then you augment all of the
39:15
different disposable things you're you're creating because
39:18
it is so fast and easy to build. Yeah, I, I saw one case recently
39:22
that one guy had his MRI scan and there's only one software or
39:29
a few software that can do that and it, and it runs on Windows.
39:33
And he said, I want to, to see my files and I want to see in my
39:39
Mac or Linux or whatever. So he went there to code code
39:43
and Vibe coded his MRI file visualizer that runs on the
39:49
browser. So this is something crazy.
39:51
Yeah, wouldn't happen like a few years ago.
39:53
Wow, you at ifood. I'm sure in your current
39:57
position you're using a lot of software still that is like
40:01
you're and you're probably paying for a lot of software.
40:05
Are you going to just stop paying for it?
40:09
No, I, I I don't think so. I think when you build a company
40:14
around software is not only the code, like you have a lot of
40:18
things happening around the code that are very important.
40:22
You have the operation, we have support, you have a lot of other
40:26
things happening there. And so I, I think this is not
40:31
going to happen. It's not going to stop, but
40:35
maybe we'll want to build more stuff that previously we
40:39
wouldn't like to build. So for example, I have this back
40:43
office thing, maybe I'm using external provider today.
40:47
But now the cost to build a similar software is so small.
40:52
And and if we had a customized, a custom made one would be so
40:57
much better for us that now it makes sense to actually put the
41:01
the team, the time and the effort to build that because the
41:05
cost has decreased. So I think that that could
41:07
happen, yeah. Yeah, with the envelopes
41:09
community, we were able to save a ton of money because we vibe
41:14
coded a newsletter platform. We were spending like, because
41:19
once you get to a certain amount of subscribers, you're paying
41:23
per subscriber every time you send a newsletter.
41:27
And well, each platform has different cost structures, but
41:31
the platform we were using, we're paying up the ass for it.
41:35
And it was like, well, you know what, maybe we can try and vibe
41:40
code something so that we don't have to pay 1000 bucks a month.
41:44
And it totally worked. And so now we saved 12 grand.
41:49
Yeah. Yeah, the the, the word already
41:54
changed like in in, In my opinion, what we're seeing is
41:57
that we completely moved the abstraction level that we work
42:02
with code. So right now we're not thinking.
42:05
It's the same thing that happened when we moved from
42:08
assembly to Fortran. For example, this is imagine
42:13
John Bacchus arriving and saying, hey, I created this new
42:16
programming language. It's really cool.
42:18
It's called Fortran and it's easier to program a computer in
42:22
that I imagine that people would say, oh, this is, this is not
42:26
that good because if I call in assembly, I can choose every
42:30
register, I can make every decision, I can choose every
42:34
instruction the computer is using.
42:36
And and he say, yes, but this is a higher level of abstraction
42:40
and so on. And this is something that
42:42
completely took the word like right now, most of programming
42:45
happens in the high level programming language.
42:48
And with those AI tools, what I see is that we're giving another
42:52
hop in this abstraction level ladder.
42:56
So we're working in a new higher level of abstraction.
43:00
It's the it's the kind of the same shift that we had before.