Latest / Agentic Conversations (formally mlops.community) / Developers May Stop Depending on Libraries

Transcript

Developers May Stop Depending on Libraries · Agentic Conversations (formally mlops.community)

0:00
Models have got so powerful, the a lot of these kind of open
0:03
source libraries, the way I describe is they're kind of
0:05
liquefied. When MCP was launched, we didn't
0:07
have clawed code, we didn't have goose, we didn't have fast
0:10
agent. And you know, we would bring an
0:12
MCP server set of tools to A to a model and we would prompt it
0:15
and we'd kind of be delighted if it did one or two calls.
0:19
Yeah. But the consequence of this has
0:21
been incredibly important. Sean Smith is a software
0:24
engineer, open source advocate, and one of the most influential
0:27
contributors to the rapidly growing MCP ecosystem at Hugging
0:31
Face. He's helped to create tools used
0:33
by developers around the world to build more capable AI
0:36
systems, including the Hugging Face MCP Server and Fast Agent.
0:41
His work focuses on making advanced AI more accessible,
0:45
interoperable, and useful for the next generation of builders.
0:49
It is wild when you're here. Most folks say yeah I haven't
0:54
written a line of code in a while now.
0:57
Ideas have become more important than just writing code.
1:01
Pretty much anyone can build an LLM app in a few lines.
1:03
In a few lines of code it is. In an afternoon session.
1:07
Wow. So that's kind of incredibly,
1:09
incredibly Valve. Yeah, it's like a small door
1:11
that opens up into a mansion. It's.
1:13
Absolutely extraordinary. Yeah, if you migrate your code
1:16
to Rust and then migrate it to another language, it's a cheat
1:20
code because you get better performance.
1:22
Since it's so opinionated, you really have to make it work with
1:26
Rust. It's like New York.
1:28
If you can make it with Rust, you can make it anywhere.
1:41
When I think open source models these days, I mean, I know Gemma
1:44
Ford just came out, but Quinn models are incredible.
1:48
Yeah, you've got the Min Max models.
1:50
There's a lot of really good options that aren't necessarily
1:53
homegrown American. Yeah, I know Reflections trying
1:56
to make an open source one. So far they haven't come out
1:59
with anything, so yeah, more patiently waiting.
2:02
Yeah, and Allen AI also is another instituted.
2:07
And they're doing great stuff. Yeah, but it's, it's less
2:10
directly competitive like 4B models and large language
2:14
models, right? It's more they do a lot of what
2:17
I would consider creative. Maybe creative is not the right
2:21
word, just like different types of models.
2:26
So research. You know, I think it's an
2:30
important part of progressing the industry that people
2:32
actually try different approaches, produce research
2:35
models and share those results with the public.
2:38
And it's always the case that while a model may not be useful
2:43
for general purpose, it may be the case that it's a particular
2:46
property of a model is useful for a specific for a specific
2:50
use case, well for other researchers to build on.
2:52
Yeah, and that's the beauty of research out in the open.
2:55
Precisely. You used to get a lot more
2:59
companies sharing how they trained their models I feel like
3:04
and less and less now. I remember the Llama 3 paper
3:07
came out and that was great to see.
3:09
Even Llama 4 was really good. That is an open model still, but
3:13
you saw like Open AI was doing it a little bit.
3:17
Yeah, Open AI were very, very generous with GPTOSS.
3:21
And again, I think, you know, it is worth in this conversation
3:24
kind of drawing distinction between an open weight model
3:28
where the weights of the model and so that you can run
3:31
inference with it is what is shared and something which is
3:34
fully open source, which would be the entire training pipeline
3:38
and environments and data sets that went into the model.
3:42
So there is a distinction between those two things.
3:44
And obviously what we're talking about is open weight models and
3:48
not necessarily fully open source pipe.
3:50
Yeah, the fully open source is very hard to do right,
3:54
especially if you're worried about somebody suing you for the
3:59
data that you used. Potentially that's one aspect to
4:04
it. The other thing is the amount
4:08
of, you know, organization effort and time it would take to
4:11
also do that in a reproducible way.
4:14
But, but certainly, you know, we see a lot of a lot of very
4:19
useful open data sets. And to your point about the
4:22
research, that's also critically, critically important
4:26
because what's happening in post training in, in the kind of
4:29
things that end users would be able to do with fine tuning and,
4:34
and customisation is, you know, massively opened up by people
4:38
sharing their reinforcement learning environments and
4:42
obviously a huddle face. We have libraries like, you
4:44
know, TRL that help people customize models themselves.
4:49
So so the publishing of research and the making accessible so
4:53
that our users can can implement it and take these ideas.
4:58
And one of them is. How much have you been going
5:01
down the rabbit hole of the RL environments?
5:04
A reasonable amount I referenced.
5:06
I referenced this in my talk actually.
5:08
So I gave a talk at this in this conference and one of the kind
5:12
of opening things was that so in terms of like a rabbit hole, not
5:17
so much. But the consequence of this has
5:19
been incredibly important because if we kind of think
5:22
about when MCP was launched, we didn't have clawed code, we
5:26
didn't have goose, we didn't have fast agent.
5:30
And you know, we would bring an MCP server set of tools to a, to
5:35
a model and we would prompt it. And we kind of, I'd be delighted
5:39
if it did one or two calls another time.
5:41
The Sonic models were specifically good, being able to
5:44
chain two or three tall calls together and in with those
5:50
earlier models, when we talk often a lot about skills, I'm
5:53
sure it will come up in this conversation.
5:56
That kind of ability for the model to navigate and ingest
6:00
content and then make decisions on it wasn't actually there.
6:03
And it's RL that's really given models that capability.
6:06
So, you know, placing models into an environment and
6:11
rewarding them based on the right behaviour and training on
6:16
that means that we've now got these, you know, incredible self
6:20
propelling tool loops where we can get 8090 a hundred turn tool
6:24
loops. And that opens up, obviously,
6:27
the immense power of these coding agents.
6:29
Yeah, it's funny. I remember two years ago at
6:32
Nurips, RL was the huge thing again.
6:35
Again, I guess again, because it's not like it just fell out
6:40
of favor for a little bit and then came back and you see it
6:46
became the hot thing two years ago and now we're reaping the
6:49
benefits of it. Yeah, absolutely.
6:51
I was saying again, I think one of the the kind of other quite
6:54
amazing things is SWE bench where.
6:58
Yeah, Sweet Bench is great. Yeah, exactly.
7:00
And their reference agent, their mini SWE agent is 100 lines of
7:06
code with a single tool, a free form, not even adjacent tool and
7:09
it can get 76% which is close to state-of-the-art.
7:13
So models using a single tool are able to score that high on,
7:18
on SWE bench. And I think as as kind of
7:21
integrators as people building solutions and and platforms,
7:24
kind of understanding how the models have been trained in that
7:28
way means that we can build far more efficient solutions.
7:31
Well, talk to me a little bit about Fast Agent and what the
7:34
inspiration was behind it. So I mean the the inspiration
7:38
behind it was it's originally a fork of a project called MCP
7:43
Agent, which was a kind of an early framework.
7:47
So, and again, this has changed so much over the last 12 to 14
7:50
months that previously if you were looking to do LLM
7:54
integration, you'd be kind of looking at frameworks and
7:57
libraries and looking at the API surface and thinking, is this
8:01
something that I can kind of I can kind of work with.
8:03
So you'd be out the window now. Yeah, exactly.
8:05
Yeah. But you'd be thinking, right,
8:07
Well, how is the API surface? Is this something that is easy
8:10
to use? There's an NCP kind of
8:13
accelerated that somewhat because with MCP, rather than
8:17
building your own custom tool functions and integrating them,
8:20
you, you kind of just had a standard way of, of doing this.
8:23
So the, the kind of inspiration there was that actually, you
8:27
know, you can build on the MCP primitives and start to build
8:31
some really, really complex applications.
8:33
And in particular, you know how it's the model context protocol,
8:40
but particularly how you can load context into the model and
8:43
how powerful that is to actually get that from NCP service
8:47
because you can, you can port them between other tools and
8:51
frameworks and you can reuse them in different ways.
8:55
So, so the, the kind of goal initially was to kind of build a
9:01
almost like an inversion of control framework, but for using
9:05
models and context. And I had specific goals in
9:08
mind. So I was looking at particular
9:11
use cases around kind of enterprise consulting, so
9:16
regulatory impact assessment, organizational design impacts.
9:20
And in those kind of environments you often want to
9:23
chain large chunks of of context together, you want to integrate
9:27
external information. And so the, so the whole frame
9:32
was kind of designed for those sort of, I would call them low
9:35
volume innovative workflows. So that was kind of where it
9:39
went. And then it's become a reference
9:41
platform for MCP to fill MCP specification support including
9:48
sampling and elicitations and those things.
9:50
And yeah, a very capable CLI and agents as tools or sub agents
9:56
system which let you build some quite complex applications.
9:59
I'm not sure I've I've fully understood the becoming a
10:01
reference platform. Can you explain that?
10:03
Yeah. So with MCP, it's the protocol
10:08
itself has got quite a large surface area.
10:10
So it's split, you know, the high level we saw about things
10:15
being application controlled or model controlled and user
10:19
controlled. And then in each of those
10:20
buckets, clients and service have different features.
10:24
And of course the thing that we're kind of most familiar with
10:26
is that we have NCP servers and they supply tools and then your
10:29
LLM can call the tools. There's a lot more sophisticated
10:33
features in there. So, you know, we have resources
10:37
for for bringing content into the application.
10:40
You can list them and find those, find that content.
10:44
And a lot of MCP implementations don't necessarily support that
10:48
full feature set. So for people building MCP
10:51
servers, it can be quite difficult to have a reference
10:55
client where you can say, right, well, you know, I want to try
10:59
sampling or I want to rely on elicitation.
11:03
And so Fast Agent has been a great platform for people to
11:06
build those kind of integrations.
11:08
So it makes it very, very easy to connect to MCP servers in
11:12
development and makes it very, very easy to use some of those
11:16
advanced features and and test them out.
11:19
So it's a way for me to leverage this working well in the way
11:25
that it's supposed to work and test if it is valuable for my
11:29
use case. Exactly.
11:30
And a good, a good example of that is so as part of the NCP
11:37
repository, we have a a server called the everything server,
11:40
which is an NCP server which contains all of these features.
11:44
And you can then you can then use all of these features so.
11:47
It lets you play around with it. It's like try before you buy
11:49
type thing. Precisely, precisely so.
11:51
So that's quite important and it, it also acts as it can also
11:56
acts as an MCP server platform. So you can deploy any of these
12:00
kind of agentic workflows and tool bundles as a server itself,
12:06
which again is hugely powerful. So one example of that we have
12:11
at the moment is the Hugging face.
12:15
We've, you know, we're quite keen on being very efficient
12:18
with people's context windows. Yeah.
12:21
So, so we have a tool which has got, we have a couple of tools
12:25
which have a very, very small token surface area, but give you
12:28
a huge amount of functionality. OK tell me more.
12:32
I like this. Yeah.
12:33
So, so there's two that I think are probably worth talking
12:38
about. And again, kind of just playing
12:42
back to the previous conversation about reinforcement
12:44
learning, these kind of things have now possible because of the
12:48
slightly stronger models. Yeah, so the first of those is
12:54
what we call our dynamic space tool.
12:57
Now on Hugging Face, we've got a way for people to deploy machine
13:01
learning applications. So when you see a new image
13:04
generation model or an OCR model or a text generation model,
13:08
quite often there'll be an application that's bundled with
13:11
it which you can go to and you can use to test the application
13:14
out. And so that's a lot of fun.
13:16
And we have, you know, it's a genuinely useful environment for
13:20
people to deploy applications. We have zero GPU platforms.
13:25
So you can actually deploy these applications without it costing
13:28
you a lot of dollars, right? So.
13:31
And you're hosting it too, right?
13:33
Yeah. So we, yeah.
13:34
So we host it the zero GPU platform as you deploy, for
13:38
example, your own image generator model and you can then
13:41
use it without incurring the huge expense of renting A
13:45
renting a GPU. Now what the dynamic Spaces does
13:51
is give the user the ability through plain natural language
13:54
prompting to say things like, well, I want to create an image,
13:57
I want it to be photo realistic. And then we can select the
14:01
appropriate machine learning model and dynamically generate
14:05
calls to that model so that you can then integrate it.
14:08
And you can say I want to change the camera angle.
14:10
So we know that we have low end to adaptation models that can
14:15
change the camera angle. So it will then use that.
14:18
So you can paste an image URA in and say, well, I want to look at
14:20
that, look at that from a bird's eye view.
14:22
I want to see the camera angle from a 45° image.
14:26
So through a kind of very, very narrow tool surface.
14:29
I think ChatGPT measured it at 45 tokens.
14:32
You open up the ability to access.
14:37
Oh, that's. Behind infinite machine learning
14:39
models, infinite multimodal and textual and audio capabilities
14:45
through throughout very. So that's kind of incredibly,
14:49
incredibly. Powerful.
14:50
Yeah. It's like a small door that
14:51
opens up into a mansion. A.
14:52
Small door that opens up into a mansion and you can then chain
14:55
those things together because the output from 1 can be used
14:58
for another. So because because we restore
15:00
the content. The second example of that is
15:06
our new hub query tool. So it's it's still labeled as
15:08
experimental, but one of the great things about our users is,
15:14
is that when they have an account, they have a certain
15:17
amount of influence that they can use.
15:20
And that gives us the opportunity.
15:22
For example, with the new hub query tool, again, it's a very
15:25
small token surface, about a hundred 150 tokens.
15:28
But unless you just do free form queries to to the hub and you
15:32
can say, you can say, you know, find me trending image
15:35
generation models which have got a number in their name.
15:38
And so that call goes to the MCP server and then it gets routed
15:41
to actually a fast agent MCP server which generates some
15:46
Python code. Yeah.
15:47
So we have at the moment it's using GPTO SS120B.
15:51
So it's an incredibly fast model.
15:54
And the system prompt is loaded with our API surface or
15:58
customized API surface to make it more efficient.
16:00
And we generate some Python code in one shot, execute the code in
16:05
a secure sandbox, we use Pedantics Monti framework, and
16:09
then we return the results. And we return the results
16:12
without post processing them because the LLN that called us
16:15
can actually do that processing. So it's incredibly efficient.
16:19
And why not do any semantic search?
16:21
Or like why generate the Python code versus just doing some
16:26
hybrid search? Yeah.
16:28
So I mean, we, we have hybrid and semantic search, but what
16:32
the, what the ability to generate the code does is it's,
16:35
it allows them all to communicate effectively any kind
16:39
of natural language or change query that you want.
16:41
So for example, I can say who in the Hugging Face organization
16:44
follows me or gave me the GitHub profiles of all the people in
16:48
the Hugging Face organization that follow me and do that with
16:51
them. So it gives you this kind of
16:54
infinite joining possibility, OK.
16:59
Yeah, yeah. So you can really narrow down
17:02
that search space or you can make it very unique to what
17:05
you're looking for. Yeah, precisely.
17:06
So I can see, for example, what, you know, I can ask in natural
17:10
language terms, what models have my followers recently liked.
17:13
Yeah, which again is quite use of information and isn't
17:16
something which is easily accessible through.
17:18
Yeah. Through filtering, yeah, yeah,
17:20
if you're clicking or filtering. So it kind of gives you that,
17:22
those infinite possibilities. Have you played around?
17:25
This just makes me think of a perfect.
17:27
It's a perfect use case for MCP apps.
17:30
Absolutely, one of the reasons I'm so excited about MCP apps.
17:35
So when we look at the traffic we get to the to the Harbin, and
17:39
this will probably sort of lead us to talk more about some of
17:41
the agentic flows is we see, you know, a lot of usage from
17:47
interactive applications like our own chat UI platform or
17:51
Claude AO ChatGPT. So, you know, typically in that
17:54
mode, people are accessing it through through a chat interface
17:59
and they're working with the content interactively.
18:02
And then of course we have, you know, agentic systems like clock
18:05
code and someone which we'll we'll come back to later.
18:08
And so for users which are working with this data
18:11
interactively, often what they're doing is using the MCP
18:15
server for navigation. So they're, they're interacting
18:19
with the hub and they're saying, OK, So what models are trending?
18:22
Or I'm looking for a particular model in this particular
18:25
parameter range. And the thing that happens with
18:29
normal MCP tool calls is you, you, you send the tool call and
18:34
the, the tool returns a set of information and that information
18:38
ends up in the querying models context window.
18:42
And then it generates expensive output tokens replaying the
18:45
information that the tool just gave you.
18:47
And if the model doesn't need to do any extra processing of that
18:50
data, you've effectively just, you know, kind of burned
18:53
electricity. For no real reason.
18:56
And so NCP apps is a wonderful way to actually meet users where
19:01
they are, particularly in those rich, rich chat applications,
19:05
because we can use the models where they're great for doing
19:08
that navigation. But we don't necessarily need to
19:11
replay all of the content and generate expensive output tokens
19:14
and fill the context because the users can see can see what's
19:18
happening. And so we've done.
19:21
We were, we were very early in adopting MCPUI&MCP.
19:27
UI is the same as MCP apps or it is.
19:29
It's not no. So, so MCPUI when I first came
19:33
across about 12 months ago, so the first version of the dev
19:36
summit, it was, I think it was on my kind of final slide
19:38
because we, we just kind of learned about it.
19:40
I was like, this is this is something to be excited about.
19:43
Anita and Liad had had been toying this idea where you could
19:47
use parts of the MCP specifications.
19:51
So resources, resources in particular because it lets you
19:54
deliver arbitrary content and then supplementing that with,
19:57
with talk all information. And it was a, it was a great
20:01
pattern because it was something that you could build into your
20:05
clients relatively easily. And it was quite easy for server
20:08
authors to display content through a, you know, basically
20:13
through a frame, through a frame window.
20:16
And that got sort of reasonable adoption.
20:19
So we, we implemented it in our server a hugging face because
20:23
there were some kind of quite key applications where that
20:25
could where that would work and a few other clients did the
20:29
same. And so there was, you know,
20:30
quite, quite neat SDK, server SDK and client SDK.
20:37
Around about that time, Open AI launched a very similar
20:43
technology called Access DK, which kind of built on it.
20:46
And it was a bit more, a bit more restrictive, not
20:50
restrictive in a bad way, but it was far more opinionated in how
20:53
you should be doing these things.
20:55
Yeah. Was it just?
20:55
React components, right? I think it was, yeah.
20:58
So it's. React components and you needed
21:00
to sort of follow a particular set of patterns and it gave it
21:05
gave you some extra capabilities around, you know, being able to
21:08
interact with the with the calling application and and the
21:12
window and so on. So it was yeah, it was a
21:16
slightly different direction. And to be honest, probably for
21:20
what what they needed a a better fit because it gives you more,
21:25
it probably gives you a more cohesive user experience and
21:29
also meant that the applications that were being hosted were were
21:33
in several ways more powerful because of the the kind of two
21:36
way interaction that that that enabled.
21:38
And so fortunately those two kind of ideas were looked at
21:45
side by side and anthropic or involved as well.
21:49
And then that consolidated with MCP apps.
21:52
And so MCP apps is the official extension which kind of you kind
21:58
of blends the best of all of those, all of those technologies
22:00
with so MC. PUI morphed into MCP apps,
22:04
correct? Yeah.
22:06
Yes, there was. Another one I want to say where
22:09
it allows MCP to use different components of your website
22:13
easier. Is that web MCP?
22:16
Yeah, so. Web MCP I'm not so familiar
22:19
with, but yeah I. Haven't played around with it
22:21
enough either. Yeah, I think with.
22:22
Web MCP the the idea is more that you're able to interact
22:27
with browser elements directly so it's I think the angle there
22:31
is more that you can automate what the browser's doing rather
22:34
than kind of integrating within a chat yeah having.
22:36
It come into you and and do it through there.
22:40
OK, So that is how you added MCP apps support internally.
22:47
So yes. We, we were quite early to adopt
22:50
MCPUI because, you know, as we kind of discussed a lot of, a
22:55
lot of the models that we host are multimodal in nature.
22:58
So it gave us a way to kind of embed that in a, in a quite
23:02
clean way. So something that I'm planning
23:05
to launch actually very soon is proper MCP app which starts to
23:12
use some generative UI. So for example where we were
23:14
talking about the hub query to where you could ask arbitrary
23:17
queries, we can present the data using a generated UI which is
23:23
specific for formatting the results of that query.
23:26
Been using a toolkit called Prefab which Jeremiah Lowens
23:32
roduced. Kind of found by accident.
23:34
Yeah it was funny. It was from the Fast.
23:36
MCP, I was talking to him yesterday.
23:38
Yeah, yeah, absolutely. Yeah.
23:39
So I don't know if he mentioned brief acts.
23:40
Yeah, yeah, yeah, yeah. This is quite funny.
23:43
So, so we built this query to where we were generating the
23:46
Python And returning the results.
23:48
And I was thinking, I kind of want a really nice way to turn
23:52
that into AUI. And it was it was late on a
23:54
Sunday evening and I saw on the on my gear hub feed the Jeremiah
24:00
had kind of Donna commit that was referencing sparklines.
24:03
I was like, what? What's that?
24:04
So Edward Tuft, I don't know if you ever come across Edward
24:06
Tuft, he's. So Edward Tuft.
24:08
Wrote a series of books on how you display visual information.
24:12
They're beautiful books. He self bound and published
24:14
them. What was the name?
24:16
Edward. Edward Tuft.
24:18
So yeah, one of his. Yeah, one of.
24:20
His I think most famous books was the visual display of
24:24
quantitative information. But they they're gorgeous
24:27
artefacts. You can't, you know, don't buy,
24:29
don't buy non physical copies. Yeah, it has to.
24:31
Be a coffee table book you have? Yeah.
24:34
A proper coffee table book. Yeah.
24:36
He's also famously wrote an essay called the Cognitive Style
24:39
of PowerPoint, which kind of breaks down how the, you know,
24:41
how the challenging disaster could have happened because of
24:45
the way that, you know, people lay out information on
24:48
PowerPoint. It's some kind of hide the
24:49
important information things. So he's yeah, he's there's a lot
24:53
of fun in his. Anyway, one of the terms he
24:55
coined was sparklines, which these kind of like small graphs
24:57
that fit in, fit in the space of a word.
25:00
So if you're kind of looking at, say a sequence of win losses and
25:04
you want to show that across a set of 20 football teams, then a
25:08
sparkline would kind of fit fit in the display so that you can
25:12
see that information very quickly.
25:13
So if you kind of have an up for a win and a down for a loss, you
25:16
can see lots of very rain downs data.
25:19
So I saw this GitHub commit and I was like, that's, that's
25:21
interesting. And I kind of clicked through
25:23
and found this library and it's like, hey, what's he cooking?
25:26
So kind of. Straight away he's like, well,
25:28
you know, you probably shouldn't because it's going to break.
25:30
Because it's like, really, really.
25:31
I don't care. Yeah.
25:33
Yeah, I don't care. I'm happy, yeah, happy to be
25:35
part of the experiment. And it just came.
25:37
Out they just released it like Gai, I think, or at least yeah,
25:41
an early. Version So anyway, the the short
25:44
of it is that it's a very so far it seems to be a very, very
25:48
clean way of being able to turn kind of arbitrary content into a
25:52
beautiful user experience through MCP apps and of course
25:56
the the advantage there and all Python.
25:58
Native, which is perfect for what you were doing, yeah.
26:02
Yeah. I mean, we've got, yeah, I tend
26:04
to jump between TypeScript and Python quite a lot.
26:08
But again, MCP makes that quite easy, right.
26:09
So we can kind of route, I was going to say.
26:12
TypeScript before the whole coding revolution, because I
26:15
definitely wasn't a no, it's like, all right, everything's
26:17
TypeScript, let's do it. I've got.
26:19
A fairly long history of different programming languages.
26:21
So I I tend to I'll be honest, right?
26:24
Without LLMS I would probably struggle, but I find that LLMS
26:28
helped me with the syntax, so you can kind of just get the
26:31
ideas the ideas right. So I spent, I've spent a lot of,
26:35
so quite early micrograd used to do quite a lot of software
26:38
development. I spent quite a long time doing
26:40
things that aren't software development and I've come back
26:42
to it quite late. So see, I don't, I don't have.
26:46
I'm not necessarily too attached to any particular to any
26:50
particular language. Yeah, I have.
26:52
I have things which frustrate me about all programming.
26:55
That's for. Sure, as you should as.
26:57
Yeah, as you should. That's why.
26:59
Good. Products get made on top of that
27:01
because it's like this is really painful.
27:03
Maybe let's try and see if something's out there that can
27:06
fix it. Absolutely.
27:07
And we've, we've been spoiled I think with the quality of some
27:12
of the SDKS that have appeared over the last four or five years
27:15
that have made really, really difficult jobs really quite
27:18
easy. So yeah, I think the, you know,
27:22
the software industry's got very, very good making a lot of
27:27
complex stuff really accessible. Your NCP would be one example of
27:31
that. But also if you think about the
27:36
amount of knowledge and maths and deployment skills that are
27:40
needed to run generative large language models, the fact that
27:45
pretty much anyone can build an LLM app in a few lines in a few
27:49
lines of code of it is in an. Afternoon session.
27:51
It's absolutely. Extraordinary.
27:53
Yeah. So we're super fortunate there.
27:55
I think one of the things which has changed over the last three
27:58
to four months and the way I've kind of described it is models
28:02
have got so powerful. The a lot of these kind of open
28:06
source libraries, the way I describe is they're kind of
28:08
liquefied, right? Because the models are able to
28:12
do more of the work. Yeah, the models.
28:13
Are able to kind of generate a lot of this kind of boilerplate
28:16
code. So what it means to own and
28:20
distribute a library has become quite different because, you
28:24
know, models are able to, you know, it's changed the value of
28:28
those libraries somewhat. Yeah, right.
28:30
And and again, I think that means that we have more talking
28:34
about the distribution of ideas rather than the distribution of
28:38
code, the execution. Of it and that's really.
28:40
Yeah. And I think that's, I think
28:42
that's quite an uncomfortable jump for for a lot of people.
28:46
I think the consequences of that are still kind of sinking in.
28:49
But I think certainly over the next few months that's going to
28:51
be. Are there ones that?
28:53
You're thinking of, specifically and.
28:55
I'm thinking more generally about the.
28:59
I think a lot of people call it the Opus moment, but certainly
29:02
the twin releases of Opus 4.5 and GPT 53 codecs just gave
29:09
enough of the step change in coding abilities along with
29:13
actually simplifying harnesses. The that meant that, you know,
29:18
interacting, engaging with and building code became a slight
29:22
different experience. So certainly for my own usage,
29:26
if I think back to how I was using models to generate and
29:29
write code in May or June, it's completely different to how I to
29:33
how I use them now. It is.
29:36
Wild when you hear most folks say, yeah, I haven't written a
29:40
line of code in a while now, yeah.
29:44
Yeah, like So what are? You doing?
29:48
Yeah. And it gives, you know, we, we,
29:50
it means that developers have some very, very different
29:53
trade-offs to make. So, you know, you're always
29:56
fighting entropy. And obviously, if you're kind of
29:59
working with large language models, you're always worried
30:01
about entropy. And so trying to make sure that
30:03
your code bases maintain enough integrity and that you're
30:06
putting enough manual effort into ensuring that the quality
30:10
of the outputs is high enough is kind of traded off with pure
30:13
speed, because models will give you functionality very, very
30:16
quickly. But there's a trade off around
30:20
the quality of the design. And I think that this was the
30:24
kind of trade off that good product managers were making
30:27
over the periods of months as they were building traditional
30:31
software products or Enterprise Products.
30:34
And now that kind of trade off is is kind of in front of every
30:38
developer's desk. It's in your face and you.
30:40
Have to decide it in a session like.
30:43
Oh yeah, yeah, absolutely. And.
30:44
You know, I come from I come from a world where, you know,
30:47
we'd be making P&L business cases for whether to redesign
30:51
parts of systems or to build new features.
30:53
And as I say now that now that that kind of decision is at
30:56
every developer's desk with, you know, do I do I maybe push the
30:59
model a bit harder and accept some accept some design debt for
31:03
the for the speed or do I it's yeah, it's a very, very
31:07
different way of of building things and even.
31:10
I, I heard something really fascinating the other day where
31:15
folks were talking about migrating and migrating from one
31:17
language to the next. And apparently you get better
31:23
performance if you migrate your code to Rust and then migrate it
31:28
to another language. Or just basically migrating to
31:31
Rust will give you a little bit of a, it's a cheat code because
31:35
you get better performance since it's so opinionated and you
31:39
really have to make it work with Rust.
31:41
It's like New York. If you can make it with Rust,
31:43
you can make it anywhere. Yeah, I mean, I think.
31:46
To be honest, I think that's a great observation because I
31:49
think also you can take advantage of different language
31:52
and tools, dynamism versus strictness versus performance.
31:56
I mean, something which I was been talking about the kind of
32:01
the liquification of software is we're we're now at the point,
32:06
particularly with open source, where if you if you think, well,
32:08
I kind of want this feature, I want it in my in my own product.
32:11
Do a model will happily clone a repo for you, look at the
32:14
underlying implementation and then reimplement it in a
32:16
language that that you want. So, yeah.
32:19
So again, I think you know the the great thing is ideas have
32:23
become more important than just writing code.
32:26
It is uncomfortable, but I think that because it can't be
32:29
precious about your code. Or that what you created as
32:32
being yours. And that's the whole idea of
32:35
open source, right? It's absolutely.
32:36
Yeah. You're giving it out for anyone
32:38
to use. Yeah, but now anybody can use it
32:41
and not even it's not yours. Yeah, we.
32:44
We. Do lose things, right?
32:45
We do lose things as a as a consequence of this.
32:48
And something that something I do feel quite strongly about is
32:54
that we we must make sure that software does not become a pay
33:00
to play activity for people that can afford expensive models,
33:04
right? This is I think this is one of
33:06
the the most important things that we need to we need to get
33:09
right. Well, yeah, when we.
33:10
Saw it. We were just saying that Gemma 4
33:12
came out. We haven't played around with it
33:14
yet, but that's a great I'm looking forward to.
33:16
Exactly. Because.
33:17
I've seen the initial vibe checks and it looks really
33:21
strong. And so the thing that I often
33:25
wonder about though, is that if we're not using the expensive
33:29
models, you're still having to use a lot of your time to set up
33:36
the 'cause you're not OK. I guess maybe Gemma 4, if it's a
33:40
very small model, maybe you're setting it up in a few GPU's.
33:45
Maybe it's just one GPU. Yeah, I think.
33:49
But it can get expensive. I guess I'm going to say I'm.
33:52
Going to say something which isn't necessarily cheap, but
33:58
isn't necessarily out of the out of question either,
34:01
particularly, you know, that particularly hugging face.
34:04
If you're doing something interesting, you should let us
34:06
know because obviously we're always interested in helping and
34:09
promoting open source, open source projects, building on
34:13
this stuff. But for example, Gemma, Gemma
34:17
for new to me, I don't know the model at all, not play with it,
34:20
but we'll just use it as an example.
34:22
We say, right, Well, it's it's not as strong as GPT 54 coding.
34:26
It can't be right. And if but if you wanted to use
34:31
it for cost effective coding, are there fine tunes that you
34:34
could do of the model that worked very well for particular
34:37
languages and products? Because now you have the
34:40
weights, you have the ability to to change and optimise how the
34:45
model works. So it could, you know, it could
34:47
easily be the case that through some through some application of
34:53
specialisation and thinking, right.
34:54
Well, I want to solve, I want to be able to write Python code
34:57
cheap or I want to be able to port from TypeScript to Rust.
35:00
Rust, yeah, and get. That lift, yeah, yeah, but.
35:03
The but the for some extra effort in the setup you could
35:07
you. To be honest, when you put that
35:09
kind of effort into the setup, you can then quite easily beat
35:12
state-of-the-art because you're specialising in one particular
35:15
task. So I'm again quite optimistic
35:17
that that those kind of techniques of opening up more
35:22
and more, they're becoming more and more accessible to people.
35:24
We're keen on making them very accessible to people.
35:27
Let's talk about skills. Because I know that you're
35:29
pretty passionate and you've probably got a few cool ones.
35:33
I love chatting and hearing about folks skills that they're
35:36
getting the most bang for their buck out of because a lot of
35:39
times it's not that complex and it's just something that is a
35:44
knowledge gap. Like when you tell me here's a
35:47
cool skill, I get to go implement that and it doesn't
35:50
take me hours to set up. If anything, it takes me maybe a
35:55
20 minute session to create with Claude.
35:58
It's just that I have to know what the skill is.
36:01
Yeah. So what are your most used
36:03
skills these days? So I was saying.
36:06
My most my most used skills, or certainly some of the most fun
36:09
ones, I'm not sure that necessarily fit into the most
36:11
used was one of them. So we wrote a blog post actually
36:16
that did really, really well on using Claude to train other
36:20
models, right. So you don't have to use Claude
36:23
right. But it was, it's kind of a great
36:26
headline because people get massively engaged and then.
36:29
Did. Open.
36:31
Or on sloth, they actually did it.
36:33
Yeah, yeah, yeah, yeah. Yeah.
36:35
So, yeah, so we have a, we have a model trainer skill and
36:39
similar to that, my colleague Mervy has got some image and
36:48
vision training skills so that you can kind of fine tune and
36:51
train image models, but they make it extraordinarily
36:56
accessible to actually take a model and walk you through the
37:00
options that you have. And then at the end of it, you
37:04
end up with a model that you can run and deploy locally yourself,
37:07
or you can deploy on our zero GPU infrastructure and you know,
37:11
or classify your pets, for example, if you've, if you've
37:14
trained a vision model on your pets.
37:15
So people, people love that idea.
37:18
And one of the great things about those skills is you don't
37:22
need to be a machine language researcher, but you can, you can
37:26
download the skill and it will walk you through the options
37:29
that you have and set up the environment.
37:31
And it's amazing how quick you've got something running.
37:35
How quick you can get something? Running and yeah you may need a
37:38
little patience you may need to sort of try one or two different
37:41
things but but yes in the model training skill great fun all
37:45
those researchers that. Spent years of their lives
37:49
learning that yeah, now it is a skill yeah, but we.
37:53
We want that because the more the people experiment, the more
37:56
that they learn and the more that it kind of contributes
37:59
back. Yeah, we kind of build.
38:01
Build more and more stuff. Another, another favorite of
38:05
mine is a skill the the hugging face tool builder skill.
38:11
And what that does is so this is kind of, let's call it kind of
38:15
like a blend of code mode and shell, because what the skill
38:18
actually does is it lets you build your own arbitrary Hugging
38:22
face tools. So our open API surface.
38:25
So, so the API surface is, you know, we let people create
38:31
repositories or deploy models and all those kind of things.
38:34
It's quite large. So what what the what the tool
38:36
builder does is it helps through the shell navigate our API
38:40
surface and let you build custom tools that you can then reuse or
38:45
pipe together to do other things.
38:48
So oh. Interesting.
38:50
So you build the custom tools and then you're able to do those
38:55
things that you want to do. And it's, maybe I'm getting
38:59
confused here then would you, it's not necessarily like
39:03
working with MCP, it's you have a skill that's building a custom
39:07
tool. Exactly.
39:08
And yeah, yeah, so. Maybe so maybe.
39:10
You know, a regular thing that I would want to do would be to
39:13
find image generation models that have been uploaded over the
39:16
last 20 days, find associated papers with them and download
39:21
them in markdown. And then also put a like on the
39:24
data sets that are attached to them.
39:26
So the data sets would be some of maybe some of the training
39:28
data or, or something else. And so rather than, you know,
39:32
rather than having to spend time kind of coding your own script
39:36
to do that or very inefficiently trying to get a model to do that
39:41
repeatedly, you can, you can use the skill to kind of navigate it
39:45
and then it tests it, it puts help text in there so that
39:49
models can use that. So same repeating replicating a.
39:53
Certain a certain workflow in a way precisely so you can you.
39:57
Can build arbitrarily complicated things that use our
40:01
APIs, and it is literally arbitrarily complicated and you
40:04
end up with a tested component that is then suitable for you to
40:08
use. Yourself or for models to use
40:11
because it generates the help text so you can feed feedback in
40:15
and compose them with other with other things.
40:17
So I I have a lot of fun with that right.
40:19
So yeah, that is that. Sounds very valuable yeah.
40:23
And I, I was wondering like, OK, how does this fit with the MCP
40:26
server versus you create your own tool as a skill so with MCPI
40:33
think? That we kind of still
40:36
discovering how people best want to use our products and services
40:39
with agents and chat UIS and those.
40:42
And kind of when I kind of think about MCPI normally think about
40:47
it from three perspectives. And these perspective all have
40:51
actually quite different needs and opinions.
40:54
First is kind of from the consumer or casual perspective.
40:58
So these are these could be developers, it's not just
41:01
necessarily consumers, but these people that want to quickly
41:04
consume what we're doing so that they might, you know, they might
41:08
be using claw to AI and they they want to click Figma, they
41:12
want to click linear, they want to click hugging face and they
41:15
want to be able to interact with those services quickly and
41:18
easily. They don't want to be messing
41:21
around with auth and they want a nice interactive human focused
41:27
environment to get things done. There's enterprise, which has
41:32
often similar, similar goals to the consumer because you know,
41:34
they tend to have quite a large service area of people accessing
41:37
through chat applications. They have additional needs
41:40
around auditing and all regulatory compliance security.
41:43
Scanning all that, Yeah, yeah. And they're kind of it's.
41:46
Kind of very important to them to make sure that what people
41:51
are using and how it's being deployed.
41:54
He's safe, right? Because you don't, because you
41:57
know, if you're in an enterprise, if you're giving
42:00
people arbitrary shell access to, to, to computers that
42:04
you're, you're opening up a surface that you've, you've
42:06
probably rather not. So MCP works extremely well in
42:10
those environments because it gives you that kind of right,
42:13
that right blend of security and flexibility that this kind of
42:18
super important. And again, you know, looking at
42:21
kind of that, that easy to consume nature of say things
42:24
like MCP apps or tool results because that's great.
42:28
And then the, the third community kind of normally think
42:30
about is what is the kind of developing and development and
42:33
engineering community? And I kind of put myself in that
42:38
bucket mostly. And our needs are really, really
42:41
different and our risk profiles are quite different and our
42:46
desire to get the most out of the models as quickly as we can.
42:51
You know, if a new model is released, then people want to be
42:54
experimenting with it straight away then, you know, we, we're
42:58
much more comfortable with providing the models access to
43:02
more, more of our computers, more of our resources, let them
43:05
navigate around on file systems and so on.
43:08
And so kind of between the kind of more controlled tool surface
43:12
that we have with MCP and you know what, what you can do with
43:17
providing access to to your agents with CLIS, we kind of
43:22
trying to get that right, but also I.
43:25
Think. Because only in the conversation
43:28
we were talking about the importance of reinforcement
43:30
learning and, and, and the shell access.
43:33
When we kind of think about shell access for models, that
43:37
doesn't necessarily mean an insecure Yolo mode on your own
43:43
desktop. What's really, really nice about
43:46
tools that that take bash commands is this extraordinarily
43:50
token dense surface. And it's because the kind of
43:56
commands that you that you have with batch of things like
44:01
navigate through this hierarchy. Tell me in plain text what's
44:04
here, Tell me the content of that particular file or part of
44:08
that file. So so the fact that so the fact
44:11
that those kind of navigation commands become native to the
44:15
model is something that we can exploit through all our other
44:17
talk, right. So, so when we kind of think
44:20
about giving access to shell, I think just bashes a recent a
44:26
recent piece of software that's been released.
44:28
Oh, I haven't seen that. Yes.
44:30
So it's again. It's a minimal, it's a minimal
44:34
surface area. So, you know, it doesn't give
44:36
you access necessarily to real resources of file systems, but
44:40
it does give an interface for the model to to navigate, to
44:45
navigate a safe area with shell commands because it's incredibly
44:49
token 10's and models, models have been trained to do so.
44:52
Just bash. Just bash.
44:54
Yeah. All right, I'm going to check.
44:55
That out give me a lot of homework here.
44:57
Yeah, no, it's. Yeah, that's, that's good fun.
45:00
Yeah, OK, I need to play more with it.
45:03
Agent FS is another one which was quite a while.
45:06
It was something again I'm quite interested in because again, we
45:09
can give, we can give models access to these capabilities
45:12
without the security OK concerns.
45:16
Well, yeah, that. Wraps it up nicely in my head
45:18
and that helps me bucket it for when I would be using MCP, when
45:23
I would be using just the the tool building skill.
45:27
And it also, I was thinking about it as like when I want a
45:30
very custom type of workflow, It's probably that tool building
45:35
skill that I'm going to go with. But I also AM in your case, like
45:39
a hobbyist or a developer that can run fast and loose.
45:44
I'm not doing this in my enterprise and I don't need to
45:47
worry about all the other fun enterprise Y things.
45:50
Yeah. And the the I mean.
45:54
The really kind of the fun and exciting part for well, semi for
45:59
me at the moment is, you know, I'm kind of really enjoying
46:03
having models and agents with flat so much.
46:06
You have to do all this kind of stuff really quickly.
46:08
But the fact that we can then also provide similar experiences
46:11
in a in a safe sandboxed environment exposed through
46:15
those kind of consumer applications is extraordinary.
46:18
I mean, you know, the kind of things that the people can do
46:22
with natural language and how rich we can make the experience
46:25
of people using natural languages is superb.
46:28
Right. And we, you don't lose anything
46:30
from the lower layers by doing that.
46:32
Yeah. It just makes everything more
46:34
accessible. And the more accessible things
46:36
are, the the more valuable people can add on top of it.
46:39
Because they're not, they're not struggling with, you know,
46:42
they're not struggling with things that people should, just
46:44
shouldn't be struggling with, with computers.