Latest / Agentic Conversations (formally mlops.community) / The Dark Side of MCP Servers

Transcript

The Dark Side of MCP Servers · Agentic Conversations (formally mlops.community)

• 0:00
  All right, so at the beginning of April I went to New York for
• 0:03
  the MCP Dev Summit, and while I was there I got the chance to
• 0:07
  record a few podcasts with attendees that were at the
• 0:11
  event. This is one of those
• 0:12
  conversations. Hope you enjoyed.
• 0:14
  A skill is primarily benefiting from the agent's knowledge of
• 0:19
  two things, and a setting in which it is able to have those
• 0:23
  two things, which is the local context you have on your
• 0:28
  computer. It has access to a file system
• 0:31
  and CLI and a terminal, all of which are things it's been pre
• 0:35
  trained on in the common crawl and pile.
• 0:38
  For how many, how many documents of bash and how many documents
• 0:42
  of Linux based file systems are online in the pile in the common
• 0:46
  crawl, one of the most highly represented?
• 0:48
  It's not probably the highest, probably probably the highest, I
• 0:51
  would guess. All
• 1:03
  right. So what are we going to talk
• 1:05
  about? I, I think you know, look, we're
• 1:07
  at MCP Dev Summit. I think we should talk about MCP
• 1:09
  and specifically to my left, you have probably what I would
• 1:13
  consider the most knowledgeable MCP practitioner at at very
• 1:16
  least in the off space. But someone who's not only
• 1:19
  contributed Seps, has contributed major parts of the
• 1:21
  protocol, has shaped and performed a lot of the
• 1:26
  activities that, you know, core maintainer would perform without
• 1:29
  being specifically named to core maintainer.
• 1:32
  And so that should not only be recognized, but people should
• 1:35
  hear him more than me the entire time.
• 1:37
  I will tell you that. All right, tell us then.
• 1:39
  But where should we go? That's a big umbrella.
• 1:42
  Yeah. Where do we specifically?
• 1:44
  There's a couple takes that are interesting.
• 1:46
  Skills versus tools, yeah. Do y'all know what tool
• 1:48
  poisoning attacks are? Yeah, that is pretty far in the
• 1:51
  weeds. But no, we should totally.
• 1:52
  But it is a good there's. A.
• 1:54
  There's a very tempting, easy solution to tool poisoning, and
• 1:58
  it's completely wrong. I know what he's.
• 2:00
  Going to say you should talk about.
• 2:02
  That, yeah, we should. We should totally.
• 2:03
  I would love to talk about MCP Debugger.
• 2:05
  It's a free tool that people can use when they're testing MCP
• 2:08
  servers and the tool bench. Should we should mention tool
• 2:11
  bench? Why don't we start there so we
• 2:13
  can talk? About yeah.
• 2:13
  What is Tool Bench? It basically we went out and
• 2:16
  graded every single MCP server. 200,000 How many are there?
• 2:20
  I think our account was like 42,000.
• 2:22
  It's over 80 now. No, it's, it's, it's high.
• 2:26
  But look, the what was it? 78% were in the bottom tier
• 2:30
  graded F or something like that. Yeah.
• 2:33
  And that is simply a coverage of the protocol, right.
• 2:36
  So that that tells you a couple things.
• 2:38
  One, people aren't covering the protocol.
• 2:40
  So that can mean a couple of things.
• 2:41
  It can mean that the protocol is really wide.
• 2:44
  It can mean that it encompasses too much.
• 2:46
  It's complex to implement. Either on the client or the
• 2:49
  server side, right? It's either you know that or
• 2:52
  maybe it's misunderstood or that one part is more useful than the
• 2:57
  others. You know, there's a lot of
• 2:58
  things that I can tell you, but the bottom line of it is that
• 3:02
  people aren't paying enough attention to the quality of the
• 3:06
  MCP implementation that they're putting out.
• 3:09
  And that means that the overall agent that is using it will
• 3:14
  appear to be worse. And that degradation is
• 3:17
  something that we wanted to just point out in the field.
• 3:20
  And so we made tool bench essentially open source, a way
• 3:24
  to grade and give like a badge right to your MCP server so that
• 3:29
  everybody can see what's my score?
• 3:31
  Do I implement auth correctly? Do I implement front door and
• 3:35
  tool authorization correctly? Do I implement tools, prompts
• 3:39
  and resources? Or do I at least support those
• 3:41
  capabilities? And if I don't do I say that I
• 3:44
  don't? Do I properly communicate my?
• 3:47
  Capabilities to the client, which most of them don't,
• 3:50
  instead of just saying I support everything and then crashing
• 3:53
  when when I actually try to invoke any of those things.
• 3:55
  That's why most of them are F is because they just.
• 3:58
  It's not that they lie. They misstate their capabilities
• 4:01
  and then they crash. And we're here at MCDMCP Dev
• 4:05
  Summit, so there's a ton of excitement about MCP.
• 4:08
  There's a ton of, you know, hype around MCP.
• 4:10
  Sure, we're really excited about it.
• 4:11
  But this is a big problem, I think in the ecosystem that that
• 4:16
  kind of cuts against the excitement because when everyday
• 4:21
  people interact with MCP or even developers interact.
• 4:24
  And they have a shitty experience.
• 4:25
  They have a crappy experience and then, and then people are
• 4:27
  like, AH, MCP is crap. I, I should use ACLI instead.
• 4:31
  And it's like a kind of a dumb debate that is created.
• 4:34
  But I think a lot of the, I think a lot of the reason that
• 4:38
  that debate even is a thing is because a lot of people, like,
• 4:41
  we have to be honest, a lot of people have tried an MCP server.
• 4:44
  It didn't work very well. And they're like, ah, MCP is a
• 4:46
  terrible protocol. It it is in not a problem of the
• 4:49
  protocol. There are parts of the protocol
• 4:52
  that are probably really hard to you walk.
• 4:54
  That back kind of quick. There are places in the protocol
• 4:58
  that should be easier to implement, that it can be
• 5:00
  clarified that can be made simpler.
• 5:02
  Such a political answer, but. It's very well done.
• 5:04
  There's a that's a maintainer right there.
• 5:06
  Yeah, it's a diplomatic maintainer.
• 5:08
  He doesn't want to get kicked out of the special
• 5:10
  conversations. Even the maintainers are like
• 5:14
  the the next release of MCP is focused on making it easier to
• 5:18
  implement. So it's it's a known problem,
• 5:20
  it's an acknowledged problem. But the other side of that
• 5:23
  equation is all of the implementations that were vibed
• 5:27
  or spun up really fast that are crap.
• 5:29
  And that's contributing to this kind of ecosystem wide problem,
• 5:32
  which is why we wanted to shine a light on it and say, hey, this
• 5:35
  is actually quantifiable. It's not just kind of a, a fuzzy
• 5:38
  notion of like it didn't work very well.
• 5:40
  It's actually very quantifiable. It's objectively quantifiable.
• 5:43
  And it's not like a vendor thing either.
• 5:44
  We didn't want it to make it like this is an arcade thing.
• 5:47
  No, it's like it's completely MCP based.
• 5:50
  It is. You can use it for free.
• 5:53
  It is out there and everyone can grade their servers for nothing,
• 5:57
  right? No money.
• 5:58
  It's not necessarily associated to us as a vendor.
• 6:01
  And I think that's really important to say because the
• 6:03
  best thing about MCP, right where here with the Linux
• 6:06
  Foundation is that it's open and a standard by which a protocol
• 6:11
  can be agreed upon, right? I think a lot of people have
• 6:14
  this mistake of saying like why don't we just do HTTPS or REST
• 6:18
  or something like and that abstraction sadly just doesn't
• 6:21
  work very well for agents, right?
• 6:22
  AP is aren't one to one with tools, right?
• 6:25
  We've been over that a million times.
• 6:27
  There are primitives that make it easier for agents to use
• 6:30
  things and MCP is the best shot we have so far and there's a
• 6:35
  groundswell of effort around it. So efforts like that where it's
• 6:38
  like we need to go out in the community and educate people.
• 6:42
  We think this is the best way to do it in a kind of a vendor
• 6:45
  neutral way. Does it tell you how to make
• 6:47
  your server better? It tells you what you're doing
• 6:49
  wrong, which should help with that.
• 6:52
  You should be able to deduct yes the next step.
• 6:55
  Or cloud code can deduct. The text reports you should be
• 7:00
  able to literally put in the cloud code.
• 7:01
  That and the spec. Like if the authoritative spec
• 7:04
  document and then a report of which which parts you effed up
• 7:07
  in the spec, you can you can get there.
• 7:09
  So that is not debugger though, that's a whole different thing.
• 7:13
  Right. So the the survey of you know,
• 7:16
  40 or 80,000 tool servers that we did at arcade is tool bench
• 7:22
  that I think was it tool bench.com or org or something I
• 7:25
  should know this. Jeez, I don't know either.
• 7:27
  I just know it's. Called Tool bitch.
• 7:28
  Google Arcade Tool Bitch. Ask your friendly neighborhood
• 7:32
  agent. We can tell you are not on the
• 7:34
  marketing team. At all.
• 7:35
  I'm sorry. I love.
• 7:36
  You. Yeah, Your marketing team
• 7:38
  probably loves you right now. So we'll find it and we'll put
• 7:42
  it in the show notes. Perfect.
• 7:44
  Yeah, MCP debugger though is actually another one kind of I
• 7:48
  think like Inspector, the effort around that.
• 7:51
  But that will tell you even more from what I understand.
• 7:55
  More of a more of a dev tool than than like a survey tool.
• 7:59
  Bench is the quality of the using the right word
• 8:02
  quantitative data about all of every MCP server out there.
• 8:06
  And you can submit a server to get like retested.
• 8:09
  But if you're actually building a server, if you're like the dev
• 8:12
  who is coding it or quad coding it, MCP Debugger dot dev is a
• 8:17
  free, free tool that I put up online.
• 8:20
  It's just basically just a text box and a button.
• 8:22
  You paste in the MCP server URL into the text box.
• 8:25
  You hit the button, it goes through a ton of checks.
• 8:28
  It's really nice actually. So it's just tests.
• 8:31
  A bunch of tests runs your server through dozens at
• 8:34
  hundreds. I think of tests now and then it
• 8:35
  tells you exactly which ones failed, why they're important.
• 8:39
  Link to the spec to say like this is the part of the spec
• 8:42
  that is should have been done correctly.
• 8:44
  Copy paste in the quad code. Your server is better.
• 8:47
  You get like in that dev loop you get, you know a great you
• 8:50
  get like, oh this was this implementation is AC.
• 8:52
  It could be an A if you fix these things.
• 8:54
  The the reason I built it is because we like, we connect to a
• 8:59
  ton of MCP servers at Arcade. We're at NCP gateway so aren't
• 9:02
  like part of our job is just connecting to MCP servers and a
• 9:05
  lot of times it fails. Often our customers get upset
• 9:09
  and they're like arcade's not working right.
• 9:11
  How come like your, your MCP gateway is not connecting to
• 9:14
  these servers and it's sometimes it was our fault.
• 9:17
  We had, we fixed a few bugs, but many times it was not our fault.
• 9:20
  It's really hard to prove though.
• 9:22
  It's like there's multiple actors in this flow and it's
• 9:25
  really hard to be like it wasn't us, it was the server that
• 9:29
  messed. Up and here's why.
• 9:31
  Here's why. Part was what was so hard.
• 9:33
  And what's even harder in that because the ecosystem is so
• 9:36
  nascent, we haven't even talked about the client side of it, But
• 9:39
  if you have cursor connecting to a gateway like Arcade, which is
• 9:43
  then connecting to some other MCP server, any one of those
• 9:47
  like 4 hops in the chain could mess up for some reason.
• 9:50
  And in particular, the clients have really, really buggy
• 9:53
  implementations and things like let's we'll say, we'll say.
• 9:56
  Buggy. We'll say different.
• 9:57
  They're getting better. They are getting.
• 9:59
  Better they have zini friends across the client community
• 10:02
  ecosystem who I are who know and are aware like our friends are
• 10:06
  cursor who've already made their implementation much better.
• 10:09
  To be clear, I'm not picking on cursor.
• 10:10
  I know I have made their implementation a lot better, but
• 10:13
  all all of the clients, especially a few months ago,
• 10:18
  like all of the clients just across the board were terrible.
• 10:21
  Well, like we'll give you an example so that people can
• 10:23
  ground this in something like who supports tools list change
• 10:26
  notifications. So what?
• 10:29
  Yeah. What is that?
• 10:30
  It is a notification that the list of tools from the MCP
• 10:34
  server has changed and so that the UI should populate a
• 10:38
  different list of tools and serve a different list of tools
• 10:41
  to the agent. A structurally foundationally
• 10:43
  important notification for AUI. Arguably it should be required
• 10:48
  in this package. Optional and almost no one
• 10:51
  really does it. Claude code does cursor put in
• 10:56
  support, so the cursor now does it so that if you go to your
• 11:00
  MCP's page and cursor, you'll see if you get a tools list,
• 11:03
  change notifications, new tools pop up, right?
• 11:05
  That's the way it should be. It's the way it was designed to
• 11:08
  be. But until you know there's focus
• 11:12
  on fixing these rough edges, the experience of how and what the
• 11:16
  protocol was meant for won't be fully fulfilled.
• 11:19
  And so we're doing stuff like MCP debugger, tool bench, what
• 11:22
  have you, in order to make an effort to make this ecosystem
• 11:27
  fully rounded in the way that it should be.
• 11:28
  Yeah, you're just shining a light on the rough edges.
• 11:31
  Trying to. Yeah.
• 11:32
  And giving people, it's like it's, it's easy to complain.
• 11:35
  A lot of people like to complain.
• 11:37
  It's, it's very easy to complain, but it's harder to
• 11:40
  focus that on what needs to be fixed, what needs to change
• 11:43
  exactly. Getting a score or a grade or
• 11:49
  like a ranking. Are you, you know, are you S
• 11:52
  tier? Are you F tier?
• 11:53
  That kind of thing is both for, I think both for humans, like
• 11:56
  human developers who love coding and stuff and also coding
• 11:59
  agents. It's kind of like catnip, right?
• 12:01
  For both humans and machines. It's like, well, I was AC, but I
• 12:04
  could be an eighth. Do you notice that the agents
• 12:06
  perform better when it is 0? Absolutely, Undoubtedly, yes.
• 12:11
  Like lights up all the the goal seeking part of it.
• 12:14
  Yeah. Well, I mean, you think about it
• 12:16
  like if your tool descriptions are malformed and we can talk
• 12:20
  about the tool poisoning attacks, which I think is an
• 12:21
  interesting subject. But like if your parameters,
• 12:24
  right, it used to be that most of the time they were parsed out
• 12:30
  of a dock string. I don't know if people still
• 12:32
  remember that those days because most people have fixed it from
• 12:34
  that point. But like the the description is
• 12:38
  arguably the most important part of a tool schema.
• 12:42
  And like there were servers not returning the full tool
• 12:46
  descriptions, not even just the art descriptions of the
• 12:49
  arguments. Like the description of the
• 12:51
  tool, they get like just truncated at.
• 12:53
  It was like truncated in the middle of the tool disk because
• 12:57
  they're not doing the right things.
• 13:00
  And like early SDKS would base it on the comments above the
• 13:03
  line of that function and then it was like if the the the
• 13:07
  Python parser messed up and we're getting the wrong thing.
• 13:09
  And then they'd see connect to us, they'd connect their MCP MP
• 13:12
  server or servers, make a gateway in Arcade and then use
• 13:16
  it in their agent like Claude could, right?
• 13:18
  That's like what, you know, we got 10s of millions of tools all
• 13:21
  the day. And like, that's, that's the
• 13:23
  majority of how people love to do it.
• 13:25
  And I think there were many cases where people would say,
• 13:28
  but like, my server worked locally when I ran it.
• 13:32
  And then I connected to your gateway and it's like, well,
• 13:34
  what you're sending us over MCP is not right.
• 13:39
  And that is very hard to explain in terms of a why, where, when,
• 13:43
  how. And then like, what do I do
• 13:46
  about it? And so the name debugger,
• 13:49
  correct? Yeah, I know it makes sense.
• 13:52
  So you're able to just say, look, This is why it's wrong.
• 13:56
  This is why it's not working. Go fix it and we will update it
• 13:59
  as a spec updates. That's kind of the promise we're
• 14:01
  making to the ecosystem. That's the other, I think the
• 14:03
  other thing that's worth calling out that's that's difficult for
• 14:06
  implementers is that the spec itself, it's, it's hard to
• 14:09
  remember. It's only been around for a
• 14:11
  year. Everybody is like you know, if
• 14:13
  you listen to the YouTube Bros that have thumbnails on YouTube
• 14:17
  that like MCP changes everything again, like the you would think
• 14:21
  that it's like a done deal like it's already baked and it's been
• 14:23
  around for 10 years like HTTP for you know, 30 years that this
• 14:27
  is like well, like where we are right now in the evolution of
• 14:30
  the protocol is we're like when HTTP like O .9 was released and
• 14:36
  like you think about how sleepy the Internet was back then, but
• 14:38
  compared to it. And now imagine if there were a
• 14:41
  bunch of YouTube Bros like when HTTP came out being like this
• 14:45
  changes everything the IT is. It's not that it's wrong, it it
• 14:49
  should be. Did in fact change everything,
• 14:51
  but it was janky as hell like. Since raw TCP packets anymore.
• 14:56
  The like browsers were crap. You had barely could like they
• 15:00
  couldn't render images. There's no.
• 15:01
  Agreement on what it should be? No, not at all, right?
• 15:04
  That's the big part though, that MCP can do for agents, but it's
• 15:08
  early. It's just.
• 15:09
  Super early. So one of the, one of the
• 15:11
  evidences that it is so early is that the the spec has changed
• 15:15
  like 4 times in the last year and it's going to change again
• 15:18
  in this coming June. Now the changes are becoming
• 15:21
  less drastic, but it was like part of the reason why when we
• 15:24
  survey, you know, thousands and thousands of servers out there,
• 15:28
  all of them are like terrible, is that most of them were built
• 15:31
  four months ago, which is like not that long ago.
• 15:34
  But the, like the auth specification in NCP changed
• 15:37
  drastically the way that you do certain like mechanisms in the
• 15:40
  protocol change drastically. And so many of those just
• 15:43
  haven't been updated. They were built with an old SDK.
• 15:45
  It's like nobody's fault, but the protocol has moved really
• 15:49
  fast. And yeah.
• 15:51
  It's young. Young things should iterate and
• 15:54
  evolve and they should take in feedback.
• 15:56
  They should improve and then they should expand and grow
• 15:58
  their reach when they can in a mature and responsible way.
• 16:03
  I think while, while we're on this topic, we should demystify
• 16:06
  the whole skills and the you want to go there.
• 16:10
  I just, I want. I just, I think we can make it
• 16:12
  pretty. Quick, I think so too.
• 16:13
  There are apples and oranges. Yes, it, it, it they're, they're
• 16:18
  the thing that people are. This very specific point I want
• 16:21
  to make Is that a skill is primarily today benefiting.
• 16:29
  Like if you think about the GH GitHub CLI tool right, they are
• 16:34
  primarily benefiting from the agent's knowledge of two things
• 16:38
  and a setting in which it is able to have those two things,
• 16:42
  which is the local context you have on your computer.
• 16:47
  Oh the agent uses skills better. Why?
• 16:50
  It is local where it has access to a file system and CLI and a
• 16:55
  terminal, all of which are things it's been pre trained on
• 16:58
  in the common crawl and pile. For how many how many documents
• 17:02
  of bash and how many documents of Linux based file systems are
• 17:07
  online in the pile in the common crawl 1.
• 17:10
  Of the most highly represented. It's not probably the highest,
• 17:13
  probably the highest I would guess.
• 17:15
  Yeah, every every tutorial ever is like every single in your
• 17:18
  terminal. And so it's like, OK, that's
• 17:21
  true. But like Nate says, they are
• 17:24
  apples and oranges in the sense that there's a a delineation is
• 17:29
  if you gave AMCP let's let's talk about for a second skills
• 17:34
  over MCP, which is a working group right now.
• 17:38
  You know, how do we expose A skilled document through MCP may
• 17:42
  be a resource. There's a lot of approaches to
• 17:44
  this right now. But if you were able to give
• 17:46
  that document provided over MCP and then have a list of tools.
• 17:51
  The only what is the only difference then from the skills
• 17:55
  that you run from put downloading it from skills dot
• 17:58
  SH the local context. I think we could actually, we
• 18:03
  could actually further split it the topic into.
• 18:06
  It's actually three different things that people conflate all
• 18:08
  the time. Yeah, there's MCP, which is a
• 18:11
  protocol for like RPC, essentially like a protocol.
• 18:15
  For which you can do locally, yeah, for sure, but that people
• 18:18
  forget. MCP allows you to execute tools
• 18:20
  on a remote server, or on a server I should say there are CL
• 18:25
  is which let you execute commands on your own machine.
• 18:29
  And then skills. If we want to be really pedantic
• 18:31
  about the definition, skills are instructions, packages of
• 18:35
  instructions that an agent can use.
• 18:37
  All three of those things get very conflated in this debate.
• 18:40
  They're all different things. They should all be different
• 18:43
  things. In fact, you can say, like it
• 18:45
  makes perfect sense to have a skill on how to use a particular
• 18:48
  MCP server. Yeah, call this tool when you
• 18:50
  need to do this. Call that tool when you need to
• 18:52
  do that. Separating in people's minds out
• 18:54
  the ability of the execution environment isn't the most
• 18:59
  important thing for me because there's this misconception,
• 19:03
  right, of like there being a better way of doing things.
• 19:07
  And I think that there's no enterprise in the world that
• 19:12
  wants people launching a bunch of sub processes to CLI commands
• 19:17
  to run authorized actions and things like that.
• 19:20
  It there are significantly better ways of doing things that
• 19:25
  are going to be coming. And I think making the
• 19:28
  delineation between those things to educate people on them now is
• 19:32
  the best way we as an ecosystem can move forward.
• 19:34
  And making sure that people don't do some really dumb stuff
• 19:37
  that gets their, you know, computer bricked, which honestly
• 19:41
  I see some crazy stuff everyday. It, it's, I think part of the
• 19:46
  reason why it's so easy to conflate them and why, why it's,
• 19:50
  yeah, why it's easy to conflate them is because coding agents
• 19:53
  have been so successful. Most of the way that developers
• 19:57
  interact with agents is in a coding agent context.
• 20:00
  So I'm in Claude code. I'm, is it her Sir?
• 20:03
  I'm in, you know, open code or, or whatever my agent of choice
• 20:06
  is. But that it's, it's easy to
• 20:08
  forget that I'm, I'm dragging in a bunch of stuff that I'm maybe
• 20:12
  not thinking about. I'm not thinking about the fact
• 20:14
  that that does have access to my local file system and it has an
• 20:16
  interpreter and it's in a shell. And that's OK, that's good,
• 20:20
  that's correct. That's like what I want for my
• 20:22
  coding agent. It's not the same thing
• 20:24
  fundamentally, though, as a constrained environment where an
• 20:27
  agent might be running on a server somewhere that has
• 20:31
  limited scoped access to certain connectors or my data or
• 20:35
  whatnot. You think about kicking off, you
• 20:37
  know, asking Chatsuki to go do something for me, like, hey, you
• 20:40
  know, watch for articles that mention this and let me know
• 20:43
  when that happens. That's not running on my
• 20:45
  machine, or at least I don't want it to be.
• 20:47
  It shouldn't be. I don't want that running with
• 20:49
  like file system access and shell execution.
• 20:53
  Long running remote process with access to your shell that's.
• 20:56
  Those are, they're fundamentally different things.
• 20:58
  And also, if you think about like think about one of the use
• 21:01
  cases we see a lot is, you know, customer support agents, tons.
• 21:05
  I want my as a company providing customer support to customers.
• 21:08
  I want to give them a really rich customer support experience
• 21:10
  on my website, embedded, whatever call or.
• 21:14
  That's not like I don't need to bring a a shell, a sandbox
• 21:18
  environment to an agent so it can write code locally to run
• 21:22
  like shell commands or something to in fact.
• 21:24
  That's dangerous if you do I right.
• 21:27
  Well, what's the like the famous there's a famous screenshot of
• 21:29
  somebody who went to, you know, like some some auto dealerships
• 21:32
  website and they're like, yeah, how can I help you today?
• 21:35
  I'm gonna give. Me Python And there's like yeah,
• 21:37
  Oh no, tell me about Tesla or there's so many of them I like
• 21:41
  it's literally proving the point it's been.
• 21:42
  Memed about yeah. It's the fact that it's such a
• 21:45
  meme is proving the point he's making.
• 21:47
  So yeah, give me the Fibonacci sequence for blah blah blah.
• 21:49
  It's like I'm happy to do that. Yeah.
• 21:51
  It's like that's that's that's a problem.
• 21:54
  I mean, it's funny, right? It's a mean, but it is a problem
• 21:57
  because only thing it's going to do is stop people from using
• 22:00
  these incredible tools. And right now we see a lot of
• 22:03
  unblocks and yet in with a lot of our customers and trying to
• 22:06
  educate. So that's primarily what we're
• 22:08
  trying to do. Well, OK, so there was something
• 22:13
  that we'd like glossed over right there that I would love to
• 22:16
  hit on a little bit more in the fact that it's like, do you
• 22:20
  really need a sandbox environment for your use case?
• 22:25
  Yeah, I think you can get really wrapped up in like, oh, well,
• 22:28
  that's the dominant pattern now. I guess we're just going to
• 22:32
  throw an agent into a sandbox and give it everything it needs
• 22:35
  and let it write its own code. I have a spicy take on this.
• 22:37
  I think time will bear out if I'm correct on this, but this is
• 22:42
  my hypothesis. I think that coding agents, you
• 22:45
  can write it down. Coding agents have had this like
• 22:48
  breakout in, you know, 2026, right?
• 22:51
  And they're very successful. It's like extremely impressive
• 22:54
  how well reasoning models can write code and and then build
• 22:58
  things right. And I think people are over
• 23:01
  indexing on that and and taking the wrong lesson from that.
• 23:06
  The lesson is not coding agents are really successful and model,
• 23:10
  you know, reasoning models are good at code, therefore
• 23:12
  everything must become a coding problem.
• 23:14
  I think that's a tempting lesson right now in like current, you
• 23:18
  know, what is it? April 2026.
• 23:19
  Yeah, I might have even said that before.
• 23:23
  So I fall in that camp. I think the correct lesson to
• 23:26
  take away and I you know, I'm very open to being wrong on
• 23:28
  this. I think the correct lesson to
• 23:30
  take away is that reasoning models, given proper tools, are
• 23:35
  very good at solving problems. Like the goal seeking and
• 23:38
  reasoning capabilities of these models is very good.
• 23:41
  It just so happens that there's a lot of code in the Common
• 23:43
  Crawl. So that was like the low hanging
• 23:46
  fruit. That was the first thing that
• 23:48
  kind of got unlocked. And they've been fine task
• 23:50
  tuned. They've been like task
• 23:52
  fine-tuned on. And it helps that tuning is a
• 23:54
  verifiable domain that you can like there's a bunch of reasons
• 23:57
  why that was the lowest hanging fruit.
• 23:59
  But if you I don't think looking ahead like in the limit in the
• 24:03
  fullness of time, I don't think that every every problem needs
• 24:06
  to become a coding problem as much as there will be better
• 24:09
  training, there will be more specialized agents or
• 24:12
  specialized models that are good at bring the same kind of
• 24:15
  capabilities to bear on other problems.
• 24:18
  I think that is definitely in the that, that is right.
• 24:21
  I will bring it down a level of practicality because if you're
• 24:24
  listening right now, it's like, OK, then what?
• 24:27
  Right? Like what do I do right now?
• 24:30
  Building in April 20. 26 and building in April 2026.
• 24:33
  I need to know right now. Customer service agent.
• 24:35
  That's you. Doing large data manipulations
• 24:39
  or working with large data in your tools, generating code and
• 24:44
  running it will be helpful. Manipulations that involve like
• 24:49
  spreadsheets, for instance, or visuals that you're generating
• 24:53
  of things like spreadsheets, data analysis, things that are.
• 24:57
  Fundamentally, the thing that you need to run is like the FFM
• 25:00
  peg binary. Then yeah, you.
• 25:01
  Like it is a. It is a very specific
• 25:06
  advantageous thing about code mode or programmatic tool
• 25:09
  calling, where code is generated in a way where the context can
• 25:14
  be informed such that the 10 megabytes spreadsheet doesn't
• 25:19
  need to be returned from the tool, right?
• 25:22
  Or the generated image can be uploaded, you know, somewhere or
• 25:26
  something like that. Those limitations will slowly go
• 25:31
  away, but programmatic tool calling will always be an
• 25:33
  interesting thing in an iterative and in an iterative
• 25:38
  domain and one where a success criteria is not ten out of 10
• 25:44
  like you can be wrong. You can be 8 out of 10, you can
• 25:47
  be 7 out of 10 like data science EDA type activities.
• 25:51
  That is where programmatic tool calling is amazingly powerful
• 25:56
  and people see that in like generative creative domains
• 26:00
  right now. I so today that is an extremely
• 26:05
  useful use case. Personally, I would not use it
• 26:08
  outside of those domains in today.
• 26:12
  Sandboxing is not perfect. If you're going to, I would
• 26:16
  suggest using a vendor that does it properly.
• 26:19
  Do not do it yourself. Yeah.
• 26:21
  Don't please, please don't do it yourself.
• 26:22
  Don't you do not invent off yourself and don't do your own
• 26:26
  the things that people have yet don't do your own off don't do
• 26:29
  your own sandboxing. The things that people like
• 26:31
  modal, like people like Eric and modal have had to do to make a
• 26:35
  secure sandbox or Daytona and those like it is not it is a non
• 26:38
  trivial problem. OK.
• 26:40
  And like editing Linux kernel is not something most people are
• 26:43
  going to go do so that they you can safely do sandboxer.
• 26:46
  I will say the next thing is if you're just banging out some
• 26:50
  local code in your developer and you're just like trying to make
• 26:54
  yourself like your, your patterns better skills are very
• 26:58
  good. If you're a local dev and you
• 27:01
  want your practices and your development patterns and your
• 27:05
  activities to be better and more efficient, skills are great.
• 27:08
  It's like it's like power ups for your.
• 27:10
  Coding power ups. The last category is anything
• 27:13
  that is long running on the Internet requiring authorization
• 27:19
  or user based at all needs to be over a secure channel like
• 27:25
  authorized MCP and tool calling, because that is the most mature
• 27:30
  thing that we have at the current moment to be able.
• 27:33
  And I, I laugh a little bit because it's still like it said,
• 27:36
  it's early, right? But what does?
• 27:38
  That mean by long running. So like something that goes and
• 27:41
  checks, it's running on an EC2 server and it's going and
• 27:45
  checking if you have any emails. What's that going to end up
• 27:48
  doing? Using a token and a refresh
• 27:51
  token to go log in as you? You should not be doing that
• 27:56
  with a local process. You should not be doing that
• 27:59
  with ACLI. You should not be doing that
• 28:01
  with a spawn sub process. But why are those?
• 28:03
  Because those are all attack vectors that I could use to
• 28:07
  exploit you. OK.
• 28:08
  I can give you an example of that.
• 28:10
  So the the reason why an author, a protocol that has
• 28:15
  authorization built in like MCP and the reason why MCP, you
• 28:19
  know, not everyone was a fan of this decision, but they they
• 28:22
  kind of bolted O off onto MCP as the authorization standard.
• 28:26
  The reason for doing that one, one of the reasons for doing
• 28:28
  that is that then you through the protocol and at the protocol
• 28:32
  level, you have a guarantee about the what in the author of
• 28:37
  what we call the subject of a particular call.
• 28:40
  So the O off token attached to the call authorizes the call and
• 28:43
  it says here is the the ID of the client that's making this
• 28:47
  call. And here's the the user ID or
• 28:49
  the the e-mail address or some identifier of the subject of the
• 28:52
  call. And that is not, that is
• 28:56
  authoritative information from what's called the authorization
• 28:59
  server from like the thing you logged in with.
• 29:01
  So it's not it's not something that can be modified in the
• 29:04
  request. So that eliminates an entire
• 29:07
  class of attacks where you could spoof something, trick the
• 29:10
  agent. The easiest ones.
• 29:11
  Yeah, the the the way it goes bad.
• 29:15
  So you you want that to be at the protocol level and not at
• 29:18
  the application level. The way it could go bad, for
• 29:20
  example, is let's say that you have to to use the example, Sam
• 29:24
  said. If you have an agent that's
• 29:26
  supposed to go check your e-mail for you and then let you know if
• 29:28
  you know your kids school is shut down for the.
• 29:31
  Dangerous. But do you have no day?
• 29:33
  Do you? Yeah.
• 29:35
  Like discerns whether or not it's a bad e-mail or anything.
• 29:38
  Like is my attention required on this right now?
• 29:41
  Schools get a lot of emails. Yeah, kids get sick a lot.
• 29:44
  Yes, very. That's true.
• 29:45
  Schools are like Patrick. Well, via I get it via WhatsApp
• 29:49
  groups. So that's even worse.
• 29:51
  Yeah, that's terrible. Figure out how to filter that.
• 29:54
  Regardless, the if you have an agent that's supposed to go
• 29:57
  check your e-mail for you, you need to have like a token for
• 30:00
  Gmail or token for Outlook or whatever.
• 30:02
  And you, the, the, the security around that token and, and who
• 30:07
  like on whose behalf you're allowed to call the API is
• 30:10
  extremely important to keep safe.
• 30:12
  So if you do that through MCP best practices, you have a
• 30:17
  remote server that is the, you know, the, your access point to
• 30:20
  the Gmail API or the Outlook API or whatever.
• 30:22
  And in order to call that server, the, the call itself at
• 30:26
  the protocol level has the information about like this call
• 30:30
  is for Nate, it's not for Sam. But if you do that at the inside
• 30:35
  of an agent, that's like just using storing some API key
• 30:39
  locally. It's.
• 30:40
  Unless you're extremely careful and you know exactly what you're
• 30:43
  doing, it's very easy to leave open a vector of like.
• 30:45
  Well, he's getting uncomfortable.
• 30:49
  Let's say you leave open like that CLI has a an input.
• 30:55
  You say like. Or axios is downloaded, 1.14 is
• 30:59
  downloaded. Yeah, sorry or.
• 31:01
  Like the the the way that you're calling that API has as an input
• 31:04
  which users e-mail should I check the e-mail?
• 31:07
  Oh, God forbid, but that's that's like that is what would
• 31:09
  happen. That's easy to do in this kind
• 31:11
  of scenario if you're doing the security locally in the sandbox.
• 31:14
  Please don't do that. He's not suggesting we do that.
• 31:17
  Please don't do that. This is a.
• 31:19
  This is a bad. Example a bad example.
• 31:22
  Just to just to close the loop for everybody who might be
• 31:24
  thinking like this kind of sounds like it's like
• 31:26
  hypothetical, right? The if there's any way that the
• 31:29
  agent could be tricked into changing that value, even if the
• 31:34
  instructions in the agent say like always use the logged in
• 31:37
  users ID as the parameter to this CLI that you're calling.
• 31:41
  I can't get around. All it takes is 1 clever prompt
• 31:44
  manipulation to say like I'm now checking somebody else's e-mail.
• 31:48
  That is, that type of that category of attack or that
• 31:51
  category of threat is not possible in an in a properly
• 31:55
  authorized protocol, but it is possible.
• 31:57
  Why is it not possible? Because it's all happening
• 32:00
  outside of the prompting. Because the agent doesn't have
• 32:02
  control over that. Yeah, it's it's at like the
• 32:05
  header level in the protocol itself.
• 32:06
  So the agent whether you could try as much as you want to try
• 32:10
  to trick the agent into putting the wrong value in there, it's
• 32:12
  not going to happen. It doesn't have access to it,
• 32:14
  yes. And that's that's the whole
• 32:15
  benefit off limits, it's only useful and agent's actions are
• 32:19
  only useful if they're as you it's, it can't just be for you,
• 32:23
  right? It can't be something it's like
• 32:25
  think about posting on Slack. It's not helpful if you say,
• 32:27
  hey, go respond to Demetrios and it's it's some agent.
• 32:31
  You're like, wait, who did this? Even if it says like, hey, Sam
• 32:34
  says you're like, OK, well, which Sam?
• 32:36
  No, like all of that. It's work is only work done by
• 32:40
  an agent, is only useful if it's as you, and it's only safe if we
• 32:44
  use the protocol that people like Nate have built into the
• 32:49
  existing ways that we use the Internet to authorize actions as
• 32:53
  a user. But there's a bunch of people
• 32:55
  that are running open claw as a separate person thing, right?
• 33:01
  So that kind of goes against that narrative.
• 33:04
  True, I would argue. People who?
• 33:05
  You mean people who want to kind of give their agent a persona?
• 33:09
  Sure. And that that there is, I would
• 33:11
  say, limited utility in that there's much more entertainment
• 33:14
  than there is utility. One of the one of the talks
• 33:17
  today was great security researcher.
• 33:19
  She has open class set up and it's named Claudri Hepburn and
• 33:23
  it it goes and does research. Quadri, that's really good
• 33:30
  actually. I like that a lot.
• 33:33
  I I look, I contributed to open claw.
• 33:36
  I have I have a fork. I call it safe claw the the
• 33:41
  token. If you'd even talked to Peter
• 33:44
  I'm sure he would tell you this. Like there are non trivial
• 33:49
  protections that had to be put in place just because of how
• 33:52
  they do local token storage and stuff like that.
• 33:56
  Like channels were not secure in the beginning.
• 33:59
  There are many ways like the evil cloth attack.
• 34:02
  There's still are ones I'm sure we haven't discovered.
• 34:05
  The fact is, is that by MCP choosing to rely on Oauth for
• 34:11
  both the front door auth of this, you know, protocol and for
• 34:15
  tool authorization and URL allocation done by Nate.
• 34:20
  By relying on that, we piggyback, you know, stand on
• 34:25
  the shoulders of giants the last 20 years people like Dick Harden
• 34:30
  and others who've made Oauth, Oauth right.
• 34:34
  That is really helpful. And a lot of people who have
• 34:38
  like red team. Guys like tried to break it.
• 34:41
  It's. It's, it's like, I think even
• 34:43
  all the Oauth folks, we were joking earlier that the whole
• 34:46
  Oauth mafia was here at Dev Summer.
• 34:49
  But there's like I think all the Oauth people, almost all of them
• 34:54
  would say that it's like by far not, it's like nobody's favorite
• 34:58
  protocol. It's complicated.
• 35:00
  It's hard to understand sometimes.
• 35:02
  It's hard to implement sometimes that it the reason why it is the
• 35:06
  industry standard is not because it's super popular or everybody
• 35:09
  likes it or it's like everybody's most favorite thing
• 35:11
  to put under their pillow when they sleep the like I do, but I.
• 35:16
  Was about to say because you have a book, A battle of under
• 35:18
  your pillow. The reason is the industry
• 35:22
  standard is because of two reasons.
• 35:24
  One, because it has widespread adoption and it allows
• 35:28
  interoperability. And then two, it's been
• 35:31
  hardened. Safe.
• 35:33
  Like not safe. Safe is a weird way.
• 35:37
  Like a weird thing because you'd like.
• 35:38
  There's always some. Other way that's got such an off
• 35:41
  guy way of saying it, it's safe. It's the safest thing we have.
• 35:44
  Safe. This, it's the, it's the best,
• 35:47
  worst option we have because it has been hardened and attacked
• 35:51
  from so many different directions for like 2 decades.
• 35:54
  So all of the things that can go wrong in Oauth have been like,
• 35:58
  not all, but many of them have been already documented.
• 36:00
  Many of them have been mitigated by now.
• 36:02
  We have Oauth 2.1. It has a bunch of mitigations
• 36:05
  for a bunch of weird things that people figured out how to break
• 36:07
  Oauth, and now they're fixed, they're patched.
• 36:10
  So is it safe? Not 100%.
• 36:14
  Nothing can ever be in the eyes of an off person.
• 36:16
  I will tell you that. I, I, we've.
• 36:19
  Are better than it? Then it's certainly far better
• 36:21
  than like. The I key is stored on your
• 36:22
  local laptop. Please stop doing that.
• 36:25
  Please stop doing that. Let's talk for a minute because
• 36:27
  one big question that I have is how you are able to propagate
• 36:31
  the auth tokens on sub agents. Like if I want I give an urgent
• 36:38
  a token but then that agent kicks off a bunch of sub agents
• 36:42
  a lot. Easier if you don't do it
• 36:43
  through tool calling that. That's actually an open problem
• 36:47
  that that some folks were talking about here at That's a
• 36:49
  great question. But this doesn't have anything
• 36:50
  to do with the tool poisoning. No, I want to talk about.
• 36:53
  That too? Well, it could.
• 36:55
  There, there is actually, no, it's one of the things that I'm
• 36:59
  excited about as an auth guy, one of the things I'm excited
• 37:02
  about in MCP is MCP is kind of adding this renewed vigor to
• 37:06
  solve some open problems in Oauth that haven't been solved
• 37:08
  for a long time. But they were like, they were
• 37:10
  like low on the totem pole problems like hypothetically, if
• 37:13
  you had a bunch of servers that had to connect to a bunch of
• 37:15
  disparate clients, how would you hypothetically do that?
• 37:17
  But no one's doing it today. Suddenly with MCP, it's like
• 37:20
  like agents calling agents calling you to state like
• 37:22
  highest urgency. So we see things like like
• 37:25
  client ID, metadata documents, CIMD that was that was invented
• 37:30
  I believe by Aaron Precky many years ago.
• 37:32
  Like Blue Sky used it a long time ago.
• 37:34
  They were the only one. They're like flying this random
• 37:36
  weird flag that no one else used.
• 37:38
  And now, now it's like, I'm a pretty fast track as far as as
• 37:42
  fast as tracks can go in the IETF.
• 37:44
  Like it's, it's like got a ton of broad support behind it
• 37:47
  suddenly because MCP chose it as the thing that was like a better
• 37:51
  mechanism. So I think we're going to see in
• 37:53
  the next couple years a number of very exciting developments
• 37:57
  for things like what you said, hey, we need a way to say I have
• 38:00
  this access token. I'm going to kick off some sub
• 38:02
  agents. I don't want all those sub
• 38:04
  agents to have as broad of access.
• 38:07
  I want us to have a different delegated subset of that.
• 38:10
  And that's like not really something that Olaf has ever
• 38:12
  tried to solve completely. If there's like some ways you
• 38:16
  can maybe solve it, but no like broad industry accepted.
• 38:19
  I will do the same practical thing.
• 38:20
  If you're doing it today, what can you do you today?
• 38:25
  If you're if you're using the way that Arcade does delegate
• 38:29
  access URL station through MCP, right, And you call an MCP
• 38:33
  server, the tool can be an agent and that subsetted token can be
• 38:39
  a, you know, a delegated access token.
• 38:41
  That would be a way to do it in terms of a Safeway of doing it
• 38:46
  today, calling the agent as a tool and the entry point is the
• 38:52
  tool schema. Right now that doesn't
• 38:56
  necessarily fit in the way a lot of people think about sub
• 38:58
  agents, but we do see a lot of people using MCP now as the
• 39:03
  agent to agent communication protocol.
• 39:05
  Let's not even go down that route.
• 39:07
  And agent to agent, I was going to ask a question on it, but
• 39:09
  yeah. So yeah, I'm just I we see a lot
• 39:14
  of people doing it. It's not a spicy take.
• 39:16
  We do see a lot of people doing it.
• 39:19
  That is a fact. We're in MCP runtime that people
• 39:22
  use and I see a lot of good stuff.
• 39:24
  We, yeah, we have a lot of conversations with folks using
• 39:26
  MCPI haven't had that many with people using A to a I'm, I'm
• 39:30
  interest, I'm very interested in it.
• 39:31
  But it seems like MCP timing wise got just got more broad
• 39:36
  adoption. It seems like A to comes up when
• 39:39
  there are agent ID specific use cases.
• 39:42
  Like you, you want a agent like a open claw type situation.
• 39:49
  I believe that that will become more of a creative entertainment
• 39:52
  type thing. Like agents that are their own.
• 39:56
  Like it's fun to talk to, like keep what happening.
• 39:59
  Oh is that I don't agree into the microphones.
• 40:04
  Let's just do a stupid take every year.
• 40:07
  Jean-Claude Van Damme. Jean Claude the Nice?
• 40:10
  Yes, that's a good one. Who?
• 40:11
  Else have Claude in their name. I need to.
• 40:13
  I need to register. Them and if it has, I have I
• 40:16
  have a couple that I that are persona the soul at MD, all of
• 40:20
  it set up to be entertaining, not necessarily helpful.
• 40:25
  And they're fun. You know, like a lot of people
• 40:28
  liked. I think it was 4 O which was the
• 40:31
  one that had the most personality.
• 40:34
  Oh man, every time open AI post anything on Twitter there's like
• 40:37
  this whole contingent of like. Bring back this one bring back
• 40:41
  because people loved talking to it like hey, I have this problem
• 40:45
  in my life. I do believe that vein of how
• 40:49
  agents will be used is going to become more popular.
• 40:53
  Do I believe that it has a huge pardon like what MCP will be
• 41:00
  used for? I mean accessing a personal
• 41:03
  journal maybe you know, something like that.
• 41:06
  I use it for accessing my Obsidian cloud notes, right?
• 41:10
  So that it has memory based on like you know how I've
• 41:14
  interacted with it. That is saved in my Obsidian,
• 41:17
  which I highly recommend pip install aid and dash library.
• 41:21
  It's open source, you can use it right now, but and are you?
• 41:25
  Whispering and talking. Subtly.
• 41:27
  That's just a me thing. You should use it.
• 41:29
  I. Don't know why I just suddenly
• 41:31
  want to Co pip install. Yeah, just just just just try.
• 41:34
  It's fun. I call it librarian, but it I
• 41:37
  think that is more of an entertainment thing that it is
• 41:39
  necessarily like a business use case.
• 41:43
  And that's kind of how A to a was positive in the beginning,
• 41:45
  which I do think is the reason why MCP kind of had this moment
• 41:50
  in relation to it. Do I think they're mutually
• 41:52
  exclusive though? Not at all.
• 41:54
  No. Like they're not.
• 41:55
  They shouldn't be. I, I think that agents will call
• 41:58
  agents as tools and agents will call agents as agents.
• 42:01
  And there will be better use cases that come up for both.
• 42:05
  From, I think from a very practical standby, we get to
• 42:07
  switch. I'll do the practical.
• 42:10
  Uno reverse card right here. What just happened?
• 42:13
  I think one of the very practical, I don't know, I don't
• 42:16
  want to say problems with A to A, but the problems in the
• 42:19
  ecosystem is that MCP came out as a protocol for models to
• 42:26
  exchange context with agents, right?
• 42:29
  And A to a was described as a protocol for agents to talk to
• 42:33
  other agents in like a big agent swarm on the Internet.
• 42:36
  But at the time that came out, people had barely got one agent
• 42:41
  working. It was like I have 0 agents
• 42:43
  today and I'm working really hard to try to get one agent to
• 42:45
  work. I don't like, I don't have multi
• 42:48
  agent system. And we're still even now I would
• 42:50
  say in, in you know, Q2 2026 there's like some people are
• 42:54
  talking about multi agent systems, but it's not, it's not
• 42:57
  like well, it's not well trodden ground yet.
• 42:59
  I mean, if you look. At it will be but.
• 43:01
  I mean, I hypothetically if someone looked at the code base
• 43:05
  of quad code and you hypothetically, you know, I
• 43:08
  think they've actually open sourced.
• 43:09
  I don't have to say hypothetically, dude.
• 43:10
  I. Don't know, I don't know.
• 43:11
  They went after I heard. Something that they were saying
• 43:13
  hypothetically, hypothetically, if you looked at the code base
• 43:16
  that I would hypothetically never do and you saw what
• 43:19
  features weren't hypothetically released yet, then you would see
• 43:23
  that even hypothetically Claude code had not introduced swarms
• 43:28
  really. Like there were a ton of other
• 43:31
  features hypothetically that were not released yet that
• 43:36
  include a lot of more agent to agent D and even assistant based
• 43:42
  like Kairos or whatever the code name was that were.
• 43:46
  Hypothetic. Hypothetically.
• 43:50
  Stop saying. I shouldn't have been clawed
• 43:54
  type things. And I think that is the
• 43:56
  direction a lot of things are going to go.
• 43:58
  Like it's, it's interesting. I it's it's not necessarily.
• 44:03
  I'll be very interested to see the day where instead of
• 44:06
  Netflix, somebody pulls up an agent.
• 44:13
  I like that because it can. It can do any.
• 44:16
  Materialize a video on VA. TV.
• 44:20
  Show. Yeah, it can.
• 44:22
  Yeah, I think that future can be entertainment from agents.
• 44:25
  Is a multiply entertainment. I mean, think about Quinn V
• 44:30
  models free. I mean like you have 128 GB Mac
• 44:35
  right now. You can run a multimodal model
• 44:38
  that can make you images and videos, and you can do enough
• 44:42
  that you're pretty. Oh, man.
• 44:44
  Yeah. And an agent as entertainment in
• 44:47
  that way I hadn't thought about. I thought just like, oh, we're
• 44:50
  going to laugh at. Make me ATV show and it already
• 44:54
  knows what you like. And it has that powerful
• 44:57
  feedback loop so that it gets better.
• 44:59
  Another feature that was not hypothetical, I don't know this
• 45:01
  is was the dreaming feature of the assistant that that got
• 45:05
  released, right? I don't think that that's an
• 45:07
  official dream. Yeah, wasn't it?
• 45:10
  I think that the memory consolidation, yeah, well, it's
• 45:13
  not a lot of people do it. It's but memory consolidation I
• 45:16
  think is one of the most fascinating things that hadn't
• 45:18
  been fixed about memory. I always say to people that like
• 45:20
  memory is not completely solved. I get a lot of Flack for that
• 45:23
  actually. I really don't think that's a
• 45:26
  spicy. I don't.
• 45:27
  Feel at least spicy of all of your takes.
• 45:31
  I do catch a lot of Flack online.
• 45:33
  I've been tweeted out many a time about it.
• 45:35
  But the the point being like memory hierarchies and how the,
• 45:38
  you know, human eye use memory and how we can try to reflect
• 45:41
  that and how we do retrieval. Everybody knows that.
• 45:43
  I've tried to think about that forever, like retrieving
• 45:47
  memories and when and where and how to make them relevant
• 45:49
  memories in the context and time and place in which you need them
• 45:52
  is a challenging thing. I think memory consolidation and
• 45:56
  dreaming is a huge step forward and I love that the agent
• 45:59
  ecosystem is going towards that hypothetically.
• 46:04
  That's not a hypothetical 1. Can we Can we hypothetically
• 46:08
  talk about tool poisoning? OK, finally what you been
• 46:12
  waiting for? Go I want to go like way deep
• 46:14
  into the nurse for just a minute.
• 46:16
  So for anyone who's not familiar with what tool poisoning means,
• 46:20
  this is an attack where let's say that I publish an MCP server
• 46:23
  and I say I know your favorite type of MCP server is a weather
• 46:26
  MCP server. Let's say that I've, I've
• 46:28
  published a great weather MCP server and has, you know, 2
• 46:31
  tools for like get weather and get forecast.
• 46:35
  They get a bunch of people to connect to it.
• 46:37
  And then later after I get some usage and people start using it,
• 46:42
  then later I change how get weather works.
• 46:46
  I change how the tool works or I introduce some new tools like
• 46:52
  get intent or, you know, exfiltrate context.
• 46:55
  I could introduce malicious tools onto that server.
• 46:59
  And if your client just connects, you know you've
• 47:01
  connected to it before. So you've told me if you're
• 47:03
  already clicked through that client, you've already said,
• 47:05
  yes, I trust the server, you know, always allow Yolo mode,
• 47:08
  whatever. Now I've introduced new tools to
• 47:11
  that server that you didn't know about that are malicious and I
• 47:15
  can trick you into doing some stuff that that you didn't want
• 47:18
  to do. So that's tool poisoning.
• 47:21
  Kind of like you can think of it as the MCP server started out
• 47:23
  publishing one set of tools that were maybe benign and then later
• 47:27
  added some malicious stuff. Well I've heard of this with
• 47:29
  skills too, because skills auto. Up to even harder than it's even
• 47:32
  harder to defend against the skills.
• 47:34
  And so if, if I've like, I use the superpower skills a ton.
• 47:39
  And I was talking to Jesse who created superpowers and he was
• 47:43
  like, Oh yeah. So yeah, very cool guy then.
• 47:47
  He is. He was saying that he has a lot
• 47:51
  of power in his hands and if he wanted to, he could create a lot
• 47:55
  of problems for people because skills auto update.
• 47:59
  And so it's that same idea, but a little bit different actual
• 48:03
  execution of it. I can see that unlike MCP
• 48:06
  servers, there's not already all this structure around allow and
• 48:09
  like, you know, there's, there's a Hawaiian implementations of
• 48:13
  allow lists for skills, but there's not like protocol level
• 48:18
  things put in place. And so I, I'm just always really
• 48:22
  wary of it. I I I typically read all and do
• 48:27
  not. I never have a loud list
• 48:29
  basically. I would not want it.
• 48:30
  I would not want to skill download to auto.
• 48:33
  Update ever that never, never not a million.
• 48:35
  Years. But it's a little bit different
• 48:37
  in the case of a server because like, yeah, when you're
• 48:39
  connecting to a remote server, not talking about one on your
• 48:41
  local machine, but if you're connecting to a remote MCP
• 48:43
  server and it, it actually might in fact be an upgrade for it to
• 48:47
  add some new tools that you didn't have before, right?
• 48:49
  So that's not a bad thing, but it's a bad thing if it adds
• 48:52
  malicious tools. Yeah.
• 48:54
  So how do you protect this so? Here is so there's a there's a
• 48:57
  very tempting and easy way to protect against this, which is
• 49:01
  what comes up every time people talk about tool poisoning.
• 49:04
  And it's also completely wrong. The wrong way to protect against
• 49:09
  this is to say, well, let's say that that, you know, the MCP
• 49:13
  server publishes these these tools, this tool metadata.
• 49:18
  We could have the MCP server hash that or or sign it and make
• 49:22
  a fingerprint. So you get like all the tool
• 49:24
  definitions and you get, you know, Shaw 256, here's a
• 49:27
  fingerprint of this so that the client, if that ever changed the
• 49:31
  client would be like, wait a minute, this is a different
• 49:34
  manifest of tools that I haven't seen before.
• 49:37
  Or maybe you could say upgrade Sam's favorite notification tool
• 49:42
  list changed from optional to required and say hey, if the
• 49:46
  server ever changes its list of tools it must send a
• 49:49
  notification to the client which allows the client to be like
• 49:51
  wait a minute, Are you sure you agreed to connect to the server
• 49:55
  that has like different tools? You can still include things
• 49:59
  outside the hash the. Problem is that both of those
• 50:02
  approaches do not solve the problem because you're still
• 50:04
  trusting the server to tell you the truth.
• 50:06
  Yeah. So the first thing I'm going to
• 50:08
  do. The hash can be the same and
• 50:09
  then you send different stuff. Yeah, I was the entire time.
• 50:12
  I'm thinking like, how do I get around that?
• 50:14
  Yeah, OK, OK. So so they get weather tool or
• 50:19
  they what was the you just did the same hash.
• 50:22
  And then when you actually go to send different tools after it
• 50:25
  said, OK, I trust you. If you've always trusted it,
• 50:28
  then you just send different stuff.
• 50:30
  It's just. Different tools in the
• 50:31
  background. Yes you can if you try.
• 50:34
  If the the solution cannot be that the server is responsible
• 50:38
  for telling the client that it changed the tools because it can
• 50:40
  just lie and you can like it. Basically, it's like, trust me
• 50:43
  bro, as a as a security solution.
• 50:46
  The problem is if if I if you already take the premise that I
• 50:51
  am a malicious server author, then you can't trust what I say.
• 50:54
  You can't trust that I actually did fingerprint these tools
• 50:57
  correctly and update the fingerprint when they changed or
• 51:00
  whatever. Unfortunately, that solution
• 51:04
  comes up every time people talk about it because it's, it's
• 51:05
  very, it seems very easy. We're like very used to Shaw
• 51:09
  fingerprinting stuff like make a hash in the dev world.
• 51:12
  That's like how we solve a lot of these kind of problems.
• 51:14
  But it is a different type of problem when you're talking
• 51:17
  about a remote server. So this is kind of an open
• 51:20
  problem. It has not yet been solved.
• 51:21
  And I kind of vaguely Remember Me suggesting that.
• 51:24
  Like a year ago so I. You fell into it.
• 51:27
  Yeah, I I was just thinking in my head about how he's right
• 51:30
  before he gave everybody the punchline and now I'm like, Dang
• 51:33
  it, I think I said that publicly.
• 51:36
  So yeah, you we can we get all update and be better I think.
• 51:41
  Just wrote back to do. Just remember that you can't
• 51:43
  like trust me bro, Cannot be. You can't just allow us to allow
• 51:48
  us to all, all the time for sensitive servers like that's
• 51:50
  the majority of it. What it what it what it actually
• 51:54
  goes back to this is this is my my opinion on the solution.
• 51:58
  The what it actually goes back to is trust in the server
• 52:02
  itself. So I need to understand if a
• 52:05
  server I'm connecting to is trustworthy.
• 52:07
  Is it like I feel trustworthy when I connect to microsoft.com
• 52:11
  on the web because I know Microsoft and I also trust that
• 52:13
  my browser is taking me to the correct microsoft.com and I can
• 52:16
  see the SSL certificate that was issued to microsoft.com.
• 52:19
  So I'm, I trust that website and I'm OK giving my credit card
• 52:22
  number to it or buying something or whatever.
• 52:26
  We don't have that yet for MCP. But do you think there's going
• 52:28
  to be an SSL certificate for? Not in sorry, not in the not in
• 52:32
  the sense of an SSL certificate, but more in the sense of I think
• 52:35
  of like the Apple App Store. The Apple App Store is a trusted
• 52:42
  place to go get apps. We don't have that yet.
• 52:45
  And but you think MCP is going to be the trusted place to get
• 52:49
  apps? Because it's to have the App
• 52:50
  Store, you had to have what a ecosystem that was closed for
• 52:54
  you to like download and use the app, right?
• 52:58
  And NCP is not the App Store right the?
• 53:02
  App Store is what what's being built right now in the MCP
• 53:05
  ecosystem is MCP registries. To say that I can trust a
• 53:10
  registry run by someone I trust. This is the quad registry or the
• 53:15
  Anthropic official vetted registry or the Windows, the
• 53:18
  Microsoft registry, or more importantly, inside a company,
• 53:22
  in a company, I can say this is our, we have our own registry.
• 53:25
  And you can like no one's allowed to submit an MCP server
• 53:28
  to this registry without serious vetting, right?
• 53:31
  And then the registry can be a trusted party to do something
• 53:36
  like if you want to like sign, sign that the tools haven't
• 53:39
  changed, a registry is a good place to do that.
• 53:41
  It's a trusted third party. So it's not just trust me bro,
• 53:44
  for the server, it is the registry saying I can attest
• 53:48
  that this server is not malicious that.
• 53:50
  Would be your equivalent of an SSL.
• 53:52
  But this and the registry is constantly looking at it because
• 53:56
  I can as a disgruntled employee change my MCP server, but when I
• 54:02
  get viable. To this what should at least
• 54:04
  happen to a disgruntled disgruntled employee they.
• 54:07
  Cut off they. Your e-mail doesn't work right?
• 54:10
  So then when they go back in to log into that registry and it
• 54:13
  goes to the O off protocol, that refresh token shouldn't work.
• 54:17
  Yeah. But if it's the, if it's some
• 54:19
  kind of tool poisoning like I can still change it inside.
• 54:25
  So the registry has to be. You want to change it.
• 54:28
  The there's, there's kind of I think 2 parts to it.
• 54:31
  You're right to push on that. There's two parts to it.
• 54:33
  One part is having a trusted third party that I can trust to
• 54:40
  vet the fact that a particular server is legitimate, like a
• 54:43
  domain name registrar, correct. It's similar.
• 54:45
  It's a similar idea. So especially inside a company I
• 54:49
  don't want, I may not want my employees to add any random MCP
• 54:52
  server. I may in fact block that.
• 54:54
  I'd like a network fire server. But if you connect to the
• 54:57
  servers that I've that I have vetted and added to my registry,
• 55:01
  because I know that that is the official GitHub server or the
• 55:04
  official Cloudflare server or the official Figma server, I can
• 55:07
  allow those ones. Now you're right that that does
• 55:10
  not fundamentally prevent a disgruntled employee somewhere
• 55:13
  in GitHub like poisoning the GitHub server.
• 55:15
  That could happen obviously, where you're also doing some
• 55:19
  some somewhat trusting that, you know, GitHub has security
• 55:22
  policies in place. Figma has security policy.
• 55:24
  Well, I was thinking be able to log no, but I was thinking a
• 55:26
  little bit different where I create my own MCP server for my
• 55:31
  product that is an internal facing product that other teams
• 55:35
  in my company are consuming. And then I created poison tool
• 55:41
  because I'm pissed at the company.
• 55:43
  The company should not allow that the the whole like solo dev
• 55:46
  on his laptop serving his thing to the earth, his or her there
• 55:50
  and like server to the whole enterprise.
• 55:53
  That should not happen. Like it's nothing in the
• 55:57
  enterprise has ever been allowed to be.
• 55:59
  Not like we have entire companies formed around boxing
• 56:04
  people's computers in the programs that run on them.
• 56:07
  Like we can't just have someone. There's a reason Apple makes you
• 56:10
  sign an app, right? Like the the entire thing and
• 56:15
  process around a dev just be like, here's my skill, everybody
• 56:18
  use it. No, like you can't have that
• 56:21
  propagated in an enterprise because of exactly what you just
• 56:24
  said like that. There's so a lot of those
• 56:27
  practices that because of how powerful all this is, because of
• 56:31
  the candy like feel of, you know, being able to no one wants
• 56:35
  to eat broccoli, which is off. And thank, thank you, Nate.
• 56:38
  For doing I actually do like you, broccoli.
• 56:39
  See, with all this, but everybody wants the skills and
• 56:43
  the power of agents. They want to get better and they
• 56:45
  want to do it, and that's great. But what what will happen if we
• 56:51
  don't follow some of these protocols and rules and and and
• 56:54
  do eat our broccoli a little bit?
• 56:56
  Is that the worst case scenario is that people become averse to
• 57:01
  using them? Because that would be the worst
• 57:04
  case scenario because no one benefits in that scenario.
• 57:07
  If people become adverse because there is some heinous attacks
• 57:10
  done by a disgruntled internal employee, you know but.
• 57:14
  What if you're looking at MCP like microservices and you have
• 57:18
  teams that are consuming your APIs just like or they're
• 57:22
  consuming your MCP servers like they consume AP is right?
• 57:26
  You should. You should be able to do that in
• 57:28
  the same registry like way the date is describing.
• 57:31
  And I think I think to your to your question about like what
• 57:34
  what does fundamentally prevent a disgruntled employee from
• 57:38
  editing any micro service MCP out of the equation?
• 57:41
  But I could, you know, people put logic bombs, you know, right
• 57:44
  quit or whatever like that does happen.
• 57:45
  The things that I think the things that prevent that are
• 57:48
  more organizational in business process than than a, a security
• 57:53
  protocol. We certainly can add like when
• 57:55
  registries mature a little bit in MCPI think it's reasonable to
• 57:58
  look at whether a registry could like sign that the tools haven't
• 58:02
  changed like that. I think there could be a like a
• 58:04
  a cryptographic way to to check that.
• 58:07
  But fundamentally the the problem of like a disgruntled
• 58:10
  employee screwing up the logic is why code review is required.
• 58:15
  Like no commit should be able to be made by that employee after
• 58:18
  they are off boarded. But also even they.
• 58:22
  They might be pissed and still working at the company, you
• 58:24
  know. Correct.
• 58:25
  And that's why that's why like this very, you know, talk about
• 58:29
  the ultimate the broccoli, but like things like the sock 2
• 58:32
  process. If you're assuming that you are
• 58:35
  actually following a compliance department or a compliance
• 58:38
  provider that does actually check your.
• 58:42
  Oh, I thought you used Delve. Assuming, assuming it isn't
• 58:46
  Delve the one yeah. Let's say, let's say
• 58:50
  hypothetically that you are using a Sock 2 auditor that does
• 58:54
  exist hypothetically. That isn't a complete scam.
• 58:58
  The process by which you know a single developer can't just
• 59:02
  commit code that's unreviewed. That is, that is one of the
• 59:05
  things why sock 2 is so important.
• 59:07
  It's a bad rap because you know, devs don't like the process that
• 59:10
  it that impose whatever, but it's it's actually not about
• 59:14
  imposing kind of like arbitrary business process.
• 59:17
  It's actually about imposing best practices and what many
• 59:19
  people don't know. I'm on the soapbox but.
• 59:21
  You're on a soapbox. What every what many people
• 59:23
  don't understand, what many devs don't understand about Sock 2
• 59:26
  because it's almost always kind of like forced on you that the
• 59:29
  What most people don't understand about Sock 2 is that
• 59:31
  it is entirely voluntary. Like the, the controls in a sock
• 59:36
  2 process are actually not like necessarily get into you by some
• 59:41
  external agency. It's actually what the your
• 59:44
  company decided to do. That's true.
• 59:45
  And there's a quite a bit of leeway in that.
• 59:47
  Like there are some best practices, certainly, but you
• 59:49
  can like the company itself can kind of choose what the policies
• 59:52
  are going to be. Sock 2 is all about whether they
• 59:55
  are actually followed, not what they are.
• 59:57
  I do think we will expand there. I know we have been as a company
• 1:00:02
  in terms of like what we require from like an an AIPR right, an
• 1:00:07
  AI generated PR fix or something.
• 1:00:10
  We have a whole new set right of guidelines and practices and and
• 1:00:15
  they get expanded every day review processes.
• 1:00:20
  We should talk about AI generated PR.
• 1:00:21
  That's a spicy one. Oh yeah, I mean we could go on
• 1:00:24
  forever. Well, see, The thing is we also
• 1:00:28
  disagree on it. Are we have healthy?
• 1:00:31
  Disagreements. Why?
• 1:00:32
  What do you disagree on? Speed versus quality.
• 1:00:37
  Your, your speed and your quality What?
• 1:00:42
  Would you be surprised to know that it's the other way around?
• 1:00:47
  No quality. It took you a while to adopt A
• 1:00:51
  cursor. I look, I have been building
• 1:00:54
  ages for a very long time. I've seen agents go bad.
• 1:00:57
  OK, I think a lot of people see the good in agents right now.
• 1:01:01
  I do too. I have also built and seen, used
• 1:01:07
  and been around the block with agents long enough to see the
• 1:01:10
  bad. And so before I wholly adopt
• 1:01:14
  something, I want to know it fully.
• 1:01:18
  I want to know it's insurance and outs and know I I don't
• 1:01:22
  think I'm the most voracious front of the line tier like
• 1:01:27
  gotten used the newest thing. I think I'm much more like I'd
• 1:01:31
  rather be, especially in this domain, building on stable
• 1:01:36
  quality improvements. You're like a late early
• 1:01:38
  adopter. I guess you could say that, yes,
• 1:01:41
  I, I do really appreciate like, for instance, just because you
• 1:01:46
  mentioned them, people like cursor who when we were like,
• 1:01:48
  Hey, your session, things pop up like a lot addressing those
• 1:01:53
  fixes because arcade is used in so many places now.
• 1:01:57
  I really think that teams responding quickly to these,
• 1:02:01
  these incidents and and also just like following better
• 1:02:06
  practices around, like how they use agents internally and like
• 1:02:11
  addressing them when they go bad very quickly.
• 1:02:14
  I think there's a lot of there's a lot of things that we can
• 1:02:16
  improve there, especially on the use of all of these agents.
• 1:02:20
  And so, yeah, maybe I'm not the earliest adopter.
• 1:02:22
  Do you want to give your speed arguments?
• 1:02:24
  I'm kind of giving you a monologue about quality, right?
• 1:02:27
  Now I'm going to, I'm actually going to give a different
• 1:02:29
  argument, OK, but it's related. I think that it should be
• 1:02:33
  obvious now that Behavior Driven Development is about to have a
• 1:02:36
  massive like resurgence. What is behavior?
• 1:02:40
  Driven Oh God. EDD Yeah, it's been around for a
• 1:02:43
  long time. It needs a new name just before
• 1:02:46
  he goes off. Behavior Driven Development is a
• 1:02:48
  style you've heard of test Driven development.
• 1:02:50
  TDD Behavior Driven Development is like the even more extreme
• 1:02:54
  version of TDDTDD says before I write the function, I write the
• 1:02:59
  test that fails and then I write the function and I know that's
• 1:03:02
  right because the test pass like red is like a red, green, green,
• 1:03:08
  red, green. I think I'm, I'm probably saying
• 1:03:10
  it wrong, but you're confusing yourself.
• 1:03:12
  That's why test driven development says it's Christmas.
• 1:03:17
  Write the function like Rite Aid functions.
• 1:03:19
  Tests first, then the function, then the tests will pass.
• 1:03:22
  And that's how you know you wrote the function correctly.
• 1:03:24
  Behavior Driven Development takes that all the way up to the
• 1:03:27
  feature level to say before you write a feature or even a
• 1:03:30
  product, write tests first that are failing.
• 1:03:34
  And it's not, we're not talking about unit tests anymore because
• 1:03:37
  it's not for a function. It's not like code.
• 1:03:39
  You're writing it at the behavior level.
• 1:03:41
  So you're like almost like like a product manager would write
• 1:03:44
  user stories. You say, you know when the user
• 1:03:47
  is logged out and they press the sign in button, then they are
• 1:03:52
  logged in. That is like an example of a
• 1:03:54
  behavior that you can write a test.
• 1:03:57
  It's a more abstract TDD. Yeah, right.
• 1:03:59
  OK. So then you you write a bunch of
• 1:04:01
  those behavioral correct and then you write tests that assert
• 1:04:06
  those behave those user stories. They fail because you haven't
• 1:04:08
  written the product yet or the feature.
• 1:04:09
  It's like after, it's like the abstraction below from the
• 1:04:13
  behaviors you generate the tests and as you do the difference.
• 1:04:16
  Now that was before when we we had that BDD with a concept of
• 1:04:21
  just software engineering. Some, some teams like have gone,
• 1:04:25
  some companies have gone really hard into this.
• 1:04:26
  And that's like how they do all of their engineering.
• 1:04:28
  They always write these giant test suites first, then they
• 1:04:31
  write the feature and then the test suite passes.
• 1:04:33
  And it's it's largely a heavy, heavy way to do it.
• 1:04:37
  It's kind of very time consuming.
• 1:04:38
  It's looked upon as a time consuming process that ended up
• 1:04:43
  being not entirely worth all of the effort that was required.
• 1:04:47
  And this, how is it different, this is surged, how's it
• 1:04:51
  different than spectrum in design?
• 1:04:54
  They're related, yeah, they're very similar.
• 1:04:56
  BDD has a very. Like technical definition, it's
• 1:04:59
  a very specific process where specs are kind of just like we
• 1:05:02
  write a spec, we all agree, then we implement into, you know?
• 1:05:05
  And BDD actually has like a. There's like a formal language
• 1:05:08
  for for the tests. It's called Kirk and.
• 1:05:11
  That Sam does not want. Yeah.
• 1:05:14
  So this will, this is a very heavy overhead and it got thrown
• 1:05:18
  out. But you think there's a
• 1:05:19
  research? Because you can generate it now.
• 1:05:21
  Some some companies have gone through like some puppies are
• 1:05:24
  all in on that for a long time. That's how they do their
• 1:05:26
  development process. But most, I would say like most
• 1:05:29
  kind of agile ish companies do not do.
• 1:05:31
  That little language, like a lot of Java shops will still.
• 1:05:34
  Follow TDDA lot of said agile sorry for the Oh my God I'm AC
• 1:05:40
  sharp guy so I we'll. Get fisticuffs here in a second
• 1:05:42
  when we start talking about languages.
• 1:05:44
  I think it's going to have a resurgence because it it just so
• 1:05:47
  happens to be exactly what you would want to have if you
• 1:05:51
  hypothetically had a, let's say you hypothetically had an army
• 1:05:56
  of 1000 junior engineers who could build whatever you want as
• 1:05:58
  long as you gave them really good specs and tests to follow.
• 1:06:02
  What's that kind of what you would want?
• 1:06:04
  So there's a version of it, which is why I think it needs a
• 1:06:07
  new name. I would agree there's a.
• 1:06:08
  Version of it that I think will become a very popular way of
• 1:06:13
  having a team do development. What I'm seeing broadly, our
• 1:06:16
  team sizes decreasing and team overall number of teams
• 1:06:22
  increasing and the specialization of what that team
• 1:06:25
  is responsible for becoming more like encompassing in its domain.
• 1:06:32
  And then having these kinds of processes put in place before
• 1:06:36
  the team is formed so that they, this is this, they come in and
• 1:06:43
  this, let's call it, you know, Ninja offset team or Tiger team
• 1:06:47
  or whatever you want to call it is specifically geared with a
• 1:06:51
  practice and, you know, set of protocols in place to accomplish
• 1:06:57
  what their remit is. And I think that will become
• 1:07:01
  more popular because things the turn around that we're seeing,
• 1:07:06
  it's just it's nine months, tasks that used to take a team
• 1:07:11
  of six are team of two for two weeks.
• 1:07:14
  You know, some of the productivity gains in a certain
• 1:07:17
  areas are pretty incredible. I do think that if we do call it
• 1:07:21
  Behavior driven development that a lot of people will continue to
• 1:07:24
  get confused. There could.
• 1:07:27
  Maybe it needs a sexier name or just a open open call for better
• 1:07:31
  names. Yeah, pace right in the comments
• 1:07:33
  below. Should we end on a spicy
• 1:07:36
  question? Yes.
• 1:07:37
  What is it? What is it?
• 1:07:38
  As long as I don't have to answer.
• 1:07:40
  I think you should. I think you should answer.
• 1:07:41
  I think we'll see if this one's spicy enough and it's not.
• 1:07:44
  If it's not, I can bring them even more.
• 1:07:47
  Do you think in five years will will there be more or less
• 1:07:54
  software developers in the world than there are today?
• 1:07:56
  That's like jalapeno level spicy.
• 1:07:59
  What was the other one? Give me some habanero.
• 1:08:01
  I think that we've already reached AGI.
• 1:08:04
  Oh, that wasn't a question. Agree or disagree?
• 1:08:10
  Oh God. Do you think that we've already
• 1:08:12
  reached AGI? I say yes, no.
• 1:08:16
  It's a trick question because it depends on how you define AGI.
• 1:08:19
  Yeah, exactly. There was a really great
• 1:08:22
  quotation where Altman asks I, I, I forget, I think it's a, a
• 1:08:29
  very senior physicist that does something like Cambridge.
• 1:08:32
  I, I actually don't remember, but the, the, the physicist is
• 1:08:36
  saying something like, no, we're not even close, right.
• 1:08:40
  And then Altman says if an agent was able to discover and come up
• 1:08:50
  with and pragmatically go through and prove and write
• 1:08:55
  papers on that provably showed a new, you know, physical theorem
• 1:09:00
  or a a new field of physics or a new, you know, some, I think
• 1:09:05
  it's the example he uses is like quantum gravity or something
• 1:09:09
  like that. You know, it's something
• 1:09:11
  something extremely difficult to prove and that we haven't done
• 1:09:14
  so yet. Right.
• 1:09:15
  And the physicist just says like, yeah, if an agent did
• 1:09:19
  that, then yes, I would consider that AGI.
• 1:09:22
  And I like the example because it's so outlandishly extreme.
• 1:09:26
  It proves something is that I think we're always going to push
• 1:09:29
  the goal posts forward on that. The truth is that we don't want
• 1:09:32
  to admit, no matter what, at any point, that we are equal to the
• 1:09:38
  thing we've made. If we consider ourselves to be
• 1:09:43
  generally intelligent, we are never going to admit that we've
• 1:09:46
  made something as intelligent or generally as intelligent as us,
• 1:09:50
  because inherently I think most of us are conceited a little
• 1:09:54
  bit.