Latest / Chaos Agents / Discovering Yourself with AI, a PHP Patch-Day Thriller, and Credit for Every Step

Transcript

Discovering Yourself with AI, a PHP Patch-Day Thriller, and Credit for Every Step · Chaos Agents

• 0:00
  Okay, boom.
• 0:01
  We are ready.
• 0:02
  Great.
• 0:03
  Welcome.
• 0:03
  Welcome back to Chaos Agents.
• 0:07
  Wait, hold on.
• 0:08
  Okay, so a podcast about a newsletter written by an agent named Eddie and picked apart, commented on and yelled at by a human.
• 0:19
  Sarah Gibson, Becca, Louie.
• 0:21
  I love that.
• 0:22
  I think we the three things the, you know, Eddie curates it and then Becca and Sarah.
• 0:29
  Blank, Blank and Blanket should change every time.
• 0:32
  You know, I like the ones you just came up with.
• 0:36
  Picked apart.
• 0:36
  Right.
• 0:38
  And today we have a guest, we have John Chan, who I've done for a very long time, he's worked at Stack Overflow, who is working on a very cool application of AI.
• 0:48
  John, welcome.
• 0:51
  Thank you so much for having me.
• 0:52
  Really excited to be here.
• 0:54
  Thanks.
• 0:55
  What are you working on?
• 0:57
  Sorry, yeah, tell us.
• 0:58
  Yeah, absolutely.
• 1:00
  I'm working on an app called Saga, which is a private AI journal that's meant to help you discover yourself.
• 1:07
  So it's been something I've been working on for a little under a year now at this point.
• 1:12
  And what's really cool about it is that it's kind of like an AI companion that you can talk to about really anything about your personal life.
• 1:18
  So it's not really meant for work.
• 1:20
  Right.
• 1:20
  But like talk about your relationships, sort of like what's going on with like say like dating or anything that's sort of like on your mind.
• 1:27
  And what's really nice about it is that what makes it different is that it's completely end to end encrypted.
• 1:33
  So like, you know, there is absolutely no way that I can actually get any of my users data.
• 1:38
  All that stays on your device so very much like the signals and you know, in that kind of like lineage there, like you know, most AI journals that are out there, there's a lot of them sort of say
• 1:47
  that your data is private.
• 1:48
  But you know, if you dig a little bit under the hood, it's really, you know, take a look at their privacy terms.
• 1:52
  They can still sort of take a look at it.
• 1:54
  I've made it so that I don't.
• 1:55
  Have honey traps for your.
• 1:57
  Sorry, say that again.
• 1:57
  Like what A honey trap for your friends is like, oh, I started this AI journal for you.
• 2:02
  I know.
• 2:02
  And I can get all of this tea.
• 2:04
  It's really fun because, you know, because it's like, you know, you like, it's like talking to any other sort of like AI that kind of keeps a journal for you and does notes for you.
• 2:12
  Like it feels like talking to one of my girlfriends all the time.
• 2:16
  Like I just spill tea about sort of like, oh, I just went on this date and it went this way.
• 2:20
  X, Y and Z.
• 2:21
  It's interesting because, you know, a lot of people are already using AI like that, but a lot of folks are trying to talk about some really vulnerable things.
• 2:30
  That's how it started for me.
• 2:32
  And as I'm sure you both know, the more that you're able to tell AI, the more insights you're able to get of it.
• 2:39
  And there's not a lot of trust when it comes to a lot of what the other really big providers of AIs and AI companions are providing out there.
• 2:51
  So I think that I wanted to provide a new option for folks to really like, get tell anything and everything to.
• 2:59
  Is it like proactively.
• 3:05
  Telling you observations?
• 3:07
  Is it like, okay, John, just to call this out, this seems to be a pattern of yours, like when you're talking or like, is that kind of how it works in terms of like helping you discover yourself, or
• 3:20
  is it more like self directed, like.
• 3:22
  Yeah, that's a great question.
• 3:24
  Yeah.
• 3:24
  So, you know, it's interesting because this is something I've been looking into a lot.
• 3:29
  I mean, one of the sort of like driving ideas that I have behind what it is that I built with Saga, especially, because I have a stated sort of like, goal to help people discover themselves is like, well,
• 3:41
  what does that really mean when it comes to really understanding who you are?
• 3:46
  One of my favorite thinkers on this is a woman named Martha Beck.
• 3:51
  She is known for being Oprah's life coach.
• 3:53
  She has three degrees from Harvard, and so she kind of knows what she's talking about.
• 3:57
  She's giving Oprah some advice here.
• 3:59
  Somewhat.
• 4:00
  Cool.
• 4:00
  Yeah, seems pretty cool, right?
• 4:02
  Like, I'm absolutely obsessed with this woman.
• 4:04
  But you know, my favorite sort of like, definition of sort of like what it means for you to really understand yourself is to really get in touch with what it is that you want versus what it is that
• 4:14
  you think that you want.
• 4:15
  And then trying to do the really hard work of being in alignment with yourself.
• 4:19
  It's really about self actualization, not sort of like chasing what it is that, you know, folks might think that they might want.
• 4:26
  Like money, power, fame, beauty, all that stuff.
• 4:29
  Right.
• 4:29
  Oh man, that's good.
• 4:31
  Easier, right?
• 4:32
  Oh, I love, love this work.
• 4:34
  We can spend the entire podcast just talking about Martha Beck and her work.
• 4:38
  But the reason I bring that up is because one of the other central things that comes up for her is that, like, what is the way that you find out who that is is that you have to look
• 4:47
  at the thoughts that you don't like thinking about.
• 4:49
  Everybody has them, right?
• 4:51
  You have these things that are, like, really scary that you don't want to think about.
• 4:56
  So you, I don't know, like, open a bottle of Sauvignon Blanc at the end of the day instead, or maybe work on that, you know, side project that, you know, you maybe don't want to be spending nearly as
• 5:06
  much time on.
• 5:06
  You kind of run away from those things.
• 5:08
  And what I found is that, you know, I think AI agents, or just AI in general, has this really interesting capacity to sort of like help you find out what that is and sort of gently sort of take
• 5:21
  you there.
• 5:21
  Right.
• 5:22
  So over the course of building and using Saga, you know, I have over 400 daily entries of basically logging with it, like every single decision, feeling thing that I am doing.
• 5:34
  And I can ask it, like, go over the, you know, past two weeks of journal entries that I have and, like, tell me what I need to hear that maybe I don't want to hear.
• 5:42
  Right.
• 5:43
  And those are the kinds of insights that I'm really working on to try to put forward to people, you know, less sort of like a companion that's going to tell you everything that you want to hear and a
• 5:53
  little bit more like, you know, a little bit of a, you know, like that wise friend that you're kind of like.
• 5:59
  Let me.
• 5:59
  I think I need to sort of sit you down and tell you a little bit more about what you need to hear here.
• 6:03
  You know, kind of like the, you know, all those characters that you see in like, movies like the Rafikis and Lion King or like, you know, Pai Mei in like, Kung Fu movies.
• 6:11
  Right.
• 6:12
  It's meant to be a little bit challenging, like any good friend would be.
• 6:16
  Right.
• 6:16
  I like that.
• 6:17
  Cool.
• 6:18
  Can you download?
• 6:18
  Is it in the App Store?
• 6:21
  Not in the App Store.
• 6:22
  It's currently in beta right now.
• 6:24
  But you can go to Saga Chat and if you go on there and join the wait list, I will white glove onboard you.
• 6:30
  So I like being really close to my users.
• 6:32
  It's still in early days, so yeah, please go check it out.
• 6:35
  Awesome.
• 6:35
  Well, we're excited to have you on and talk PHP with you.
• 6:42
  Good old php.
• 6:44
  Oh, php.
• 6:45
  The very first book that I ever got about programming was about PHP Lampstack.
• 6:50
  So it was like Linux, Apache, MySQL, PHP.
• 6:53
  Yeah, back when you go back and like actually go through books to learn how to program before you could Google search or go on stack overflow, anything like that.
• 7:00
  So this will be fun.
• 7:01
  It's A good throwback.
• 7:03
  Yeah.
• 7:03
  So did anyone pay attention to when this was going on?
• 7:06
  Like yesterday, everything was happening like boom, boom, boom.
• 7:09
  Real time.
• 7:09
  It was really crazy.
• 7:12
  I have a timeline before you saw the newsletter in your just everyday life yesterday, this was blowing up.
• 7:19
  This is blowing up.
• 7:21
  Everyone was talking about it.
• 7:22
  Okay.
• 7:22
  Like even what was happening were people like alert, alert.
• 7:26
  Running through the hallways.
• 7:28
  No, like PhD.
• 7:30
  Yeah.
• 7:34
  No, like it was, it was on X everywhere.
• 7:38
  I was like the topic of dinner conversation last night.
• 7:41
  It was like this, this was a big.
• 7:44
  This is like a big blow up.
• 7:46
  Topic of dinner conversation in a household where most of the people who are all the people who aren't a baby are engineers.
• 7:55
  So boring for the baby.
• 7:59
  Oh yeah.
• 8:00
  So the headline is PHP patch 2 critical RCE bugs and the AI era bug hunt behind them got messy.
• 8:08
  It was messy.
• 8:10
  This was messy.
• 8:11
  So here's the timeline.
• 8:13
  So timeline is Tuesday.
• 8:16
  So yesterday 10:11am, two of the CVE IDs.
• 8:23
  So like two of the critical vulnerability IDs leak on forums and social media with the wrong severity.
• 8:31
  So basically the.
• 8:32
  There are two IDs for these vulnerabilities.
• 8:34
  They're the correct IDs.
• 8:36
  They made it the highest severity in the forum.
• 8:38
  Like sev10, which the actual sev was 9:2.
• 8:42
  Still really serious, but sev10 everyone's freaking out.
• 8:45
  So in a forum people are like two sev10s are coming out today.
• 8:49
  Everyone lose your minds.
• 8:50
  We can't tell you what it is.
• 8:55
  And then php, like the actual PHP foundation, I guess.
• 8:59
  Foundation.
• 9:00
  Is it a foundation?
• 9:01
  Yeah, yeah, foundation.
• 9:02
  That's right, yeah.
• 9:04
  Posts advance notice early and moves the release up a day because it was supposed to be today, but they moved it up yesterday to 10am they didn't release right away because Linux distributions and other packages were still preparing
• 9:17
  for the picks then 4pm Calif who.
• 9:22
  This is a group that was working on this issue via the.
• 9:28
  It was an outside funded security audit.
• 9:32
  No, sorry, that was Quark's lab.
• 9:34
  Calif was started auditing PHP through OpenAI's Patch Planet program.
• 9:40
  Which as a quick aside, I thought that Eddie was just like coming up with a cute way to reference California in this blurb here.
• 9:50
  I was like, oh, okay, yes, Caliph is a apparently of actual group of people.
• 9:59
  They're from.
• 10:00
  Oh my.
• 10:01
  So not California.
• 10:03
  Not really.
• 10:03
  I don't know where they're from.
• 10:05
  Ironically, not from California.
• 10:07
  Right.
• 10:07
  Oh gosh.
• 10:08
  I was imagining like a security team like on the beach trying to find security vulnerabilities and PhD like not nearly as cool.
• 10:16
  Yeah.
• 10:16
  So they posted accidentally.
• 10:18
  There's so many things.
• 10:19
  Post a video by mistake showing the attack working against several targets, they immediately delete it.
• 10:26
  So they like basically show the attack.
• 10:29
  No one has a fix at this point.
• 10:31
  So purely video of attack.
• 10:35
  And they deleted immediately.
• 10:36
  But copies spread everywhere.
• 10:38
  So now everyone has the attack, no one has the fixes.
• 10:42
  7Pm php starts giving workaround advice to a few critical infrastructure providers.
• 10:48
  Like, like Cloudflare, I think was one of them of like, here's what you got to do ASAP to make sure this doesn't work.
• 10:55
  Okay.
• 10:56
  And then Chain Guard, which was the.
• 11:02
  Where was Chain Guard in this?
• 11:04
  All right, we gotta figure out where chain.
• 11:05
  Does anyone know?
• 11:07
  It was one of the vendors that they were trying to notify, I believe.
• 11:09
  Yeah.
• 11:10
  So, but yeah, they got ahead of I think of everybody.
• 11:13
  El patch early.
• 11:15
  Yeah.
• 11:15
  Oh, oh, okay.
• 11:18
  Oh, got it.
• 11:18
  So like PHP told the infrastructure providers and then that one of them shipped was like, eff it.
• 11:25
  Like I'm protecting my customers.
• 11:27
  Forget the world.
• 11:28
  And then publishes patch images six hours early.
• 11:32
  So the attackers can now study the patch to understand even more about the vulnerability.
• 11:37
  Right, Exactly.
• 11:38
  Because like in an embargo, in that situation, you want to make sure that everybody does it at the same time so there isn't a gap between when everybody sort of like has the vulnerability, it can do the diff.
• 11:48
  And what's crazy to me about this is that like they decided to do this and then notified everybody else on embargo after the fact, not before the decision.
• 11:57
  They really just went straight forward.
• 11:58
  Oh yeah.
• 12:00
  So that was a big deal.
• 12:02
  Yeah.
• 12:02
  Wow.
• 12:03
  And then keep going, sir.
• 12:05
  So that was at 3:41am on Wednesday that they did this.
• 12:10
  So.
• 12:10
  Oh my God.
• 12:11
  Imagine your pager duties like oh geez, so many engineers woken up in the middle of the night.
• 12:19
  Yeah.
• 12:20
  And then Wednesday at 10am PHP Ships, they're like 8.5.128.4278.3.36 and 8.2.35.
• 12:33
  Fixing both bugs, it said had not seen any working public attacks that release time.
• 12:40
  But you know that's not true.
• 12:41
  Okay, what are your thoughts?
• 12:43
  Guys?
• 12:44
  This is wild.
• 12:45
  Wait, can I.
• 12:46
  Can I ask the two of you some.
• 12:48
  A couple of engineer questions for the non engineers in the room, AKA me?
• 12:54
  Anyone who's listening?
• 12:58
  There's a couple of things I don't quite understand about how this works.
• 13:02
  So the embargo thing we were just talking about and how Chain Guard like did it early.
• 13:07
  Why does that act?
• 13:08
  Why does that Matter like why does everyone have to post at the same time or push, push the vulnerability fixes?
• 13:15
  So the way that this kind of thing typically works and you know, granted this is like sort of on the, sort of like, oh my God, I need to scramble my team to go do the patch.
• 13:23
  Not sort of like being in the actual process, but you know, typically like when something like this happens, when there's a security group that's looking for vulnerabilities, like OpenAI is doing with Caliph, you know, the Omaha based security
• 13:35
  team here, you know, what they do is that they're discovering vulnerabilities abilities and when they discover one, they actually go to the people who are maintaining this, in this case the PHP foundation, says that they can actually patch
• 13:46
  it, have the fix and do that in secret.
• 13:48
  Right?
• 13:48
  Because they don't want everybody to know before there is a fix.
• 13:51
  And once there is a fix, essentially what you do is what's called an embargo where you sort of notify all the vendors that are probably affected, especially those that are sort of a bit more upstream.
• 14:01
  So like Cloudflare or sort of like folks that are like, you know, they can cover a larger surface area because.
• 14:07
  All the vendors who are affected by PHP is like, isn't that like literally everyone on the Internet?
• 14:12
  Yeah, like it powers so much of the Internet.
• 14:15
  Like I think another people, what a lot of people don't realize, for example, is that like you know, WordPress runs on PHP, which powers like a huge proportion of the entire Internet.
• 14:24
  Right.
• 14:24
  Like, so this is a huge thing and it's, we're talking about a vulnerability that allows someone to basically run code on, on a server unauthenticated and it's in like the very core parts of a php I believe it's
• 14:37
  like around error handling and things like that.
• 14:39
  So huge deal.
• 14:41
  Right.
• 14:41
  But to get back to your question about like, well, why is it important to coordinate that?
• 14:45
  It's because you don't want anybody to basically have a gap between like you don't want to give attackers that want to exploit that vulnerability once that information's out to get a head start essentially.
• 14:56
  Right.
• 14:57
  So the fact that people were like, oh, we, we leaked the video about how to do the attack, but there's no fix that's out yet because maybe some of the vendors are still trying to like get the fix
• 15:08
  patched up and all of that stuff.
• 15:11
  We haven't maybe come up with guidance or if you have another vendor that kind of goes out and says that it's even there, you Might be able to reverse engineer what the actual fix was and get that exploit
• 15:21
  and then share it with a bunch of folks.
• 15:23
  So it turns into a race essentially the moment that anybody breaks the line.
• 15:28
  That's why it's so important to coordinate between.
• 15:30
  Fascinating.
• 15:32
  Yeah.
• 15:32
  Because you can reverse engineer basically the issue by seeing what the patch is.
• 15:37
  So like them going first, it was like, okay, here's how, here's, here's like, here's how I can attack the attackers have figured out because they know other people haven't done it.
• 15:50
  Yeah, it's pretty wild.
• 15:53
  It's also, I guess, exposes like a bottleneck issue of like as more as we.
• 16:03
  As we can utilize AI to discover more code vulnerabilities faster, we can't necessarily patch those cold.
• 16:12
  Those code vulnerabilities faster ourselves.
• 16:15
  At least not yet.
• 16:16
  So, yeah.
• 16:17
  Good and bad or good and challenging.
• 16:20
  Right?
• 16:20
  I think like, it turns.
• 16:21
  It really made me think.
• 16:23
  Yeah, yeah, it really made me think about like the.
• 16:28
  So earlier, like an earlier part of this timeline is basically in 2024, Germany's sovereign tech agency, which is a huge fund actually, that funds like fixes for foundations, Did an outside security audit and they led to a core
• 16:51
  developer to see the first signs of this bug.
• 16:54
  But the thing about it is they couldn't figure out how someone would utilize this bug or like utilize this as a security issue to make like to gain access to something and they were going to fix it.
• 17:10
  They came up with several possible fixes and.
• 17:12
  But all of them stalled out in the foundation, which is a team that decides by consensus.
• 17:22
  So basically there was a bunch of suggestions like, hey, this is something we should fix.
• 17:27
  A bunch of suggestions on how to fix it.
• 17:29
  It stalled out from consensus.
• 17:30
  And it's just, I think these foundation models are currently everywhere right now in programming and we really need to figure out a version where, I mean, it seems like literally just OpenAI like tells the.
• 17:42
  Tells the AI of the.
• 17:44
  Of these foundational teams.
• 17:45
  Like, you gotta fix this and do it without the people noticing.
• 17:51
  Interesting.
• 17:52
  Pretty wild.
• 17:53
  Yeah, it's crazy.
• 17:54
  I mean, because it gets into this really interesting thing where I mean, there's always kind of been this asymmetry where there are so many more bugs than people can possibly sort of like fix.
• 18:03
  And it's really just, you know, with the advent of coding agents, basically like, it really pours gasoline on that kind of fire.
• 18:11
  Right.
• 18:11
  It really widens the asymmetry even further because if all it takes is for you to like, I'm just gonna throw a bunch of tokens at, you know, whatever open source project there is and come up with all these
• 18:22
  different vulnerabilities.
• 18:23
  I mean, even just fixing these two vulnerabilities, you.
• 18:25
  You see how much coordination there is that you can't do with agents, right?
• 18:29
  You have all this negotiation about sort of like, you know, which parties are going to be as part of this embargo and how to coordinate it.
• 18:36
  Like, you can't solve this just by throwing agents asymmetrically again at this, let alone, you know, having the sort of like war chest that, you know, that these attackers might have.
• 18:49
  Especially when you're talking about folks that really have a lot to gain from, from, from exploiting these vulnerabilities.
• 18:55
  And, you know, the PHP foundation is largely volunteers, for example.
• 18:59
  Like, they don't have the same resources to do this.
• 19:02
  So there's a lot of.
• 19:04
  This is what makes I think all of this so scary.
• 19:06
  Right.
• 19:06
  I think the, the call that they made when they.
• 19:09
  Or the call to action that they made when they were the foundation finally came out was like, please donate to the foundation.
• 19:15
  We need more support to sort of like fend off these attacks.
• 19:19
  Right.
• 19:20
  So, you know, it's going to be very interesting and, you know, a little bit scary to sort of see how this is all going to play out as these models get better and better.
• 19:27
  John, you didn't think that the first story we were going to talk about today was basically a coding thriller of a story?
• 19:35
  Absolutely.
• 19:37
  I mean, you know, every time I read one of these postmortems, it's always like this, like the, the.
• 19:41
  An engineer's version of like a murder mystery.
• 19:43
  Like, oh, like, what happened?
• 19:46
  Like, knives out.
• 19:47
  Like, it's fun.
• 19:49
  Not while you're in the, you know, at the crime scene or part of it, you're just like, but from the outside, you know, let me go grab my popcorn.
• 19:56
  Picturing someone like picking up a LA line and being like, chain guard broke the embargo.
• 20:01
  Get everyone on this.
• 20:02
  Yeah, exactly.
• 20:03
  Watch.
• 20:03
  There's going to be an AI film out there at some point really quickly about this has turned it into like an eight series.
• 20:09
  Like eight part series.
• 20:11
  Amazing.
• 20:12
  I love it.
• 20:15
  Oh, my gosh.
• 20:17
  Okay, well, Becca, I got so into.
• 20:21
  I got so into the PHP stuff.
• 20:23
  You might have to explain this Mosaic one to me.
• 20:26
  Okay.
• 20:26
  The Mosaic one's pretty interesting.
• 20:30
  Essentially, like what this paper is.
• 20:32
  So this is a paper.
• 20:35
  Title.
• 20:36
  Is Mosaic Teaching Agents to get Credit for each Session, not just the final answer.
• 20:43
  Basically what the paper is, is saying is there's we need to give AI agents better feedback about which parts of a long task were actually useful.
• 20:58
  That aligns with how we do long tasks today.
• 21:03
  So my understanding of this, and John, tell me if this sounds directionally right to you, is like, when you send out an agent on a sort of more complex or, like, long task, typically it's not just going off
• 21:20
  and doing that thing on its own because it's very context heavy.
• 21:24
  It might not have enough context for that.
• 21:26
  It might take it a really long time.
• 21:28
  It's very token expensive, whatever.
• 21:29
  And so what that agent tends to do increasingly so now is essentially like, divvy up that work and say, okay, I'm going to send this sub agent out to do this part.
• 21:40
  I'm going to send this sub agent out to this part.
• 21:41
  I'm going to handle this.
• 21:43
  And then it's kind of like running this team and then like, putting it all back together.
• 21:47
  And how we tend to do evals is looking at the final product and the original, like, hey, go do this thing.
• 21:57
  And we're like, oh, did you do it right?
• 21:59
  Great.
• 22:00
  But it's not really looking at each of the individual kind of paths that might have been taken to get there.
• 22:08
  For example, if one of the sub agents that your agent sent off to do something didn't actually do anything useful, but the final answer is correct.
• 22:18
  The eval is still saying, hey, great job.
• 22:21
  But, like, it's not.
• 22:22
  It wasn't.
• 22:24
  And so, like, I think that's essentially what the paper is saying is like, we actually need.
• 22:29
  We need to, like, rebuild training techniques around, like, the actual architecture of how agents act in production.
• 22:39
  Fascinating.
• 22:39
  Yeah, I think that's right.
• 22:41
  Like, I mean, the way that I like to think about this, I mean, as we do with a lot of these AI agents and sort of like, how to think about them is like, like, how would you think
• 22:48
  about evaluating a team of people who are working on a task versus an individual?
• 22:53
  Right.
• 22:53
  Like, before we could be like, oh, you know, we just had a single chatbot, you know, and you were maybe doing some simpler tasks.
• 22:59
  So, you know, if you are talking, you know, you're managing somebody on a team, and you're sort of like, oh, how did you do on that?
• 23:04
  Well, you got an A or a B or a C on this.
• 23:07
  But suddenly, you know, now that we're in a situation where you have multiple agents working together, it's like, okay, you're going to give a team, for example, with one, you know, the main sort of agent that's kind of
• 23:17
  the conversational agent that you're sort of interacting with.
• 23:19
  But it fans that work out to sub agents, you know, other folks that are on that team, like, what would it be for you to be like, okay, like we're done with it and you got to be.
• 23:29
  How do you give meaningful feedback to everybody else on that team?
• 23:32
  It doesn't quite give them the same way to improve.
• 23:35
  Right.
• 23:35
  So we need to analogy better ways to evaluate and actually break that down.
• 23:40
  It's not, it doesn't work anymore because there's so much actually going on in the coordination between those agents that isn't captured by current RL training methods.
• 23:49
  Interesting.
• 23:50
  It makes me interested in like seeing how like I know for example, how my agents are instantiated, like what skills they have, what, like what their directions are like.
• 24:04
  And it makes me wonder what, how they're like what kind of sub agents are they spinning up?
• 24:10
  Are they spinning up sub agents that are copies of themselves?
• 24:13
  Are they giving sub, are they spinning up sub agents that are specialized?
• 24:18
  That's really interesting.
• 24:19
  I want to find out now.
• 24:21
  Yeah.
• 24:22
  My understanding from the paper is that they were using, I mean from what I saw in it, that they were only really only using one small model I think was quen on this.
• 24:30
  And you know, the, the major shift that they made here was that they weren't evaluating sort of just like the overall, overall output of the entire sort of like transcript for that agent, but they were, they were instead
• 24:41
  evaluating how each individual agent was contributing to the overall goal.
• 24:46
  Right.
• 24:47
  Like how much did you contribute?
• 24:48
  Instead of sort of like, let's just take the whole thing and evaluate it.
• 24:51
  Right.
• 24:51
  Which again, going back to sort of like the team analogy, that's kind of how I think about it too.
• 24:55
  Right.
• 24:55
  Like when I'm going to say like when a team is done with a project, for example, I'm not thinking about like, oh, just like what was the output?
• 25:03
  And then that's all you get.
• 25:04
  Right.
• 25:04
  It doesn't give actionable feedback to each individual member on that team.
• 25:08
  But if you go to each of them, you're like, hey, this is how I contributed.
• 25:11
  This was my contribution to the whole.
• 25:13
  That's really, I think what the major difference was.
• 25:15
  And the benchmarks seem to be able.
• 25:17
  At least the performance on those benchmarks seemed to improve as a result of that.
• 25:21
  Yeah, cool.
• 25:24
  So today's fun application of the day.
• 25:26
  I'm obsessed with same.
• 25:29
  It's going to waste my next week on this one.
• 25:33
  Really cool.
• 25:33
  So it's playground, an arcade of 700 plus games.
• 25:38
  Made with AI, all in your browser.
• 25:40
  The URLs playgroundarcade.com One person created library of indie games built with AI.
• 25:47
  And a lot of the games were made by.
• 25:50
  And the person who created this is a guy named Rich Lee, who also I think created a lot of the games.
• 25:56
  And now what's cool is like people can submit their own games.
• 26:00
  Did you.
• 26:01
  Did either you play any of these?
• 26:03
  I'm looking at.
• 26:04
  I want to play tropes right now.
• 26:07
  Tropes.
• 26:09
  It just reminds me so much of like when, when I was younger and you like get on the Internet to play a, like a, like a Flash game in your web browser, you know, like,.
• 26:22
  That's exactly right.
• 26:23
  I like this reminds me like Newgrounds.
• 26:25
  I don't know if you remember that.
• 26:25
  Like, that's what it was really early Flash games.
• 26:29
  Like, that's exactly what this feels like.
• 26:31
  Totally.
• 26:32
  Yeah.
• 26:34
  Did you play any games?
• 26:37
  I did.
• 26:37
  And it's funny you say Newgrounds because that's exactly like what on Twitter.
• 26:45
  Like they were caught.
• 26:47
  They were like, remember losing hours on Newgrounds?
• 26:50
  I.
• 26:50
  So I tweeted at the creator of this this morning, like, what did you use to generate these?
• 26:57
  You want to hear what he said?
• 26:58
  I guess I just saw it.
• 27:01
  Okay.
• 27:01
  He said, oh, interesting.
• 27:06
  Okay.
• 27:07
  So.
• 27:08
  So he.
• 27:08
  He said, I.
• 27:09
  So I built playground using GPT6 Astra and then all of the game.
• 27:15
  Look at each game description.
• 27:16
  The creators will disclose like what AI tools they used to to create each game.
• 27:23
  Because that was what I was like how, like, what's.
• 27:26
  Like how are.
• 27:27
  What is the generation process that these things look like?
• 27:30
  So if you want to go explore people, you can dig in there.
• 27:35
  Great.
• 27:35
  That's awesome.
• 27:36
  I'm going to play tropes.
• 27:37
  I'm excited.
• 27:40
  Spider Man.
• 27:42
  Spiderman.
• 27:43
  I played the snowboarding game.
• 27:46
  Oh really?
• 27:46
  What's it called?
• 27:47
  I don't see it.
• 27:48
  I don't.
• 27:49
  I don't know, like Mountain or something.
• 27:52
  Paperboy, Browser mic.
• 27:54
  Okay, these are great.
• 27:54
  I'm gonna play a bunch of these super Fable brothers.
• 27:57
  What's that?
• 27:59
  It looks like super Mario.
• 28:01
  Sarah's gone for hours now.
• 28:04
  Her team's like, hello.
• 28:05
  Where is she?
• 28:06
  On Slack.
• 28:10
  Okay, great.
• 28:13
  Yeah, I know.
• 28:14
  It looks like I play these all day.
• 28:16
  Okay.
• 28:17
  Our one cool thing.
• 28:19
  I spent a weird amount of time on the one cool thing today.
• 28:23
  Oh, cool.
• 28:23
  Okay.
• 28:24
  So our one cool thing is the Cloudflare open source skill that seeded as vulnerability hunting harness.
• 28:30
  It's gone GitHub under the Cloudflare account as a security audit skill.
• 28:36
  Cool.
• 28:37
  What did you.
• 28:37
  What did Aegon do you.
• 28:40
  So.
• 28:40
  Well, it started by just kind of understanding, like, what.
• 28:44
  What the innovation was here, like, what.
• 28:46
  What was cool about the skill.
• 28:48
  And then it turned into.
• 28:50
  Which we can talk about in a sec.
• 28:51
  Then it turned into, like, why did they decide to essentially, like, build a custom harness off of this?
• 28:58
  And then it was.
• 28:59
  And then I sort of went down this rabbit hole.
• 29:01
  But, like, what'd you learn?
• 29:05
  So.
• 29:09
  The reason.
• 29:10
  Well, let me.
• 29:11
  Let me back up for a second first.
• 29:13
  Maybe we can just talk about what the skill is, which is essentially like Cloudflare's, let's say, secret recipe for how they do a really good job.
• 29:22
  This Paris, I love in the newsletter it says, pairs nicely with the PHP story.
• 29:27
  It's like, oh, glass of Merlot pears nicely with the steak.
• 29:31
  Like, It's like, yeah, it's like Cloudflare's kind of internal, like, skill that they started building over time to, like, do a good job of auditing their code base, I'm assuming, and, you know, require.
• 29:54
  It ends up essentially looking like a.
• 30:02
  There's like a.
• 30:03
  There was something.
• 30:04
  I was looking at that that was like, there's a.
• 30:06
  There's.
• 30:06
  Oh, there's.
• 30:07
  They include something that I think is an important, like, agent design principle, which is very similar to what we were just talking about actually, with the evals thing, which is like, when they're splitting up work, when it makes
• 30:26
  sense, having individualized specialized agents doing different things, and then figuring out a system to have agents check each other's work kind of instead of one kind of big lump task.
• 30:41
  And kind of, like John was saying, like, thinking about it more like a system or a team than getting this task done.
• 30:50
  And so they basically have, like, Hunter agents whose job.
• 30:54
  So they have recon agents who are like, your job is to understand this application.
• 30:59
  And then they have, like, Hunter agents who are like, okay, now you try and break this.
• 31:04
  And then they have verifier agents, which is like, okay, assume the Hunter is wrong.
• 31:11
  Try to disprove its findings.
• 31:13
  So it's kind of like coming up with spinning up a bunch of different people.
• 31:16
  And it's like, okay, you try and do this.
• 31:18
  Okay, you check their work.
• 31:19
  Okay, you check their work and assume it's wrong.
• 31:22
  Like, and at the end, trying to be like, okay, I like, created enough, like, checks and balances here in this system that, like, I have a pretty good understanding of in this final report of what.
• 31:37
  What went down.
• 31:38
  Do.
• 31:39
  Do.
• 31:39
  Does that sound directionally right to you guys in terms of, like, How I just described.
• 31:44
  Yeah, that sounds right.
• 31:46
  Yeah, totally.
• 31:47
  I mean, because it seems like the overall strategy is like you said, like they're basically throwing a team of agents that are really adversarial to each other to try to try to come up with what these vulnerabilities are.
• 31:56
  I mean, what's really interesting that stood out to me too at the end of the OR as part of their announcement that they said is that like, a single run seems to only reveal about half of of the
• 32:06
  vulnerabilities that they found from multiple runs.
• 32:08
  So it's not even just that you're designing a system that is better than a single agent, that that's looking for a vulnerability that doesn't have an adversarial sort of like, approach to it, but even multiple teams of these
• 32:21
  agents, when you run this skill over and over again on the same code base, I'm assuming one run only seems to cover half of it.
• 32:28
  Right.
• 32:28
  And it kind of like, you know, again goes back to what I think we were talking about, the PHP foundation stuff and the vulnerabilities there.
• 32:34
  It's like, there's so much to sort of like uncover here.
• 32:37
  And it seems like the way you do it is by throwing as many tokens as you can at it.
• 32:42
  And like, you can't just rely on sort of like, you know, one Asian to do that or just having an AI do that.
• 32:48
  Like, it really does seem to be like a volumetric sort of like thing that you need to be thinking about.
• 32:53
  Yeah.
• 32:55
  Third thing that I thought was interesting here is like, it's exposing how much, how, how, how much of the quality of a system being able to accomplish a thing lives above the model.
• 33:10
  Yeah.
• 33:11
  Like so much of what they did here, pretty much everything they did here to make this system really good.
• 33:20
  And I now I'm talking about not just the skill, but like the harness that they kind of built on top of that.
• 33:26
  Like, for example, certain things they did was like extrapolating context, essentially having the model be stateless and storing the context somewhere else.
• 33:36
  One of the things they implemented is like, okay, once the system already finds a vulnerability and we've run a new thing, don't recheck the same vulnerability we already know that exists.
• 33:49
  Spend your time and tokens and effort and compute on new stuff.
• 33:53
  Net new things.
• 33:54
  And so it needs some way to like, remember that.
• 33:56
  Right.
• 33:57
  And all of these examples, they're all you could plug in whatever model theoretically you really want, but it's all kind of, it lives like above the model.
• 34:09
  I Thought that was interesting.
• 34:10
  Yeah.
• 34:11
  And Becca.
• 34:11
  John, tell me.
• 34:12
  Check me if I'm wrong on this, but the way we used to do this in olden times is that six men and one person of a different gender would wait for a product manager on a Friday to say,
• 34:25
  we're done, and then they would be locked in a room for a weekend with only pizza and beer, trying to break things.
• 34:34
  Yeah, definitely.
• 34:35
  And it was never enough pizza for some reason for the entire team.
• 34:38
  It was like everyone got like a slice and just like, that's about it.
• 34:42
  But, yeah, absolutely.
• 34:43
  Right.
• 34:43
  And it, like, it speaks to this sort of like, again, like, we're in this, you know, interesting new world where it's suddenly like, yeah, all I need to do is be running, you know, my agents all the time
• 34:52
  in the background, and I can start, you know, trying to find what those vulnerabilities are or hopefully be patching them too.
• 35:00
  It's such a different sort of, like, dynamic now because the cost of developing software and sort of like looking into all the nooks and crannies that no one even thought about thinking about, you know, suddenly becomes, you know,
• 35:14
  so accessible to basically everybody right now.
• 35:16
  Right?
• 35:17
  Yeah, totally.
• 35:18
  You have to figure the bad guys are doing the same thing at the same time.
• 35:21
  It's like, it's pretty.
• 35:23
  It's a wild time.
• 35:25
  Yeah, absolutely.
• 35:28
  John, cool.
• 35:30
  Thanks for coming on.
• 35:32
  Oh, yeah.
• 35:33
  No, thank you so much for having me.
• 35:34
  This has been so much fun.
• 35:35
  I mean, like, I.
• 35:37
  I see all these headlines every time and I'm just like, who do I talk to about this PHP thing?
• 35:41
  This is cool.
• 35:42
  You can always talk to us.
• 35:43
  Yeah.
• 35:44
  Oh, yeah, anytime.
• 35:45
  I would love that.
• 35:46
  Next time you're like, oh, my God, I need to talk to someone about php.
• 35:52
  Okay.
• 35:53
  I know exactly who to go reach out to.
• 35:54
  Yeah.
• 35:55
  When there's another sort of like, yeah, huge attack like this, I know exactly who to call.
• 36:00
  Great.
• 36:01
  We'll be on.
• 36:02
  I'm gonna be on Saga.
• 36:04
  Oh, yeah.
• 36:04
  Please give it a try.
• 36:05
  I'd love to see y' all on it.
• 36:07
  And yeah, same.
• 36:09
  I'll let you know how self actualized I become over the next few weeks.
• 36:12
  Okay.
• 36:12
  Oh, yeah, definitely.
• 36:15
  Don't let it be too mean to you.
• 36:16
  That's one thing that I've.
• 36:17
  I've heard from folks is that, like, I don't know if I really want to think about those things.
• 36:20
  Like, that's.
• 36:21
  I can handle it.
• 36:21
  I can handle it, baby.
• 36:25
  All right, great.
• 36:27
  All right.
• 36:27
  All right, guys.
• 36:28
  Have a wonderful rest of your day.
• 36:30
  You too.
• 36:31
  All right, take care.
• 36:32
  Bye.