Latest / Artificial Developer Intelligence / Claude Fable 5 is Here! Plus: Meta AI Hack, LLMs as Black Boxes, and Future of Agents

Transcript

Claude Fable 5 is Here! Plus: Meta AI Hack, LLMs as Black Boxes, and Future of Agents · Artificial Developer Intelligence

• Dan (00:00) That that always cracks me up when it's like you're doing something that's like kinda hard and then it goes, and now here's the part where you finish this. And I'm just Who's paying for who here? Rahul Rahul (00:07) hahahaha Shimin (00:09) It's that's management material or or consultancy material right there. Rahul Rahul (00:15) It's called managing up, Dan. It's in Claude's training. Shimin (00:19) Ha Hello, and welcome back to Artificial Developer Intelligence, a weekly conversation show and study session on all things AI and software development related. We go through hundreds of links and dozens of newsletters each week, so you don't have to. My name is Shimin Zhang, and with me today are my co-hosts. Dan does not necessarily have metacognitive insights into his underlying thinking process, Lasky. And Rahul, his energy drink is full of Full of c cocaine, Yadov. Hey gent, how are we doing? Dan (00:50) All right. Happy to have Rahul back. Rahul Rahul (00:51) I drink the original Coke. Shimin (00:53) Yeah, right. ⁓ On today's episode, we're gonna start, as always, with news thread mail where we're going to start with ⁓ anthropic's hotly anticipated mythos f slash fable five. And then we're gonna talk about the latest ⁓ meta AI hack. Dan (01:11) And then post processing, we're going to cover whether or not LLMs are the black box that you were promised in ML school. Is that a thing? I don't know. We'll go with it. and also the future of agents, brought to you by Rahul. Shimin (01:24) Right. then I'm gonna do a little bypent tail on a personal project of mine ⁓ called Inhabited Design. Dan (01:30) Then we're talk about the current state of AI financing in two minutes to midnight, where we deeply examine the state of the AI bubble. Deeply. Shimin (01:40) Okay. first up, Anthropic had a big announcement today. ⁓ it is June 9th when we were recording. The long-awaited Mythos five slash new code named Fable Five came out. the differentiation between the two models are that the Fable 5 has additional security. guard rails that the Mythos V does not have. So, this is their latest five series models. ⁓ the benchmarks that were provided by Anthropic looks to be yet another more or less incremental improvement on a lot of the benchmarks. a step change in some, I suppose, but especially their ⁓ the spatial reasoning for the blueprint bench two ⁓ but of course this model has the world's best ⁓ marketing campaign, right? via Project Glasswing where ⁓ banks, central banks, and ⁓ all kinds of large technology companies have been ⁓ patching their operating systems, their libraries ⁓ and their code base over the last month, ⁓ based on the security vulnerabilities found. One thing that I find particularly interesting in their system ⁓ an evaluation card is you see that if you're watching this on the screen in YouTube, there is a chart of mean calls per task over a score on a particular diamonds set of a evaluation benchmark where Fable 5 consistently has a much higher performance. as you tweak the thinking ⁓ lever. whereas Opus four point eight kind of zigzags a little bit as yeah, as as you as you said thinking too low to max. ⁓ Fable five essentially just goes straight up as you use more thinking tokens, which I think is super cool. Dan (03:20) L a little more hump shaped, yeah. Instead of up into the right. Shimin (03:33) the other kind of f fun outputs that they've included as a part of this initial release post is that ⁓ Fable Five is able to beat Pokemon Fire Red using only vision, which I think is is really impressive if you've been following the ⁓ anthropic let's play Pokemon, Claude Place Pokemon ⁓ series of experiments. Dan (03:53) See how it doesn't slay the spire next. Shimin (03:55) Yeah. Well you can run it. so yeah, it's it's fairly expensive. but let's first talk about the the safeguarding that separates the Fable Five from Mythos. So as a part of the Project Glasswing ⁓ security patching initiative. Dan (03:56) I can't afford to. Shimin (04:10) Anthropic has been releasing the Mythos preview model to a lot of large tech companies to patch up the security vulnerabilities in their code base. and they are keeping that tradition by only releasing the Mythos V unguarded model to a selected ⁓ group of consumers. Whereas the Fable 5 model, which they're releasing to everyone, ⁓ has a security guardware where they run a safety classifier so that if you ask Fable five to talk about anything cybersecurity related or biohazard related, ⁓ it would block that output and return Opus four eight's results. I think this is the first time ever where the user has no control over which model you're actually using. Dan (04:54) ⁓ you can you can actually choose. There's a toggle in the UI that you can if you click it, or at least in Claude Desktop, you where you if you set it to off, it'll just say sorry, I can't respond. ⁓ yeah, but if you but if you have that toggle set, then it'll fall back to four eight for it instead of just outright rejecting. Shimin (05:08) really? Okay. I haven't tried that. Yeah. Mm-hmm. Yeah, I saw a ⁓ thread on Blue Sky where the user was asking Fable five to produce the steps of making MDMA and ⁓ it at some point triggered a safety guardrail and it shut off the output, which I thought was I guess it's working. Yeah, and ⁓ lastly, ⁓ last thing I wanna bring up is Fable Five are only available to Anthropic Max and Pro subscription users from now until June 22nd, after which you can only use it via ⁓ usage based cost. So those things won't be expensive. It's taking up two times the token budget of your existing subscription, and they're shutting it off after giving you a taste for what 12 days or so? Yeah, so that's fun. Dan (06:03) Lovely. Shimin (06:06) have you guys had a chance to use it yet? Dan (06:08) I have not sent a query to it yet. I got real excited when I saw the announcement and tried to fire it up and they haven't enabled it yet. It at work and I haven't had time to try it on personal stuff yet. So I can tell you exactly what I'll be doing after this podcast, besides breaking down some boxes. Shimin (06:25) Yeah. That's right. I got a chance to try a few things ⁓ before our recording. ⁓ I ran it through my usual what should I do with two and a half acres in up the Pacific Northwest benchmark. And I did notice I came pretty close. It almost almost wanted to generate me the entire game. Dan (06:41) D this time did it just say Stardew Valley? Like really concise. Shimin (06:48) In this informal benchmark, it didn't do exceptionally better than the Opus four seven or four eight modules. It just gave me a few more ideas. It did point out a few other things, like some military pay levels and ⁓ some extra kind of like zoning code related things that didn't come up earlier, but it was not a step up change as I had hoped it would be. So then I ran it again on my other informal benchmark, which is the AI on AI benchmark, where ⁓ I ask AI agents what their idea of the second, third, and fourth order effects of AI would be on society and humanity as a whole. So If you look at the Opus four seven output, and I have this in front of me. you notice that when it goes to like seventh, eighth order effects, it kind of just spins out. And the answer is always like, What does it mean to be a human? which I also see on some of the weaker models, like the Chinese models, it's just like let's talk about what it means to be human, guys. Let AI tell you what what humanity means. but Fable Five. That's right. I don't know. Can can an Android tell me what what the meaning of humanity is? But the Fable Five model actually has more concrete answers here. Dan (07:56) You Android stream of electric shape. I don't know. Shimin (08:09) So I have this poll in front of me. Let's ⁓ pull through the final chains of convergence. Quote Run any of these far enough and they stop being separate. They collapse into perhaps four questions. What grounds legitimacy when labor isn't needed? What grounds equality when capacity diverges? What grounds meaning when necessity disappears? And what counts as we? still kind of in the like everything converges to what does it mean to be inhuman? But it's more concrete. Right? It's talking about equality and labor and necessity, rather than just like throw your hand up the up in the air and be like, what does it mean to be human? So in this sense, I think I think this is indeed a step change in in its ability to answer this like very open ended question. Dan (08:36) Ha ha It's still pretty hand wavy, but yeah. Shimin (08:56) Yeah, I mean such is the nature of this particular series of questions. If you ask any humans, right, like if you ask a fifth grader, what is this well, I guess a fifth grader would not know what a seventh order effect of AI would be. But like if you ask any professional economists, they would ultimately also say something along these lines, because it's fundamentally unknowing It's a chaotic system with feedback cycles. It also talked about how philosophy is important, which you know what? I take that as a win for me. So Dan (09:23) It's just advanced level sycophancy and you didn't catch it. It's it's gotten even better. Rahul Rahul (09:25) you Shimin (09:27) ⁓ darn it. Maybe. It's Rahul Rahul (09:30) Yeah. Shimin (09:30) a it's a wishful it's wish wish fulfillment sake of fancy. Dan (09:32) You'll you'll have to redo your ⁓ examples on your sycophancy site with the new I guess that's not true 'cause n no one will be able to afford to run this when it stops being free in a couple of days, so Shimin (09:37) Ha ha ha. Yeah, exactly a lot twelve days from now. It's like twelve days of AI Christmas. Dan (09:49) Get your queries in, folks. Shimin (09:50) Yeah. So how do you guys feel about the disabling the sub subscription based ⁓ payment for model usage. Dan (09:58) We'll see if that holds. Rahul Rahul (09:59) They, it's interesting that they said that if you ask it certain questions, it'll give you the Opus answer, which then to me is like, well, if you know when to route to a different model, just when I'm using Opus, route to Sonnet instead of eating all my tokens and then saying, wait for four hours, you know, add that routing to all the other things too, so that the people can optimize their tokens. Dan (10:22) I wanna say it's open router has that as a feature on their setup where you can like allow it to analyze your prompt and it will try to route you to the an the model that it thinks will best answer it out of like its enormous catalog of models. I don't know how well it works, I've never used it, but Rahul Rahul (10:38) Yeah. Shimin (10:41) Yeah, as as a rule, I tend to use the best model on max thinking, but if the models are getting so good that they definitely does the task based routing much better than I do, I I I see it yeah like a natural usage pattern going forward. Dan (10:57) Yeah. Well, you can always test it at the agent level too with the ⁓ little hacks we read about last week, right? Like have ⁓ have a max thinking kick off like a Opus agent or something if you think it doesn't need the the full shebang just for one text. Yeah. Shimin (11:05) Yep. Yeah. Well Mm. Capa capabilities. Yeah. Yeah, that makes sense. next up we have ⁓ Dan (11:19) Who cares about this one? We're done. No, I'm kidding. Sorry. Yeah. Where do we even start with this? so Meta has historically had pretty terrible to non-existent. I I feel like most large tech companies have pretty historically terrible to non-existent like customer service, right? Like if you as a user, I mean you constantly see posts on like different Shimin (11:22) This is a fun one. Dan (11:42) Aggregator sites about like, my god, I lost access to my whatever account and I can't get access to a human to actually reset it because there's been sort of mistake. You know, I've seen it for like Apple store accounts, you know, when they like published an app that Apple decided was terrible and like revoked their thing. I've seen it for like Goop Gmail accounts, like, you know, Facebook, Instagram, whatever. So so Instagram's or I should say Metas broader solution to this has been to Roll out AI as one does. so they have recently added ⁓ AI support assistant. there's just one small problem with that, which is that ⁓ word quickly spread on Telegram that apparently you can just ask the AI support bot nicely to reset someone else's account for you and it'll happily do it. Shimin (12:30) Mm-hmm. Dan (12:31) So I mean it's literally as easy as just use my new email address. I seem to have lost my old one. J just send the codes here and it does. And so they were able to take over on May 31st, is when when the initial sort of spread of this started happening on Telegram, ⁓ supposedly by pro-Iranian hackers. and Apparently there is like one extra little wrinkle involved, which is you need to use a VPN connection with an IP address that's like roughly near their usual the target's usual ⁓ login area. So it doesn't like obviously flag it as flagrant. But really in this day and age, that's really not a very tall barrier to to surmount, you know? It's like most off the shelf VPN providers give you a huge amount of choices for exit stuff. so yeah. Shimin (12:59) Mm-hmm. Yep. Dan (13:16) ⁓ turns out it was for a short period of time it was quite easy to to hack it and they got a couple of famous accounts with it, which yeah. Rahul Rahul (13:24) Obama's, yeah, Obama White House Chief Master Sergeant of Space Force. Dan (13:30) Mm. Yeah. So Rahul Rahul (13:32) I hope they drop some fun like Space Force memes. Yeah. Dan (13:36) Memes. Shimin (13:38) ⁓ Lego Lego space shuttles, all kinds of ⁓ pro Iranian music videos. So like do we think this is related to the token maxing phenomenon that we've been seeing? I think I've read that I forgot who was it from Gregory Orzov, the pragmatic engineer. ⁓ he posted that a lot of the securities and QA team has been moved around to do AI related tasks and also this max tokenizing culture has resulted in them releasing Rahul Rahul (14:07) Ehhhh Shimin (14:15) New features, probably like pre baked features without proper testing. Like I can't imagine they have an end to end test for this, right? Like Dan (14:23) Yeah. Does this new flow we added immediately compromise people's accounts? Shimin (14:26) Or maybe they do, it's just not good. Rahul Rahul (14:27) you Yeah, between that and all the layoffs they've had, it's probably like fewer people token Maxing. So this seems like what we'll get. Dan (14:39) so you're saying I missed the connection. So you're saying because token maxing is a thing, someone vibe coded the whole agent harness for this feature and as a result of that it has these holes. Rahul Rahul (14:49) Yeah. Like if your incentives are to token max and probably like show something that's valuable but you can define whatever that is. I could see that this is being like look here's a problem where there's a whole bunch of like you know open questions that no one has the time to get to. Our bot can do this and the bot did the thing. What could go wrong? Dan (15:14) But here's here's what I don't understand though. So like in order for this to have worked, the bot must be operating with like essentially elevated permissions to whatever APIs it's talking to, right? And not like assuming the permissions of the user that's I guess you're not logged in if you're going to support bot, so that's a a tricky one. But the other question I have is like Shimin (15:26) Yep. Yep. Rahul Rahul (15:33) Yep. Dan (15:36) Why that look, there's a lot of good places for agents, right? But but why why replace a form that requires you to type in an email and then intentionally gives you a vague answer about whether or not it's an accepted email, right? Because that's also a attack vector, with a chat bot in the first place. Shimin (15:41) W why is a great question. I'm wondering that too. Yes, absolutely. Rahul Rahul (15:43) That's my favorite question then. Yeah. So again, yes, I fully agree. Just because an agent can do it doesn't mean it should, right? So some of it goes back to like, why are you trying to solve, already solve problems in weird ways? Dan (16:14) And also like the ROI on that doesn't make any sense. Let's take like a form that's like probably you could run it off like, I don't know, a Lambda function or something, right? That's gonna cost you like well, maybe a Facebook scale will cost you like a thousand bucks a month or something, but not like crazy, you know, with like an LLM that like you could probably convince it to write Python code for you, ⁓ instead of help you with your Facebook account too. And Rahul Rahul (16:19) Yeah. Yeah. Hmm. Yeah. I have Chipotle for that. I don't go to Facebook for my Python code. Yeah. Shimin (16:42) Well I I think Dan (16:45) Or a car dealership, yeah. Shimin (16:48) I mean they're they're trying to handle the the case where you've lost access to the email, right? So like you're trying to change an email associated with an account, which is a complicated process. That isn't as straightforward as just create a form for it. Dan (17:02) Yeah, but how many times does that happen that isn't that is legitimate, you know? Rahul Rahul (17:03) but in there. Usually yes because usually in cases like that you have to send almost like your passport or some government issued ID to be like I am this person this is my account blah blah blah and Shimin (17:07) But we are cost cutting. Dan (17:22) Yeah. Shimin (17:23) Right. Yeah. Rahul Rahul (17:24) try and recover your email first or your accounts would come later because if your email is the vein to everything. So it just doesn't make like even if you run that scenario down it doesn't make sense to go if someone says their email is compromised all you need to check for is they live in in case you know like this wherever the the sergeant lives, sergeant master lives of the space force or the where Obama White House is located. It just doesn't seem like. Shimin (17:52) Well it's clearly s what sixteen hundred Pennsylvania F. ⁓ nineteen hundred? Rahul Rahul (17:56) It may be. I don't know if they moved it to because the library and the center is in Chicago. So they might be running the account from there. But yeah, it just didn't seem well thought through enough to, you know, for the agent to respond with that. And on top of that, you're like Dan is saying, you're giving it all these elevated permissions. Shimin (18:02) that's true. Yeah. Yeah. one last thing. We talked about this a lot. this is gonna be a golden age of cybersecurity, and this is a prime example of it. Like red teams are gonna make bank going forward, and every large corporation that rolls out any AI features would have a red team on retainer going forward. So at least you have someone to blame when stuff like this happens. All right, moving on. Dan (18:41) Yeah. Except they're all using an agent too. So if their agent doesn't catch it, then who knows? Shimin (18:49) Yeah. And that's all good news for the model providers, right? Burning more tokens than ever. ⁓ Dan (18:53) That's true. It's tokens all the way down. Rahul Rahul (18:54) Use Fable 5 before June 22nd. Dan (18:58) Yeah. Shimin (18:59) Okay, on to our ⁓ first post processing post. This is one's from Dan. Dan (19:04) Yes. so this is an article from J AI ⁓ entitled LLMs are not the black box you were promised. so the sort of subtitle is Mechanistic Interpretability Has Made Major Strides. so this is almost kind of like a meta blog post because they're really breaking down the anthropic ⁓ biology of a large language model. thing but the I kind of needed the breakdown if I'm honest to to to get through some of the heavier concepts on this so previously we've sort of had this understanding that like you really can't know what's going on within an LM because something like a concept like I don't know, refrigerator doesn't map cleanly to like one neuron in one like layer, I guess. Is that the right terminology for that? Yeah. so when that lights up, it doesn't necessarily mean refrigerator because it might be lighting up in the context of like two hundred other neurons that are all lit up too, and then that is actually refrigerator, right? Or they're lighting up a various, you know, vector values, then that actually means you know. refrigerator. So that's been kind of a tricky problem to solve until recently, where apparently what they have done is replaced the well, so they've created this circuit tracing technique. And the idea is to be able to actually understand where the concept flows through the model. So what they've done is created like what they're calling a replacement model. and this is where I really need your help, Shimon. That's essentially, if I understand it, like a a completely different model architecture that's easier to to reason about, but it can have the same it wine it's trained to produce the same outputs. Is that accurate? Shimin (20:44) I didn't think they were well I guess yes, they would be trained to produce the same feature outputs, yes. but they have yeah, j a different arch architecture. That's that's right. Dan (20:51) ⁓ Yeah, like a more traceable architecture, right? Is is my understanding. So then what they were able to do is now if you ask it a question, like, you know, what is I think the the example they have is like what is the capital of Texas? You can look at what entire feature lights up, and those features are in fact correlated directly to what like a human would think of as a refrigerator concept, right? So it's like Shimin (21:17) Mm-hmm. Dan (21:18) you know, you ask it like what is the capital of the state where Dallas is located, it has to think, ⁓ Texas, and then it has to think, ⁓ Austin, right? Are the the things and they're able to trace through and see those features essentially light up. so the the part that I took away from that that's pretty cool is that there is in fact multi step reasoning going on there. Shimin (21:39) Mm-hmm. Yeah. Dan (21:41) Which you kind of sort of intuitively know, but it's neat you know, neat to see it like proven I guess mathematically. And then the other piece that was like a little bit I guess not surprising is that the same phenomenon shows up in other AI systems that are not large language models too. So they also looked at like AlphaGo or Alpha Zero. And it was they where they were able to prove that like it had essentially learned intermediary representations of like chess, for example, that corresponded to human concepts, even though it had learned them like fully independently. So something like Checkmate, for example, is something that it had picked up. Shimin (22:17) Right. ⁓ yeah, it does make sense. So so one thing about the replacement model is replacement model was trained so that the activation is sparse by default. So it penalizes the output of the model when you get, you know, multiple neurons hitting the same concept. So it favors a single activation for a single concept. Right. Like that that's the really cool part. So if if both If both model produces refrigerator but one model used only one neuron firing and the other one used four, then the one neuron firing version will get higher weight and higher ⁓ higher learning. So yeah, so you get you get higher you know, the the gradient descent is further. And that's how they were able to isolate these things. So then you can do like, what are the things inside the refrigerator? What are the things inside my closet? Dan (22:53) One year training. Yeah. Shimin (23:06) And you see the activations, and you can really isolate the refrigeration ⁓ neuron versus the closet neuron, which which is super clever. Thanks guys. and the other thing is we spoke to David Ng about ⁓ the idea of a large language model needing to know the output of every poem to get a correct rhyming scheme ahead of time. Dan (23:13) See, this is why we keep you on your own podcast, you know? It's it's it's this level of insight that Shimin (23:32) Right. And we talk about how that's almost like ⁓ quantum mechanics or model collapsing. and this is the concrete representation of that. It needs to be able to think ahead. It needs to be able to think about related concepts before coming to an end. So when it's generating a a poem, it probably has an idea of a sonnet and rhyming couplets somewhere inside the neuron before it even comes up with the final rhyming word. Dan (23:36) Wave function. Yeah. Shimin (23:58) 'Cause it still needs to make sense knowing that word ahead of time. And so now you have these ideas of circuit, 'cause it ⁓ it must have activated a circuit somehow to know the last word should be I don't know, peaches when it's talking about pink ahead of time. and that and that's really cool. Dan (24:16) And then the other part that I thought was kinda rad was where I got my middle name from apparently this this episode, which is the the headline on it is the model has a subconscious with air quotes around it. so they're they're quick to point out that they're you know, it doesn't actually have metacognitive insight into like its own thinking per se. So like it it you know, if you actually like ask it how did you add two numbers together, then it will come up with like a lie about like effectively about how a human would do it. but the the process that kind of got it there isn't necessarily that process in the first place, you know? Like it like they're sh talking about how th it like LMs have actually come up with some pretty inventive ways of doing math on their own using like lookup tables and a bunch of other stuff, which like just not at all how like human mental math concepts work. Shimin (25:03) Yeah. And watching you realizing that's where your middle name came from this week is a form of metacognition that the AI is incapable of. So last week we were talking about what are the things that humans still have an advantage on and what are the patterns and the skills that would not be made obsolete by AI. I was thinking about this over the weekend and one of the things I came up with Dan (25:12) Ha ha ha. Shimin (25:28) was metacognition. I feel like that is one of the few things potentially that humans will still have an advantage on over AI in the next five years. ⁓ and this just proves that I was right. So I'm happy to read this. Rahul Rahul (25:40) Some some things we even humans can't explain the right like sometimes when you have good feeling the intuition on something or even judgment is a lot of the time so you can't use it's anything that you cannot teach to anyone else because you literally cannot explain but you just know based on your experience so Shimin (25:56) Mm-hmm. Rahul Rahul (26:02) I feel like this is what this reminded me of is it knows the answer and it's making stuff up to explain how it got to the answer but not really and often times when you look at you know people saying they made judgment calls and stuff you can ask them to explain but they literally cannot explain every single thing and Shimin (26:06) Mm-hmm. Mm. Dan (26:24) Or we'll do the same thing the LM did, right? Make something up after. Rahul Rahul (26:28) up, but we Shimin (26:28) Yeah. Rahul Rahul (26:30) have the concept of like the tacit knowledge where you literally just have to sit by someone and just watch them do something to learn it yourself or just by experience, some things you just cannot learn from reading or other people telling you about it. That's what makes me think of it. Shimin (26:49) That's pretty great. So like post hoc rationalization is not something that is uniquely to humans. AIs suffer from this too. And another ⁓ another interesting potential fourth order effect. so so like humans have for the longest time thought that the brain is the most important part of our body, right? We talk about like what's in his he his or her head, talk about some of our brain brainy. Dan (26:55) It's true. From the best, you know. Rahul Rahul (26:56) No. Shimin (27:13) But what and we all know that the gut actually contains a lot of neurons and that's why we say stuff like we have he has a feeling in his gut. So what if as AI catches up to our brain and frontal cortex, the gut becomes more important? Like saying someone is gutsy becomes like the highest form of inp of compliment. Potential future. Rahul Rahul (27:22) Mm-hmm. Hehehehehe Isn't that the whole high agency thing? Somewhat. Yeah. I do wonder what AI's cognitive dissonance would look like. Cause that's why we make stuff up to explain our, you know, like, don't make me think too much. Here, my brain made up something to explain it away. So maybe that's what we're seeing here is it also going, don't make me use up too many tokens on this. might crash. Here's some made up stuff. Dan (28:02) I I mean I've gotten that kind of response from Claude before. That that always cracks me up when it's like you're doing something that's like kinda hard and then it goes, and now here's the part where you finish this. And I'm just Who's paying for who here? Rahul Rahul (28:12) hahahaha Shimin (28:15) It's that's management material or or consultancy material right there. Rahul Rahul (28:21) It's called managing up, Dan. It's in Claude's training. Dan (28:23) And then I'm like, Shimin (28:24) Ha ha ha! Dan (28:25) and no, you do this and it's like, Okay. Rahul Rahul (28:27) Hehehehe. Hehehehe. Dan (28:31) Nice try though. Rahul Rahul (28:33) Yeah. Shimin (28:34) Okay, the next post on our slate is brought to us by Rahul. Rahul Rahul (28:38) The Future of Agents by Chris Roth. I think we've discussed another one of ⁓ Chris's articles in the past. This one's about all the different patterns that Chris is seeing out in the world today and some predictions on where things are going. So we can run through these one by one. First one being the rise of personal agents. So people are building a lot of these, you know, have open client cloud code and everything. But you're going to have... Shimin (29:08) Mm-hmm. Rahul Rahul (29:11) one agent that then interacts with everything else versus you're not going to go and learn the domain of each agent because then that's again too much work for you versus you would have for better or worse co-pilot might be the thing that all the Microsoft users use as their agent to then interact with everything else. Dan (29:33) Well, they're they're making some strides there. Like they so they just announced it build that that CompuDX. Yeah, the the models and then also there's the new sort of like deep well, I wouldn't say it's a partnership per se, but they've they're supporting that new Spark chip that Nvidia is pushing. So like they've taken the sort of like DGX spark, which was their original like Rahul Rahul (29:38) yeah, the models. Yeah. Shimin (29:56) Mm-hmm. Dan (29:56) Little set top box where it was basically like a 5090 glued to some unified RAM and like a little ARM processor. Sorry, this turned into hardware hut, but here we are. and ⁓ they just announced at CompuDex this week that they're gonna be ⁓ the new surface like pro whatever, obviously like a you know, play at the Apple stuff is basically gonna be a a new consumer based version of that. ⁓ including the unified memory piece, which I think is pretty neat. Rahul Rahul (30:19) Yeah. Shimin (30:20) Hmm. Dan (30:23) So like it will I mean I don't know what those are gonna be priced like. I'm sure it's gonna be pretty premium with Ramagedon still going on, but like it might be pretty possible to just like run your own, you know, whatever, like open weights model on something like that. And more so than it is right now where like you need to buy a four thousand dollar like, you know, digits or a I guess it up to probably three thousand dollar, you know, three ninety five max to be able to run it. Rahul Rahul (30:37) yeah. Shimin (30:38) Yeah, I but Dan (30:49) Or even more expensive Mac. Rahul Rahul (30:50) So yeah, we might see the same thing play out that we've seen in the past with other things. So Microsoft's model might be very powerful with Microsoft stuff, but anything outside and other. platform providers would put up walls and so then you see a solution that would be like this works across all the different platforms you don't need to worry about it and there's going to be this continuous you know same as what you see with any other technology where the platform provider is trying to win the most share of the market and so you would probably get a lot of this works across any operating system or anything, but maybe doesn't get every single thing. And then at some point you start pointing fingers, be like monopoly practices. And then they're like, fine, I'll give you two more APIs and just go away. Right? I mean, that's how it goes every time. So it's probably going to play out in a similar way. Shimin (31:44) Right, but we're talking about the rise of personal agents here. And I I think like inevitably there's going to be a rise of personal agents. Everyone will have a Siri on their phone, whether or not talks to your cloud or talks to your own Sparks box at home. That is like a Siri that actually works. Is the future that we've been promised. Damn it. And we're finally gonna get it. Damn it. We some of us already has it. Rahul Rahul (31:47) Yeah. Yep. Dan (32:06) IOS twenty seven beta out today. Well there's a wait list for Gemini based Siri though, so Rahul Rahul (32:11) yeah, they had that whole video about Apple intelligence yesterday. Yeah. Dan (32:18) Yeah, W W D C keynote also this week. It's been a busy week. There's just so much news going on, we can't even cover it all in w in an hour or so. Shimin (32:18) Okay, so Rahul Rahul (32:19) Series getting better guys, hang in there. Shimin (32:24) Cannot cover it all, unfortunately. Rahul Rahul (32:24) Yeah. Next up, yeah, everyone will have their own personal agents. ⁓ That would be bring your own agent model. But then the next prediction is, know, the trend right now is. buy the hardware, run it locally, but at some point that's just not going to be sustainable just for the same reason why we have data centers and we just don't have massive racks running in our apartments or homes all over the place, except for Dan, you can... ⁓ Dan (32:52) No, I'm I'm picking this apart because the thing that people are buying them at they're not buying like beefy Mac minis to run inference on. They're buying like a Mac mini to basically use a glorified container that they can give over entirely to the LM, right? So the the inference is still happening in the cloud in those cases. Rahul Rahul (33:08) Yep. ⁓ If you go with the personal agent piece, then you might just not be able to, you might be compute restricted. So I get this is kind of like dependent on the first thing that personal agents are going to become more. Shimin (33:27) Yeah, if if the compute is happening on the cloud, then I really don't care what format the personal agent exactly. Because there will be a flexible enough platform to build those personal agents on, as long as the compute, the most expensive part is still happening on the cloud. And I actually also disagree with this. Dan (33:32) Where the harness is running. Yeah. Rahul Rahul (33:45) Yeah. Shimin (33:47) 'Cause I think we can look to video games as an example of we think of like we now have the ability to play to stream video games that require a lot of compute power over the internet, but we but that never caught on. what is it, the Nvidia stream that died a ignominious death? Rahul Rahul (33:58) Yeah. Dan (34:04) It's still around. I mean, I think is Luna's Luna's the one that died, right? But Shimin (34:09) Right, Luna died, yeah. Rahul Rahul (34:10) It's alive, isn't it? Is it dead? Dan (34:12) Yeah, it's still around, but I don't know how how y how much usage it's getting. Shimin (34:13) Yeah. Rahul's so disappointed. Rahul Rahul (34:16) Damn, five years ago I played a game on it and I hit a button and 10 seconds later it did a thing. What else do you want? That's as fast of a feedback loop as one can ask for when you're in a fierce battle. Yeah. Dan (34:24) Yeah. Shimin (34:24) Ha ha ha. Dan (34:28) Definitely play Counter Strike in the cloud. advantage meets latency in all directions. Rahul Rahul (34:35) Yeah, it would be hilarious if you can see the boss move, but we have enough time to take action. Shimin (34:36) And I think Latency may not be an issue, but privacy is gonna be so big, right? Like I do not want any company to have so much of my personal data that they can pr generate the perfect ad targeting me. Maybe that's old man yelling at cloud, literally, but at least my generation of folks probably still care about privacy enough to not do that. Rahul Rahul (34:46) Yeah. Yeah. Yeah, that was a big, as always, selling point in Apple's video yesterday too, of it being very privacy focused and as much on device as possible and on Apple Cloud, but only for like the amount of time it needs to be to execute a request and stuff. which brings us to speaking of privacy you can bring your own key or bring your own agent the so there's two different things here one is you can provide your own API key but the problem with that is that you don't really have control of a lot of things and there's a diagram here where the app receives your key, the agent is taking action there, but the trust boundary is where the ⁓ app can see your key and data. that's not something privacy conscious people are going to watch. then, so the other thing is you bring your own agent where the trust boundary is you keep the key and the data. So you keep the memory and context. everything with you and then the app exposes different functions that you can talk to. That's where you see, ⁓ you know, MCP and CLI and all that being used today. And I think that is the architecture, the latter one would be the architecture that is gonna keep going in the long term. I don't know how much people would want to stick their keys in if they know what they're doing. Shimin (36:30) Yeah, that makes sense. Rahul Rahul (36:31) And then next up is agent clients where you have all these different AI apps that end up looking like chat apps. ⁓ Basically like there's one. Shimin (36:42) Mm-hmm. Rahul Rahul (36:45) What is, what did chat GPT wanted to be? Super app, I think. So you're going to have a lot more super apps and everything ⁓ that are able to do all the different things, Telegram and Slack and all that are already the, you know, the places where people gather to have like, to talk to each other, Telegram and Signal and stuff for personal life, Discord even, then Slack, Microsoft Teams and all that for corporate work and everything. So there's probably going to be more and more of that super app move in these where you have a single app that everything is going through. Shimin (37:24) Yeah, and their their question is, will the UX still look like a Slack or a signal or a telegram? Right? Like, do we treat these apps are designed for human-to-human interaction, sometimes in groups, sometimes one-on-one. And is that fundamentally how we want our UX to look like interacting with agents? My gut says no. There are just so many agent-specific things that Rahul Rahul (37:45) Yeah. Shimin (37:50) You don't want to treat it that way, but I can't envision what a better UI will look like just yet. Rahul Rahul (37:56) Yeah, and I wonder how much, because at the end of the day, even if an agent can do a thousand things, but we can only hold seven things or whatever in our head at any given time, you would also need to design it for that where, you know, that bottleneck in that case might still be, if a human is in there, you'd want to Even if they're not blocking in the loop just to communicate to them, you might want to design things in a certain way. You know, anytime something gets like too much cognitively taxing, people would just turn, think of any Slack channels that you muted because it's just a dump of something. You're like, I will get so much information from this. can do so many things. And then a day later, like, I don't know what's going on here. I'm going to mute this forever. Shimin (38:34) All of ⁓ yes. And like it's not even gonna be text necessarily, right? Like it's gonna be anime waifus. not that I'm into that kind of stuff, but like that's that's gonna be what a lot of people's interactions are gonna be with their agents. Like assuming video generation is cheap enough. Rahul Rahul (38:46) Yes. Yeah. True. Dan (38:56) Well, I had an interesting thing the other day where it's like I've it's a strange admission to make on this podcast, but I've never made I've never used voice mode on Claude. Just like never once. I just like always prefer typing and text. I type really fast. So like I just don't you know, I know a lot of people are like, I always use voice whenever. but I was walking the dogs and I forget what I was asking about. It's some question in some just like you know, keeping an eye on the dogs, making sure they don't run out on the road and everything else, and like trying to pay as much attention to them as possible. But I had like a brain burn where I was like, I gotta know the answer to this. So I just fired up voice mod vo voice mode. And we end up having like a fifteen minute conversation that just kind of like spanned a bunch of stuff with Claude. And I did not expect that. And it was a pretty interesting experience. Rahul Rahul (39:42) Nice. Shimin (39:42) I've been curious about it. I the couple of times I tried it ⁓ it didn't pick up my words clearly enough to actually make it useful. But I'm happy to try that again. Yeah. Dan (39:50) Yeah. I had headphones on too. I don't know if that helps or not. but it was it was interesting. So not saying it's for everyone, but like I could see that type of interaction like in the moment it made a lot of sense and was pretty handy. you know, 'cause then I was able to like keep thinking about the thing that I was thinking about and and carry it forward. But Rahul Rahul (40:03) Yeah. Good. Dan (40:09) I was just gonna say I could envision a world where you go for a walk and still get work done, you know. Which is kinda hard to do with a laptop. Unless you've got that crazy thing from the comedy show where he had the Rahul Rahul (40:16) there. Shimin (40:18) I hear stories of I hear stories of people doing that today and ⁓ I almost don't want to be those people. Like I wanna have some time away from the agents, but you know, to each his or her own. Rahul Rahul (40:23) Yeah. Dan (40:30) I think it depends on how you use it, right? Like if you're if it's during your work day and you're gonna be your primary task at that moment is sitting there, like, you know, doing a sort of interactive session with an LM to like get some work done. Who's to say that you shouldn't do that on the move? But should you then continue that on the couch while watching T V at like, you know, 10 PM? ⁓ yeah, I don't know. Shimin (40:50) that's a dream. Yes. It's the future that Wall-E promised me. I want it. I want it now, damn it. Dan (40:55) Yeah. Rahul Rahul (40:56) Hahaha ⁓ Dan (40:58) No, but I'm saying like, you know, I think work life boundaries can kind of cut both ways in that direction, right? Like ⁓ like you should be able to ta leverage this technology to make your life better if you want to go outside and touch grass and also work at the same time, more power to you, right? But like at the same time it shouldn't like meaningfully detract from your life either. So Rahul Rahul (41:04) Yeah. Dan (41:19) But 'cause think that's the things everyone's saying, right? Or was saying, like, I guess the the discourse shifts, but like what, three, four months ago when we were talking about it? Was that like that HBR study had come out and everyone was going, ⁓ AI hasn't like proven much productivity, which I think we're kind of maybe still in that place. I don't know. What do you all think about that statement? But then we're at the time that the discourse was like, but everything is sped up and people are facing like burnout and everything else because of like Rahul Rahul (41:45) Yeah. Dan (41:47) Just the pace has has notably increased. and that's true for software engineering when that happened, but like now I'm personally at least seeing evidence of it in other industries too, where people are just saying that like the expectations have shifted for tight like intellectual work because you can use AI to summarize and in some cases draw conclusions and sort of outsource your thinking too. Shimin (42:08) Yeah, we're the we're the canary in the coal mines. We're like just like the open models are like four to six months behind the closed source frontier models. all the other white collar jobs are four to six months behind the programming world. Dan (42:12) True. Yeah. Ha ha ha. Rahul Rahul (42:23) Yeah. Dan (42:24) Yeah, that's fair. Rahul Rahul (42:24) The, I, on the whole audio versus, you know, type or voice versus typing, I feel like for some like things that don't really need as much mental, you don't need to think about it as much and it's more just like here and there, I could see voice being. useful but to me anything where I have to think hard or I have to keep something in my you know focus for a long time I prefer looking at it and reading through it because it it kind of like sears in my brain better than if Claude said something to me, it would have to tell me that same thing 10 times for me to really get to the same place where if I had read it once. So there's, I don't know if it's just me, it's similar to like, do you read books or do you listen to them? And then how much do you absorb when you do one or the other? Dan (43:17) Mm-hmm. It just made me think about like, you know, we're kind of spitballing on the we don't r don't necessarily know what these interfaces will look like. But like one of the things I like doing when I'm faced with a a big problem, like conceptual problem, is I draw pictures. Right. I just like it's helps me I'm a very visual learner, I know that about myself. And so like it helps me to like sketch things out. and I'm like no one's really thought about that as a LLM Rahul Rahul (43:33) Yeah. Yeah. Dan (43:43) interaction point maybe because they are large language models, right? But like what would it look like to interact with some of these systems in other ways where like you might draw something and then have it draw back or contribute to your drawing or have a sidebar of like, you know, that's text, but it's like looking at what you're doing or Shimin (43:55) No, that's that's a gr yeah. That's a great idea. We should start a startup that explores that option then. Dan (44:03) Yeah. Rahul Rahul (44:04) Excalibur LLM integration. Shimin (44:07) Tus tasks tas. Dan (44:07) Meets meets Claude. Rahul Rahul (44:09) Yeah. Next up we have open standards plus infrastructure is going to get commoditized. I'll be honest, I had not heard of some of these protocols other than MCP and A2A. I had no idea about AGUI and stuff. And then at the same time, obviously Chris hasn't heard of Gemini Enterprise Agenda platform because Shimin (44:34) Ha ha ha. Rahul Rahul (44:35) You know, we had services for them. ⁓ Shimin (44:36) That's where you've been. Dan (44:41) He was on a Google sponsored vacation. That was his payment. Shimin (44:46) He has not. Dan (44:47) Now I see how it is. Rahul Rahul (44:48) Some of the things are not standardized today. So, you know, everybody's doing it in their own way. People are managing memory and context and everything in their own way over time will probably have standards that have ⁓ emerge from these for these as well so that we can, again, even like a cross platform so you can follow the same standards instead of trying to figure out from different things on different platforms. It is a bit tricky for stuff like memory and context because I think those are you know, you're stepping into harness engineering and stuff. And there's a reason why those are not standardized because Claude Code's harness is, you know, part of their IP and same with codecs and stuff. So I don't know if you can ever get to a standard because you might be just going with the Lewis common denominator almost at that point versus them really optimizing their product and giving you the most value out of it. Shimin (45:47) This one I also basically disagree with. I think because code is gonna be so cheap that the cost of creating an adapter for some standard is also zero. So there's no clear network effect from adopting a particular standard. Everyone can build their own custom standard and then create some sort of a bridge to some other corporation standard ⁓ in real time. And I it this reminds me of when I switched from Pi agent Rahul Rahul (45:50) you Hmm. Shimin (46:14) to claude code when claude subscription was no longer being applied to Pi agent and I just had Claude code open up the directory and be like look at these skills. Create claude code skills. Yeah. It was so seamless. And I I haven't noticed any real reduction in yeah, there is no moat Dan (46:24) Yeah, and use them. It's just There is no AI moat Yeah. Rahul Rahul (46:32) Nice. Dan (46:34) Yeah, I was just gonna say that's why I thought it was so shocking that Anthropic took the stance that they did around like, you know, extra usage and everything, because it's like there's really nothing stopping you from taking those same skills, Shimin and just walking to open AI or take sort of taking the reverse stance, you know, like please use our subscriptions for that stuff. So we'll see how long that lasts in the token apocalypse. But Yeah. You're ready to hear first, folks. Token pocalypse coming soon. We've had now two S ones filed, we'll get to that in a minute, but ⁓ yeah. I think that the next six to twelve months are gonna be the most interesting we've ever had on the show. Which is saying a lot 'cause it's been a wild ride already, so Rahul Rahul (47:12) Yeah, SpaceX go. I'm not going to spoil it. Let's, next up is enterprise enterprise will demand open source plus open standards. This kind of goes back to what you were saying, Shimin we mentioned token maxing earlier. I think we're starting to see, you know, already the, Dan (47:16) Ha ha ha. Rahul Rahul (47:32) end of the or the past the peak part of that because you see ⁓ copilot going usage based Uber saying yeah enough with that ⁓ Amazon had killed it's the token maxing leaderboard And so one of the big concerns that enterprises have and will optimize for is what is the cheapest level of intelligence for this specific task. And especially given that ⁓ open weights models are not far behind, they're going to be much more interested in potentially using them and even deploying them on-prem if it makes more sense from a you know, what you spend versus what you get out of it perspective. The connection to them using interoperable ⁓ open standards that I wasn't really able to make the connection with and like you're saying, if it's easier to just work with any standard, there's not necessarily a reason that they'll be everything will be standardized. Shimin (48:33) Yeah, definitely agree on the open source piece as well. Like there's a reason why the Chinese models are most heavily used on open router and in silicon valley, 'cause you don't want to tie your provider to anything that you can't control. Rahul Rahul (48:42) Yep. Yeah, and especially if you cannot get the same amount or if you spend a dollar on a model, you should be able to get at least a dollar plus something back, right? Or ideally multiples. Right now, the models are, the like big US models are, you're spending a lot, but you're not necessarily getting the same value. So it's also a way to preserve optionality where like, let's try and do the same thing, but as cheaply as possible. And then once you prove the value of maybe then to like, you know, cover the last mile and really covered the last like 1 % of the problem that the cheaper open source model over there cannot solve. Then you can always go to the higher intelligence, more costly models. that's likely where things are gonna end up going. And then finally, software will be even more malleable. I don't like ever changing UI. So I don't know what generative UI would be like. And so on this one, I agree with Chris here. I think there is a good... opportunity to personalize UI to everybody. So if done right, I could see overall where you can very hyper personalize everything to your users. But if you end up doing it in a way where it's like, the UI is changing constantly, but it doesn't necessarily as personalized to you, it's just gonna be a disaster because then it's similar to. Shimin (50:08) Mm-hmm. Rahul Rahul (50:14) a software that you use every day but it's changing every day. You'll be like, I don't care for this, it's changing too fast. I don't want to use it. Shimin (50:22) Yeah, and ⁓ you know, this is my front end person showing. ⁓ I could see a world where you have your agent build an UI for you for a particular that you can use across different apps. Like that would be kinda cool. Rahul Rahul (50:35) Yep. Somewhere in there, one thing that Chris does call out is the big models will make the cost and ⁓ when you bring your own personal agents. That didn't make as much sense to me either because even if your personal agent is using a lot of intelligence, it's probably integrating to someone that has like its own domain knowledge. Otherwise, why would people even build new apps or anything? So that piece I think was also. missing a bit. Shimin (51:06) Alright, well that is the future of agents. lots of interesting points, some of them which Dan and I disagree with a lot, so we'll see. We'll see how things shake out in six months. Yeah, no, that was a lot of food for thought. It's okay. We can disagree on things. That's the point. The point is to it's the point is to take other people down, Dan. ⁓ I don't know if you've heard. Alright. Dan (51:16) Still a good discussion. And Rahul Rahul (51:18) Yeah. Dan (51:22) What? ⁓ no. Rahul Rahul (51:29) Chris is gonna start his own podcast and Shimin (51:32) we can have a feud. I love feuds. All right. Rahul Rahul (51:33) Yeah. Dan (51:35) What what am I getting into here? I didn't I thought this was just us talking about AI. Shimin (51:41) ⁓ On to Vibe and Tell this week I have a design skill that I've been working on that I wanted to share. it's called inhabited design. And the motivating problem is a lot of times if you use a single skill for front end design, you often get the same exact output. So in this case, I initially had a prompt for designing a landing page for an energy drink. filled with ⁓ cocaine for finance pros and Claude will not do anything with drugs so ⁓ i had to settle for just an energy drink for finance pros and I ran the same experiment three times on front end ⁓ design skill that comes out of the box of Claude code and you basically see the same design and similarly when I switched over to impeccable which is my favorite front end design skill before I created inhabited design, you still see similar motifs like a stock ticker, and a very just generic looking landing page. And of course if you use it with lovable, ⁓ you get just a like AI Slop everywhere. first, let's take a look at an actual landing page and see what Dan thinks of it. Because when I previously gave Dan a landing page created by a previous version of the said this still looks like AI slop to me. So then question does this particular landing page look less AI sloppy? Rahul Rahul (53:04) Leading question. Shimin (53:06) You can say you can say no. Dan (53:08) I'm looking at it. I'm I'm deciding. yeah, I think it's a little better, honestly. I mean, I guess it's it's it's rooted in the age of that was prevalent at the time that LMs were trained on still and I don't know if there's much you can do about that, but it still feels like a a human designer did it to me, I think. Shimin (53:29) Well, thank you. Thank you for that answering that leading question, Dan. I appreciate it. but more important than talking about just the skill, I wanna talk about a few techniques that I use to combat the fact that, you know, with a large language model for a creative task, often you get the same repetitive answers. So like if you tell Claude Code to tell you a joke, it will say the same joke over and over again. Or jokes that are very similar to each other. And what happens is you have this attractor that gets trained into the large language model during the reinforcement learning with human feedback. So it tends to give you similar outputs at the end. And in the case of this landing page, it's the same stock ticker, it's the Bloomberg terminal yellow, and that's why you see the same output every time. And to really shake the magic eight ball and get something actually different, ⁓ what I did was I used a thing called verbalized sampling, which is the idea of asking the agent to give you all the options. according to the paper, these options are actually weighted by probability, but I'm actually doing just a complete uniform distribution and picking them ⁓ as if each option is weighted equally. And this gives you a more diverse answer from the model. And the second thing you could do is you can do an intent factor generation. And the idea here is if you ask the AI like to tell me something about avocado, it will say the same thing over and over again. But if you say tell me a thing about the price of avocado, or the taste of avocado, or the color of avocado, it will give you s more varied outputs. That are still true about avocados, as opposed to if you just turn the temperature up to one and then now you're it's just giving you gibberish. So the idea here ⁓ is you want to still create things that look like landing pages, but you want to insert variance and so what the skill does is it factors the design into many characteristics. ⁓ it takes a designer, it takes the typography, it talks about inspirations, and I randomly sample from from those options. So by the end of the sampling page, we have a landing page that is factored on a particular designer that's randomly sampled. with a particular inspiration, which in this case is the American Express Corporate Black Card, with ⁓ a tea house motif. And so when you go back to the original landing page and t once you see the inspirations you can see ⁓ these bronze stripes on the left hand side, those are inspired by the tea house ⁓ Kind of the the wooden slats on the tea house, right? And the whole page has more of an American Express looking feel with its black background, etc. and of course, if you're looking at doing actual design work, ⁓ like any good designer, you want to actually make it visual. So it the skill actually pulls actual references from the designer and from the inspiration and from from the domain and opens it while it's doing the design to to give you this cohesive element that also borrows ideas. and lastly it runs two convergence loops. These loops are not cheap. the skill burns a good amount of tokens, but it does give a much more varied design than ⁓ just asking, you know, the front-end design skill to give me a landing page for energy drinks and over and over again. ⁓ so listeners, if you wanna try it out, just search for inhabited design on Google. you'll find it or inhabited design skill and you'll find a link to it. And I would love to give feedbacks if you got a chance to try it out. Yeah, that's that's my pitch. That's the that's that's my vibe and tell this week. And thank you, Dan, again, for for being for being so kind. Dan (57:17) Nice. I took my time to deliberate. It wasn't just a Snap judgment. I think it's easier when you have some very sloppy examples to look at, up front. So Shimin (57:28) Yeah. I I know you have an eye for design, Dan, so it it means a lot. Dan (57:33) I don't know about that. I have a particular taste that's ⁓ anchored in ⁓ a particular time period and I think that's what I've got, you know. Shimin (57:43) That's that's that's how we know then you're old. just kidding. At that time the Reformation period in England during the sixteen fifties. Dan (57:46) That's the only way you know that I'm old, I'm doing all right, so What do you think I am a Highlander? Like see, that was an old reference. So you could you could beat me up on that, but Rahul Rahul (57:53) Great design. Shimin (57:57) Yes. you can find Dan at the showing of He Man opening this weekend. That's his favorite cartoon from his early teenager. Just kidding. Dan (58:07) It's not nice try though. Shimin (58:09) Alright, on to our very last segment, Two Minutes to Midnight, where we talk about news coverage about the state of the AI bubble. ⁓ our clock is inspired by the Armageddon clock from the Bulletin of Atomic Scientists from the fifties and sixties, and still a clock today. We are currently at five minutes and thirty seconds. I'm gonna go first. ⁓ this week Google Signed a deal with SpaceX to provide 920 million a month for XAI's data center compute capacity. And this is reporting from CNBC. Google already owns ⁓ a significant chunk of SpaceX from back in the day. So they are in the seat to gain a significant amount of, I think something like 50, 60 billion dollars when SpaceX goes IPO. later this year. so there are rumors that ⁓ this serves multiple purposes. It definitely gives Google additional compute that it doesn't already have, but it also provides SpaceX with a positive net earning for the year in their computing segment. and given that Google is an investor in SpaceX, ⁓ there is Maybe a little bit of circular financing. Maybe a little bit more than a little bit of circular financing going on here. Rahul Rahul (59:26) You know Founders Fund invested 20 million in SpaceX back in the day and they're gonna get billions out of it with this. Like 20 billion or so. It's one of the best bets in hoping or assuming the Friday IPO will get them to two trillion or whatever they're going for. Shimin (59:34) Oof. That's the VC playbook. Rahul Rahul (59:55) Yeah, it'll be crazy multiple. Dan (59:58) This this didn't make the the list in time, I think, which is probably my fault, but Martin Elderson, who's previously been a guest, ⁓ also had a post on there where about this topic where he's like, it really feels like ⁓ XAI has basically become a like GPU rental service instead of a frontier lab, which I thought was a pr pretty ⁓ on point criticism of whatever they're up to these days. Shimin (1:00:05) Mm-hmm. Yes. Yes. Rahul Rahul (1:00:17) Yep. Shimin (1:00:23) All right, next up we have an article from Dan. Dan (1:00:26) Yeah, so a little bit of good news, or at least I think it's good news, I don't know, is ⁓ yeah, the SP five hundred has rejected SpaceX and OpenAI's attempt to get sort of the early entry into the S&P 500 So our pension funds Rahul Rahul (1:00:31) Second, it is great news, Dan. Dan (1:00:46) I guess we don't really have those, but ⁓ our four one Ks, whatever. If we don't have those, yeah, we've got those. are are safe for the time being because ⁓ they won't immediately gain entrance into the index, which as we talked about in the previous episode would have essentially like artificially given them like additional billions that I don't think the market it to me it's kind of felt like cheating the market, right? You're not you're not like Shimin (1:00:50) Mm-hmm. Dan (1:01:09) Allowing the market to decide if you should be in that position or not. You're just getting it because you're part of this index fund that everyone's already bought into. So Shimin (1:01:17) I I third that and this feels like one of those spam advertisement article ⁓ headlines. It's like you can get billions of dollars with this one hack or one trick with this just just one trick. Rahul Rahul (1:01:17) you Dan (1:01:28) Yeah. All you have to do is get pre listed on the on an index. Rahul Rahul (1:01:35) I would like to give a shout out to S&P 500 Index Fund people. There is no names name, but like in a world where everybody's lacking courage by the day, these people stood up for, know, what the index was set up for. Didn't bend to someone else's needs. So great job. Keep it up. Shimin (1:01:35) Yeah. And Well and Rahul, ⁓ your article this week Rahul Rahul (1:01:58) Yeah, I didn't know Berkshire Hathaway was doing 10 billion or so in equity. Alphabet is raising 85, almost $85 billion in equity to fund their AI ambitions. They're going to like 190 billion or something in spend, but they're also one of the people who are getting heavily like... adopted in the enterprises and you know, I just go look at Gemini enterprise agenda platform and you'll know what I'm talking about. And so they Yeah, they the demand is crazy and they're funding it with you know, a lot more equity raises. And yeah, that's kind of the news there. Shimin (1:02:33) That's Dan (1:02:34) Or this this little company called Apple. who plans on using Shimin (1:02:37) Mm-hmm. Yeah. Yeah, altogether, how do we feel about the clock this week? one word I keep on thinking of is ⁓ incestuous. we have money being raised from individuals into Google that then gets given to XAI, right? And and now Apple is also renting it. So a lot of won't say self dealing, but but something like that happening in the marketplace. Dan (1:03:14) I'm Rahul Rahul (1:03:14) I'm long Google, so, but do your own research. this is, between, know, Siri intelligence, finally, they were talking about that it's trained on top of Gemini models to this thing. It's great, at least that Google piece. Dan (1:03:21) As always, we do thank you. Shimin (1:03:21) is not financial advice, as always. Dan (1:03:35) I also think that they well, they also have the TPU going for them, right? So I think like here here's a very plausible scenario that I think have plays out in the next six months. So we've got the in my opinion good news about the S P five hundred. That means that you have to trade on your own merits, essentially, right? For for these companies. We now have three of the Frontier Labs going public. Shimin (1:03:36) I'm I'm long Google as well. Not not a financial Rahul Rahul (1:03:38) Yeah. Yeah. Dan (1:04:01) Which exposes them to I think price pressure in the next there's a we didn't cover it, but OpenAI just did a secret S one this week. So we have Rahul Rahul (1:04:04) Which is the third one? Shimin (1:04:04) Mm-hmm. Rahul Rahul (1:04:11) But you're XAI as a frontier. Shimin (1:04:16) Yes. Yes, they are one. Dan (1:04:16) Yes. I mean they have their own model. People like hundreds of people pay for it. Hundreds. you know. It's more than pay for my model, but anyway. so the price pressure, this is just, you know, a plausible scenario. Price pressure becomes real because all of a sudden Rahul Rahul (1:04:19) Bye. Hehehehehe Shimin (1:04:25) That's that's that's that's Dan (1:04:34) Do the first couple earnings come out and people realize exactly how much they're spending on training and inference and all these other things, you know, depending on how transparent they are in their accounting. And then, plus it's like as soon as you're a public company, all of a sudden there's this massive push for like efficiency, right? We got like Meta's era of efficiency and all these other things that have been going on with like efficient investing license lately. So will they be immune to that? I think is the fundamental question, but like Plausibly will say that they aren't. And that triggers token tokenagettin. Token tokenageton. Well, how did I say that? Which means the price of tokens skyrockets. plus like we might get there also just through capability, right? As we're already seeing with like this sort of mythos pricing thing that we talked about at the the top. and once we're in in tokenagaton. Rahul Rahul (1:05:05) Yep. Shimin (1:05:05) Talking about, yes. Dan (1:05:20) What saves you while having really efficient hardware to do this on, which to me looks like a TPU. or, you know, maybe, but they're not that efficient. Like we've talked about that in detail on Hardware Hut because, you know, at the end of the day, there's just still GPUs. and until we start doing things like disaggregated memory and and other pieces like that to really like make a play for extremely efficient inference. Rahul Rahul (1:05:26) And where are you too? J.P.'s. Yeah. Mm-hmm. Dan (1:05:45) ⁓ like something approximating purpose built's always gonna be better, you know? I mean you see that with like what is that? I'm forgetting the name of it, but there's that other company that's like basically baking models onto a chip and they're like they're they're you cannot beat them in terms of inference performance right now. It's just like stunning amount of tokens per second. So Rahul Rahul (1:05:49) Yep. Yeah. Yeah. And Chat Jimmy, I think, is probably still killing it on the speed. Shimin (1:06:08) Yeah, o I've got a name of that company. But yes. so Dan, is that is that a vote for moving the clock forward or back? That's that's Dan (1:06:16) I don't we don't have enough info, right? That's the thing, is because we don't ⁓ until earnings come out, which is we're a long way off from this, we're not gonna really know how that's gonna play out. that's just a p scenario that I think could be plausible. So but but I will say that maybe the the no index fund thing pushes us a tiny bit closer. Not a lot. Because it it it could be like the straw, but not the straw that necessarily broke the camel's back. It was like the fifth before that one, you know in the pile of straws. Shimin (1:06:48) So so there there's yeah, 'cause 'cause there's no No inclusion on the S&P 500 index, therefore the pricing pressure has increased somewhat. So that makes the potential bubble bursting a uglier burst. Yeah. Dan (1:06:57) Somewhat, yeah. Rahul Rahul (1:07:03) Yeah. Dan (1:07:04) Or like plausible, where it may not have even felt plausible like six weeks ago, right? Not saying it's likely, just plausible. So Shimin (1:07:08) That's fair. But but not significant to move it. Okay. ⁓ Rahul Rahul (1:07:13) And those Shimin (1:07:13) some Rahul Rahul (1:07:14) still get NASDAQ and whatever the other ones. So they're not completely out there. Just not going to be getting fast entry. They're not going to be getting fast entry in the S&P 500. Yeah. Dan (1:07:19) ⁓ I didn't think about that. Okay. It's only the S and P. Hm. Shimin (1:07:25) So it's it's only a couple of hundred billion as opposed to tens of trillions. So like in the grand scheme of things, what is money? how about Rahul Rahul (1:07:29) Yeah. That's one founder's fun investment. Yeah. Shimin (1:07:36) About five five minutes and twenty seconds in honor of the twenty million investment that founders found put into SpaceX. Dan (1:07:43) Sure. Rahul Rahul (1:07:44) Oh yeah, according to Gemini, at the IPO target valuation of 1.75 trillion, that 20 million that they put in SpaceX back in August 2008 is roughly worth $26 billion to $52 billion. It's 130,000 % return on investment. Shimin (1:07:44) Alright. Do. Dan (1:08:01) All right. That'll lie between one to two yachts. Rahul Rahul (1:08:06) insane air. Shimin (1:08:06) It's nice chunk of change. Yeah. All right. Well, with the clock set at five minutes and 25 sec or 20 seconds, I think that's a wrap on the show. ⁓ thank you all for joining us on our study session again this week. If you like the show, if you learned something new, please share the show with a friend. You can also leave us a review on Apple Podcasts or Spotify. It helps people to discover the show and we really appreciate it. If you have segment idea, a question for us, or a topic you want to cover, shoot us an email at humans atipod.ai. I'd love to hear from you. You can find the full show notes, transcripts, and everything else mentioned today at www.adipod.ai. Thank you again for listening, and we'll catch you next week. Bye.