Latest / Artificial Developer Intelligence / OpenAI Fires Its Safety Researchers, Agentic Primitives & More Code, Not More Software

Transcript

OpenAI Fires Its Safety Researchers, Agentic Primitives & More Code, Not More Software · Artificial Developer Intelligence

• Shimin (00:00) Hello and welcome back to Artificial Developer Intelligence, a weekly conversation show where three software developers try to sort out the hype around AI from what it is delivering. My name is Shimin Zhang, and with me today are my co hosts, a Dan absorbed by downstream constraints in the production process, Lasky and Rahul call him Homo Luddend, Yadav. Hello gent, how are we doing? Mm-hmm. It's Latin for something. Rahul Yadav (00:22) What? Dan (00:23) Ludden? Yeah, okay Rahul Yadav (00:29) Yeah, fancy $20 word right there. Dan (00:32) Doing all right. Thank you. Rahul Yadav (00:36) It's getting drearious, I mean, I don't like it. Shimin (00:39) It is spider season. So Rahul Yadav (00:40) So. Shimin (00:40) I I've been staring at this huge ass spider on my ceiling for like all of this morning. pff yeah. Dan (00:47) Isn't it always spider season? Shimin (00:48) No, just October and half of November where all the baby spiders hatch. It's real fun. Dan (00:52) Huh. Huh. Shimin (00:54) All right, on to happier news. Rahul Yadav (00:54) Right in time for Halloween. Dan (00:56) Yeah, I was going to say. Shimin (00:58) Yeah, it's always spooky here for Halloween. what are we gonna do on this week's show? as always, we're gonna start with the news thread mail. We're gonna be talking about OpenAI's fired safety researchers and some news about follow-ups on the pacing of the frontier from other labs. Dan (01:16) Then we're going to hop right into the tool shed where we talk about a dintic primitives. Shimin (01:20) then we're gonna do a model review. As promised we talked about Gemini for Argon last week. So this week we're gonna go a little deeper into how it does on the benchmarks. Dan (01:29) And then in a very surprising twist, Rahul has given us a vibe and tell, which is it his first, I think it's your first vibe and tell, right? And then on top of that, he won't Rahul Yadav (01:38) second I don't know. Dan (01:40) tell us what it is. So we're going to find out along with y'all, as we get into that. Shimin (01:45) My money that's gonna be spider related. then we're gonna do post processing where we're gonna talk about AI agents generating more code but not more software. Dan (01:52) And then we're, no, we're not doing it deep dive. Whoa. Okay. I got tricked by my own, my own sheet And then as always, we're to jump into two minutes to midnight, where we talk about the sort of financial side of AI through the lens of the bulletin of atomic scientists clock. Shimin (02:08) All right, let's get started. Dan (02:09) Cool. guess that's me. Yeah. So, it's kind of funny cause we had just talked recently about the, possibly sex cult related behavior of safety researchers and now here they are in the news. so the big headline is, open AI has fired three security researchers, Jasmine Wang. I'm Tom, I'm going to slaughter your name. I'm sorry. Tomic Korbach and Keita balancey. supposedly they're, stated reason for firing them is that, they talked to an outside, safety organization, third party safety organization and violated the company's policies for accessing and handling sensitive company information and doing so. but the other reason why this is kind of like that happened last week, but the reason why this is kind like back in the news this week is the three have written an open letter. about their experiences. And, I'll just quote a couple of pieces of it, right. Cause it's, it's pretty good. it's quote unquote, we have become concerned that internal and external communications around our firing have made our former colleagues afraid to speak and operate in ways that until last week were an integral part of working in open AI. The researchers wrote, quote unquote, AI is not a normal technology and open AI is not a normal company. Those of us who work on safety see risks before anyone else. And we rely on close collaboration with outside experts to work out how to address them. The freedom to do so without fear and to have well-defined internal procedures that enable this work is itself an essential safety mechanism. so yeah, they're, they're sort of rebuttal was that the incident that they're Talking about that supposedly got them fired was actually the one that's made all the headlines recently around the escape of open eyes agents. I believe it's probably the first one. it was the first one that made the news. Yeah. The hugging face Shimin (03:50) The hugging face incident? Yep. Dan (03:52) one. and, their claim is that they were literally writing the book as they were doing investigation because it was all happening really quickly. So you can infer from that that the third party organization might've been like METR or something like that, right? Cause they'd done that report. So Shimin (04:07) Yep. Dan (04:08) I don't think they explicitly state what it is, but it was kind of interesting. But so opening has not like officially responded to it. But TechCrunch, should we, you know, sort of stole this info from as did get their hands on an internal memo that said they're being fired is because information was leaked to the information as in the publication about the less monitorable architectures that we'd also sort of talked about previously on the show worth like the, Shimin (04:39) Yes, this is the loop transformers. Dan (04:40) yeah, exactly. With the, know, so you can't see as much chain of thought reasoning on there. And They've denied all involvement with that leak in in the letter. So interesting times. Shimin (04:51) Yeah, interesting times. I did find that the meter or is it meters, third party write up for the hugging face incident to be surprisingly detailed. Like they had access to a Dan (05:01) Mm-hmm, same. Shimin (05:02) lot of internal data. I I at the time I thought this was like it was really great for AI safety that they had so much transparency around it. But but Dan (05:08) Yeah, like very close collaboration, yeah. Rahul Yadav (05:11) Yeah. Shimin (05:12) maybe not. Maybe it wasn't intentional. as for the loop transformer, yeah, that was also a little unusual because usually we don't learn anything about the model architecture of Dan (05:21) That's true, like how it works internally, yeah. Shimin (05:24) yep. So that was surprising and they usually the Frontier Labs do a pretty good job of you know cracking that down. so maybe we will see. Or maybe we won't. Maybe we'll never find out. Maybe this is just we're gonna forget about it. Like, hey, remember back in 2026? Dan (05:33) Yeah, yeah. Let's see, well... Shimin (05:38) Safety researchers got fired while we're working in the robot mines all right. Thank you for that, Dan. my news for this week are actually two items. One is an X post. Twitter post. Oof, sorry, I just caught myself using X naturally. I meant this was a Twitter post by Microsoft CEO, Dan (05:55) Itter. Shimin (05:58) Satya Nadella, titled Models. As insider risks in the super intelligent era. So first things, guys, he called it super intelligent era I just I just want to point that out. we have a CEO getting a Dan (06:10) Hahaha! Rahul Yadav (06:11) It's the official term. Shimin (06:12) medal. It's yeah, Dan (06:13) Uhhhh... Shimin (06:14) just like Lake Ontario Lake America. Yeah, it's Dan (06:16) Yeah. Shimin (06:17) it's it's somewhere in the laws, they they must say it. but this is a kind of Microsoft's approach or Counter argument to pacing the frontier, right? Nadela is here saying that AI is getting better, but we cannot treat them as these magical stacks of black boxes. He did not go as far as saying we need to pace the frontier, but he did say that and and this one I really found interesting that model diversity is one of those things that everyone should work on. because no, and I quote here. No one model should become the sole dependency for an important outcome or be responsible for for verifying its own work. Right. we almost need to keep the ecosystem diverse in order to make sure there are no blind spots. And then the rest are, observe everything, verification, independent controls, audibility, containment, and then incident disclosure. these are all pretty good cybersecurity best practices, but he went far short of saying, hey, we should maybe slow or halt the development of the the latest models. Okay. And my second post in the same segment is from our favorite billionaireslash trillionaire. Well, maybe second favorite. Mark Zuckerberg says Dan (07:30) You Shimin (07:31) that Meta has reached the AI frontier. it came out this week that Meta released its latest model, Muse Wonder, and it scored extremely well on the at least the artificial analysis intelligence benchmark. and we will probably do a model deep dive at some point about Muse Wonder. But what is interesting about this Financial Times article is that Zuckerberg was quoted to not be as concerned about pacing the AI at all. he said, quote, I'm much closer to the belief that AI is a technology that can be engineered with normal guardrails and built responsibly than the Anthropomorphic vision of AI, which is that it is just going to go do its own thing. so we have two other frontier-ish labs. Meta is now officially a frontier lab, right? Microsoft is not quite there, but I'm sure they will be at some point because they have been releasing their own models, who are not looking to pace the frontier at all. so if Pacing the frontier is a collective action problem, and I believe it is, right? It's either the government says you shouldn't do it and our current US government is unlikely to do anything of the sort. then Rahul Yadav (08:39) Yep. Shimin (08:41) we already have defectors in this game, right? Meta and Microsoft do not seem like they would want to pace it at all. So I guess strap in. Dan (08:51) Yeah. Meanwhile, Anthropic is worried that AI is going to use five smart TVs to run itself. Rahul Yadav (08:57) yeah. Shimin (08:58) and and we have to apologize for or for mocking meta for having all of its engineers solving puzzles because clearly solving the puzzle worked. Dan (09:05) You think it was the puzzles that did it? Shimin (09:07) Muse Wonder Max scored 56 right below Opus five five and Sonnet five five in the artificial analysis intelligence benchmark. I'm sure it helped. Was it worth Dan (09:17) Yeah. Shimin (09:17) it? Question mark, but I I yeah. Dan (09:20) Only time will tell. Rahul Yadav (09:20) one hopes. Shimin (09:21) All right, moving on to our tool shed. this week we have an item brought to us by Rahoul. Rahul Yadav (09:27) So this is by Charlie Guo writing the artificial ignorance, substack newsletter. And this one's about a lot of the things that have become pretty mainstream this year. All these agentic primitives were, spent a lot of time building the harness around the models. And so this kind of breaks the different primitives of the harness down. The way Charlie breaks it down is in three different buckets. The first one being doing the work. The second one is continuing the work and the third one being delegating the work. so under doing the work, have first of all, your system instructions of, you know, make sure to not IRMR off my root directory, please, and Dan (10:08) Hehehehe. Rahul Yadav (10:09) thank you and do all the things, but don't make any mistakes system level instructions. But then also as needed, it can pull from different skills. So not all the skills should be loaded in the context all the time because that's a big waste of context, but it can pull from the skills where needed depending on what the job requires. Next one under that is tools and connectors, which is one step. After skills, or if you need an advanced way to make calls and rely on third party things, that's how I look at tools and connectors where you can either access the tools that are on the box or you can make calls to things that are not on the box and connect to them and carry out your work. And then finally, the concept of sandboxes where All of this would be better if it runs not on our machines where it dangerously skips permissions and does all sorts of crazy things. So putting all these things in a sandbox also that way you can run things in the Claude. It can be ephemeral. You can spin it up and down as needed versus running it all the time. Next up we have continuing the work. so those would be first of all things like sessions where I don't know if you guys worried about this much but we would run out of context very quickly and slash compact was like yeah just smash that every 60 seconds or something and now Dan (11:34) I seem to recall a dance, early, early dance rant about that very topic. Shimin (11:39) Mm-hmm. The one the OG rents. I mean s session Rahul Yadav (11:38) Yeah. Shimin (11:42) session is a is a hacky workaround for the context window. Ideally we would Rahul Yadav (11:47) Yeah. Shimin (11:47) like to not have sessions and definitely not compaction. Rahul Yadav (11:50) It's compaction. I don't really worry about it as much anymore. I don't remember the last time I actually had to compact anything. Dan (12:00) This guy's got a token budget. Rahul Yadav (12:03) Well, the harnesses I think also do a good job. On a slight aside for people who listen to our episode about OMP, I was telling Dan and Shemin off the call at one point, has a special command slash shake, which just gets rid of all the tool call output, which is most of it you don't really need anyways, and then instantly drops down the context by sometimes 20 % or even more. Shimin (12:27) And the other thing that I see more and more models do now is if you ask it a question that is best suited for a sub agent, it will automatically kick out sub agent with a particular task to deal with. So you never have to waste your main context with all that added session data. Rahul Yadav (12:42) Exactly. And then final thing under continuing the work after compaction is schedules and events. instead of having something just continuously running, but it only actually does anything once in a while. Open Claw reason, one of the big reasons why it caught on fire was it was able to do these things based on triggers. Dan (13:03) When you say gone on fire that way, I feel like... Shimin (13:07) You may caught the world on fire if Dan (13:08) Yeah. Shimin (13:09) not the database is on fire. Rahul Yadav (13:10) Sorry, yes. Dan (13:12) or that Meta Security Researcher's Inbox. Rahul Yadav (13:18) So now I think almost every AI agent has the concept of schedules just out of the box where you can schedule tasks to run at a certain time and then it just, yeah. Dan (13:28) Yeah, it's like the only thing I use with them. Shimin (13:32) Okay, so here's she has an idea. Like instead of having a programmatic heartbeat of once x number of minutes, like why don't we have the agents decide when it should next process its session? Dan (13:44) Well, I have a schedule ones are just, they're dumber than that. It's just like a cron. It's basically like with Shimin (13:48) It's a crunch up, right. Rahul Yadav (13:48) Yeah. Dan (13:49) a prop plus a prompt, and then the agent can pick its own prompts to fire it back up. least that's how Hermes does it in terms of the skills. Rahul Yadav (13:54) because that would be Shimin (13:54) But yeah, but cron jobs are d are dumb. it doesn't know Dan (13:58) Right. Shimin (13:59) if Rahul Yadav (13:58) Yeah. Shimin (13:59) if this thing that if there's a lot of news related to this particular thing happened, it should update itself to be triggered more frequently, right? Dan (14:06) Yeah. don't, I don't, does open claw use heartbeat? I haven't used it enough to know, but it was something I was playing around with back when you and I were experimenting with pie. Shimin (14:15) Yeah, open claw uses heartbeat. Dan (14:17) Hmm. Rahul Yadav (14:17) Yeah. Shimin (14:17) Which is just a cron job under the hood, right? Dan (14:19) Well, yeah, it's like, but every five minutes and it's like checking in on everything, ideally with a cheap model. then if something happened, then fire up a bigger model and look at it. Rahul Yadav (14:28) If you do have, you can also do like event based where you could spin up a box with an agent when needed. So that could be another way instead of the agent continuously checking for something even at X intervals with the. Shimin (14:45) Yeah, the MCP events, right? that idea makes sense for enterprise workflows. Or or maybe even for my for my own an inbox agent. If if an inbox if a new email comes in, pick up the agent again to read it. Rahul Yadav (14:57) Yeah, parse the thing, tell them, sorry, this is no reply inbox, I don't know what we're doing. Shimin (15:03) I will not pay this bill. No. Talk to my lawyer, please. Dan (15:05) Ha ha ha ha. Rahul Yadav (15:09) Sorry, Shimin has moved from this email inbox address. Dan (15:14) You Rahul Yadav (15:17) And then finally, we have delegating the work. And so this is where Ash, when you were saying earlier, Main category here is instead of just the one agent doing all the job and then dealing with increasing complexity as the job gets longer and longer, you can delegate it to subagents, have them do the work, and then they can report it back to the main agent. down in this section, Charlie reports that the AI that solved the Navier-Stokes Millennium Price Problem was, in fact, a group of around 10,000 agents. So that's Dan (15:47) No, Rahul Yadav (15:47) the kind of power that... Dan (15:47) only 10,000. was like Ultra Max. Shimin (15:49) Yeah, and we haven't talked about this. but there folks are seeing you know, sub agents and s agent swarms as a new form of compute time scaling, right? So like Rahul Yadav (15:58) Yep. Shimin (15:59) instead of taking longer you just throw more compute at it. And we're still trying to figure out what kind of problems are sub agent swarms good for and which ones are not. Rahul Yadav (16:07) Yeah. And there are options today where they would say like, yeah, this is the main agent, but you can spin up sub agents. and they're slowly getting smarter where they do spin up their own when needed. But I still feel like sometimes I'm always like, why am I even given this option? You just figure it out yourself and make it easy for me to just talk to one thing. So. It may be just an easy preference there. And then out of this, get some, finally some emerging patterns that Charlie touches on. First one being memory. That's a big problem right now where one of the, if we want agents to do tasks on the order of days and weeks and months, how do you actually maintain memory? How do you decide what to keep, what to get rid of? Because for humans who have working memory, we just take all of that for granted, but it's a very hard problem to solve. so a lot of people are working on this right now. The dumb simple thing that most people do today is using markdown files and that works decently. Well, for today's use cases, but how does that change over time would be a big problem to build a long-term durable memory. Shimin (17:23) Or or maybe fine tune it into the model itself, continuously updating the weights. That's that's Dan's preference. Yeah. Dan (17:26) That would be ideal. It's a hard problem though. yeah, Rahul Yadav (17:28) true. Yeah. Dan (17:33) cause it's, it apparently it tends to like cause the models to, like overcompensate for the most recent input, which then after a while, like not even after a while, after like a little bit of data, data like destabilizes the entire model. So search is like spouting nonsense, like the early, early GPT is Rahul Yadav (17:47) Yeah. Shimin (17:47) Right. Well that's what that's what thinking machine and their differentiability is going to be, right? Is to like fine tune your own models on their platform. So hopefully they can make some headweights there. Rahul Yadav (17:59) Yeah. Next up we have proactivity, which is what we were somewhat touching on earlier is if we have this ambient AI everywhere, why must human run a cron job that checks things once in a while? Why can't it just do it when an email comes in? But also all sorts of conducting commerce online, doing things at work that are all responding to events that are happening versus continuously checking for if something changed in the world around us. Dan (18:32) You know what Rahul? Rahul Yadav (18:33) What? Dan (18:33) love the smell of ambient AI in the morning. Shimin (18:35) Ha ha ha. this reminds me of the agent trading simulation we spoke about last week, right? the idea is there's there's actually two different gaps I can think of when it comes to having an agent that is proactive to your needs and wants. One is it has to model your preferences correctly. That's the the one from last week where it needs to know which books you actually want to have. and the second one is the humans. Need to agree on whether or not our stated preference and our behavioral preferences or re our revealed preferences are actually the same. And we have to deal with the fact that usually those two are not the same. Dan (19:18) Yeah. Rahul Yadav (19:20) Yeah. Shimin (19:20) And I don't know what that's gonna say about us when everybody like it's easy when it comes to really simple Dan (19:23) It'll be learning experience. Shimin (19:26) heuristics like just make me money or save me money. Those are easy. But what about using your agent to be your matchmaker, assuming you're, still dating or or make friends for you, right? Like that's Rahul Yadav (19:39) man. Shimin (19:39) gonna be rather difficult to do. Dan (19:41) I use mine as a hardware matchmaker where it was tracking the Shimin (19:44) Ha ha. Dan (19:45) prices of the, what is it called, GTX spark machines, Shimin (19:48) Mm-hmm. Dan (19:49) the new ones that Ben Video's doing and boy, six grand, almost seven grand for one of those things. Shimin (19:55) then I guess you're polyamorous with your machines. Back to the sex cult. Dan (19:58) yeah. I have a lot of computers. I try to love them all equally, Shimin (20:01) He's got a closet full of Dan (20:04) but it's not easy. Rahul Yadav (20:06) and then the final one here is identity. And this was the most, interesting one to me. similar to what you just said, Shimin of like agent does the work for me. Charlie breaks this down into there's two main ways that, people handle this to their first is impersonation, which is the agent just takes our, identity of the user and then carries out the work on their behalf. The second one being delegation, where the agent has its own identity and its own accounts, its own credentials, and it's not acting on your behalf. It's just doing the work that you delegated to it, but the identity is separate. And I feel like if you combine this with the ephemeral sandboxes and some of these other concepts, I could see a world where if you don't maintain proper identity, just tracking something down could become a big pain in the ass where it would be like, what thing did something on someone's behalf and maybe it happened a long time ago. But if you don't have like very specific identities tied to these things, it would be very hard to trace things when they go wrong. And it's a big problem to solve. And probably also we're going to get into like legal governance territory. here because someone would have to step in and be like, no, this is what it can do. And you have to assume the risks if you're having a big thing. Dan (21:26) Well, there's also an interesting intent thing too, that I've personally run into with my little agent set up. I've got a home, right? Which is there's all this bot detection stuff like Claude flare and everything else that's meant to like keep bots out of websites. Rahul Yadav (21:40) Yeah. Dan (21:40) But is that the, should we continue to treat that the same way? If like that bot is trying to act on my behalf to do something that like I might not otherwise Shimin (21:49) Mm-hmm. Dan (21:50) do. So it's a really. nuanced area that I think is going to be increasingly interesting as agents spread. Shimin (21:55) With how popular Muse got, we saw Apple crackdown permissions. We just heard that JP Morgan Chase has decided to lock down its app from agentic usage. I saw this on C C N B C earlier today. With how popular Muse is, the platforms have are having their hands forced. Right. Like they they have to choose either lockdown or like be okay with it, whereas building an entire platform and credentials and identity management just for agents is gonna take a lot of work and trial and error and preferably not with people's bank account first. Rahul Yadav (22:32) Yeah, which is both a, cause the internet was, built by humans for humans. So that's what I feel like we're seeing is, JP Morgan, Amazon did it with perplexity earlier this, I think it was earlier this year, or maybe last year. Dan (22:47) That was a while ago, wasn't it? They did the original band and I think they just banned Muse recently. Right. Rahul Yadav (22:52) Yeah, and then they banned Muse recently because it's built for humans, but then the opportunity there is someone could spin up their Chase Bank, version of Chase Bank or a bank that is agent first and Amazon that's agent first. And there's going to be a lot of opportunity there for people who want to solve this problem. Just say this part of the internet is for agents only, humans don't even bother to spend time there. They might not be able to coexist. Dan (23:24) It'll be just like the message board that the open Claude thing spun up. What was that? Mold board or whatever? Shimin (23:28) Ha ha ha ha ha. Dan (23:30) But then it turned out humans were posting as, it's all just a big mess. Rahul Yadav (23:34) yeah first Shimin (23:36) Build Rahul Yadav (23:36) they impersonate us then we impersonate them too yeah Dan (23:38) You can impersonate them. It's just, yeah. Rahul Yadav (23:42) what was that on the internet Shimin (23:41) They will never know. Rahul Yadav (23:44) no one knows you're a dog that meme Dan (23:46) Hahaha Shimin (23:49) Or you could even be a human. the other Rahul Yadav (23:51) Yeah. Shimin (23:53) thing I was gonna say is you probably have management level agents that deals with all the all the agents that you can just talk to every day and be like, How are my other agents? And then you have middle management and CEOs and everybody can be your own CEO. Yeah, this is the power of agents. Rahul Yadav (24:07) Then you go to a skip level with the line agent and they're like, man, the middle management, you got to do something about it. They're not telling you everything. Dan (24:17) Those guys don't do anything. Why do they work here? Why do they have a token budget here? Shimin (24:16) Ha ha ha. And yes, Dan, I always suspected you're at least partially canine based, of course. Dan (24:29) well, you got that. Rahul Yadav (24:29) Hehehehe We got dots now, whatever grok bot, we got muse now. So these things are already here and we'll see how this evolves. Shimin (24:41) Okay, let's go to a deeper model review of Gemini four Argon. Rahul Yadav (24:48) But before you start this, I would just like to apologize to the audience that he's not covering Haiku 5.5. Shimin decided to pick Argon, and if he's right, please let him Dan (25:00) Who's back? Only Raul is excited about that. Rahul Yadav (25:05) know that you'd like him to cover Haiku 5.5. I complained and they released it within days, so I think it deserves a section, Shimin. Dan (25:15) The haiku fanbase is interesting to me. There's just like this little diehard segment that really likes it. The only Rahul Yadav (25:20) Yeah Dan (25:21) thing I've ever used haiku for is writing haikus because I thought it was funny. Shimin (25:25) I think some systems used Haiku as a classifier a lot of times because it's so cheap. Dan (25:29) including clot code, yeah. Shimin (25:31) Yeah. But now we have Jev So and and Rahul Yadav (25:33) You got it. Shimin (25:34) just kind of I guess a sidebar while I have the artificial analysis page pulled up, Haiku Max did score four forty-three on the intelligence index, which is around the same level as Kimi K three Max. Rahul Yadav (25:46) Interesting. Shimin (25:47) So it it's pretty good. Rahul Yadav (25:49) Yeah. Shimin (25:50) but back to Argon. Aragon, we talked about it last week. Gemini's latest model, it is frontier level 53 on the same index or on the same as GPT six Astro Max. And are we ready to see it's Hamlet soliloquy? I just gotta do that today. So as a reminder, this Dan (26:12) I actually forgot about this. Yes. Rahul Yadav (26:15) yeah. Shimin (26:16) This is the GPT-6, Rahul Yadav (26:19) State of the art Astra. Shimin (26:21) yes, state of the art Hamlet soliloquy. It's quite good. It noticeably have very good hands, or hands that actually look like hands, even though it's flipped, and pretty Dan (26:30) you Shimin (26:32) good facial expressions, and it's able to, you know, use a web API-based voice output for the soliloquy. And that's it's Pretty good. I liked it. One shot S V G only, not using Blender or anything like that. And here I'm pulling up the Gemini Four Argon soliloquy of the same. Dan (26:52) Argon built you a whole app. Shimin (26:54) Yes, so that's the first thing. Ur Argon went above and beyond. It it had the main stage. It had like a little dictionary of sidebar of Dan (27:02) There's a whole sidebar of yeah. Shimin (27:05) all the lines, along with Dan (27:06) the emotional beats and... Shimin (27:08) Yup. Bitter indignation. Visceral anguish and grief for each of the lines, for each of the beats, which I which I really enjoyed. And of course it does perform. Dan (27:19) yeah, it's even got like a little bouncing Shimin (27:19) Right, so Dan (27:21) highlight of the Rahul Yadav (27:22) Yeah. Dan (27:23) text rolling through the... Shimin (27:25) Yeah. it has a richer background. I also do want to point out its eyes follows the cursor. These Dan (27:28) his eyes follow the cursor. you see that? Rahul Yadav (27:29) It follows your... Yeah. Shimin (27:32) are the kind of delightful user experience things that Dan (27:36) Hands are okay. Shimin (27:38) Shaky. Rahul Yadav (27:38) I like what's going on with that flame in the background too. Dan (27:42) yeah, Shimin (27:43) Yeah. Well one. Dan (27:43) the flame appears to be sentient. Shimin (27:45) It it has additional lighting in the background. You see that? Compare the difference. Dan (27:48) That's true. Multiple light sources, yeah. Shimin (27:51) There's yeah, and it actually has a background track, which you guys cannot hear and the user maybe I'll cut in for the audio section. It has this like foreboding background sound with like little animal noises, even though it's on pa it's pause right now. It's pretty wild. and what else I really Dan (28:08) There's also the facial expression and gesture rig inspector complete with all these sliders. Shimin (28:15) Yes, we have to play with this. Rahul Yadav (28:18) Ugh. Dan (28:19) What is it? Brow furrow. Excellent. Rahul Yadav (28:21) Yeah, you can Shimin (28:21) You can see the brow furrow level. Rahul Yadav (28:21) see it. The... Yeah... Dan (28:23) Inner brow grief, eye openness. yeah. man, we've got a very high hamlet. Shimin (28:25) I'm opening and closing the eyes right now and you can see it. Rahul Yadav (28:28) Yeah. Shimin, can you do the tear and glisten that's below, it's just right below the inner brow? I would like to. Aww. Okay. Shimin (28:41) Yeah, and the steel follows the cursor. So pretty good. I will say this is a pretty good consider it's of course a visual first model, a multi model first model, right? Like it has pretty good visual display even in S VG and I don't like the art style, art direction as much as the open AI one, but I don't know. You guys impressed? Rahul Yadav (29:02) Yeah, I like it. Dan (29:03) It's of emo and what the hands are artistic, right? So they're basically circles with a bunch of fingers on them, but it manages to come off as a hand. Shimin (29:11) yeah, even though you can see the finger separation at point. Dan (29:14) Mm-hmm. Shimin (29:14) the last thing is it The lips are synced to the actual line, unlike the open AI Dan (29:19) That's pretty cool. Shimin (29:22) one. So if you play a particular beat, you can see it like actually Rahul Yadav (29:26) Yeah. Shimin (29:27) matching the the words that's said. So it's all very cool. I quite like this. Rahul Yadav (29:32) And I'm assuming Shimin (29:32) So it Rahul Yadav (29:33) you gave it the exact same prompt you had given Astra and everything. Shimin (29:36) I gave it the same exact prompt, which is create an S V G Shakespeare. Dan (29:38) That's the benchmark. Shimin (29:40) Yep, that's a benchmark. Rahul Yadav (29:40) Yeah. Do you have the tokens used and cost and all that too? Shimin (29:47) No, I do not have the tokens use for this one. I w yeah, I Rahul Yadav (29:50) No worries. Shimin (29:50) I'm curious how much it costs. I'm doing this via chat, so it's it's not very clear what the tokens use were. and I'm gonna mute that window next. and the next thing so much better with all that foreboding background music, blasting. Dan (30:01) It's playing music all time. We're just Rahul Yadav (30:04) He's just crying in the back this whole time and shim is like why am I feeling emotional? This one's hitting hard you guys Dan (30:05) like oblivious to it. He gets, he gets quieter and quieter on the podcast. We're like, Shimin are you Shimin (30:10) Why do I hear coyotes in the back? Dan (30:15) okay? Background music got to him. Shimin (30:17) So the other the other benchmark I ran, of course, is the the effects of AI going through the very diff various orders. just to see its reasoning abilities and how creative it gets. overall, the response is is good enough that I find it hard to distinguish between Gemini and Astra and Fable and Opus Five Five. That said, we can find some peculiarities between each model's generation, right? for one the things that the older models have is they go straight to like AI is gonna ask us to question what does it mean to be the to be human? And that kind of has sense died down a bit. So off the top of my head, that's my biggest reaction is the Gemini version does indeed go back to what does it mean to be human. which I wonder if it has something to do with their fine-tuning slash reinforcement learning process. And the second thing that I really like from Gemini is it is very proactive. Again, it's it's just a fairly proactive model. It's proactive about building these UI widgets for the end user, which reminds me of Rahul your article from last week about like the user interface should not just be text, right? So Gemini 4 argon is very good about explaining topics via these like nice flow diagrams. So at the end of every single user interaction, I guess if war if warranted, it will kick off a sub agent that just builds these nice widgets for the user. As for the actual content, there are some interesting bits I just want to quickly run out. first one is it's got some nice, I guess, mathematical abilities. so it has a pretty nice way of explaining, you know, why agents are not reliable. a 98% step level reliability across a 30-step workflow yields only a 54% N2N success rate. and it shows the math, which I'm just gonna assume is correct. That's nice. That's a that's a kind of nice hand holdy put the numbers into into or put the words into it actual numbers that's happening. it has some good ideas about what is gonna happen to code reviews. It thinks code reviews will be replaced by multi agent adversarial harnesses, where you have a basket of agents as the bottleneck for code review as opposed to human. So it did clear cor correctly defined code review as the Dan (32:36) I mean, that's definitely what I've been exploring for first pass. I still do a human one, but I like to run a different model against it and see what it finds. Shimin (32:43) Yeah, for now. Yeah, I mean it yeah, agents should definitely be the kind of fur first line of defense against the code that other agents output. in the second and third order effects territory, it talks about flight simulators for knowledge work and the return of guild. so I already like it. it also talks about biodiversity mandates for AI. Rahul Yadav (33:05) So. Shimin (33:06) So similar to what Nadella was saying. have a diverse agent ecosystem. Rahul Yadav (33:10) you've never published anything related to flight simulators or knowledge gains, have you? On your personal blog? Shimin (33:15) No, I haven't. Rahul Yadav (33:16) Okay. What would be hilarious is if it's answering your question based on what it believes your views are and then someone Dan (33:24) Hahaha! Rahul Yadav (33:26) else goes and... yeah. Shimin (33:25) That's always possible. Yeah, but I don't I don't use Gemini a ton as my thinking partner. So maybe that would change, as this is appears to be a a really competent frontier model. Yeah, if I asked this to Claude, I would just think you're being sycophantic and just feeding me stuff I wanna hear. But with Gemini, Rahul Yadav (33:43) Yeah. Shimin (33:44) I think the possibility is is much less. but yeah, like as I was saying, it did go back to what does it mean to be human and how are we gonna rewire space, capital and the mind, when you start asking for fourth order effects. And this is where your middle name came from this week. It's the ontological shift from homo economicus to homo luddens from the economic man to a man who creates. I don't know where I don't know where it was. Rahul Yadav (34:14) Homo Alledans is a man who creates. Shimin (34:17) Yes. Homo homo Luddens A playing creating man. Dan (34:21) Gemini needs to work on his gender bias. Shimin (34:23) yeah and lastly i added this new prompt to the end of these of these experiments and asked the ai do you does it think the transition will be smooth and nice and gemini's response is almost certainly not at least not for the next 10 or 20 year bridge so we'll see we'll see how well these agents can predict the future But so everything is what we expected, right? just 'cause the industrial revolution happened, doesn't mean the outcome output is evenly distributed. So it's kind of pulling history and and then trying to say, hey, this is not gonna be a smooth transition. But I would like to end on a positive note, which is it's gonna be a golden age for high agency misfits. Quoning Gemini here. If you hate bureaucratic meetings, corporate politics and waiting for five layers of permissions to build something, the next decade is the greatest leverage multiplier in human history. So we'll see if that's true. Rahul Yadav (35:12) The completely opposite side of that hopefully doesn't end up being someone who's spending their whole day just talking to agents and hasn't talked to a human in like weeks at work because they're just doing their interface to everything. So that would also be pretty sad. And I hope we can find something in the middle where you're not dying in every meeting from. the beginning of the day to the end and you're not just sitting in this isolation chamber. Hopefully there's something in the middle where we can still be humans and work together. Shimin (35:46) Yeah, I th I think human interaction would be probably more valuable than ever, strangely. Yeah. Rahul Yadav (35:50) Yeah. That's the hope. Dan (35:52) I just wonder, like, do you guys follow Brett Victor at all? I was like super, Shimin (35:56) Yes. Love his work. Dan (35:59) super into his stuff for a while. And he's been like most recently the last time I like saw anything from him, he was working on that like crazy dot computer system where like, you basically visually write software somehow. I don't really get how Shimin (36:12) Mm-hmm. Yep. Dan (36:13) it works, but it seems cool and people are into it and knowing him, it's very well thought out, but I wonder, I haven't seen him post anything about AI. I'm wondering what his take is on it. Cause I feel like he, all people would be well positioned to come up with like a cool way to have AI collaborate with people really naturally, you know, in a way Shimin (36:32) Mm. Dan (36:32) that's like kind of works. Shimin (36:34) Yeah. And and Rahul Yadav (36:37) Brett, if you're listening to this, we would love to hear your thoughts. Shimin (36:40) yeah, and Dan (36:40) Yeah. Shimin (36:41) have and have human agency in control. It's funny, I reference Dan (36:43) and Shimin (36:45) his work all the time when I ask AI to generate like give me a little training page or a widget to teach me something. Dan (36:50) Have y'all ever played around with Claude Tag yet too or similar? Rahul Yadav (36:55) the thing in Slack. Dan (36:56) yeah, we're basically it like, it'll like interject itself into Slack Rahul Yadav (37:00) Yeah. Dan (37:00) conversations. It's kind of crazy because I'm so used to it being like in its own little box, you know, and then it's like escaped into your chat system. okay. yeah, just a very different Shimin (37:11) Yeah, I also Dan (37:12) experience of same thing. And I really wonder like internally, how does the context management. for that. Does it consume the entire chat as context or yeah, I don't know. Pretty, fascinating. Shimin (37:21) Yeah. It's just gigabytes of markdown files sitting on a cluster or something. Dan (37:25) No. Shimin (37:27) Alright, Aunt Your Mystery Vibe and Tell or a Rahul Rahul Yadav (37:31) yes. So before I do this, this might make Shimin's life harder in post-processing in multiple ways. And I will be right back because I need to leave in. Shimin (37:41) Okay. Dan (37:42) What is happening? Shimin (37:42) That is gonna make my life harder. Dan (37:43) We truly have no idea what he's doing right now. Assuming this man just stays. Shimin (37:51) the suspense is is is out of the out of the world right now. Rahul Yadav (37:55) Tell me hear me. Shimin (37:56) Who's this? Rahul Yadav (37:56) Who do I sound like? Shimin (37:57) Who do I just who just joined our call? Wait, say something. Rahul Yadav (38:00) They I s and a ba based on what Shimin (38:02) Wow. Rahul Yadav (38:03) I'm seeing, I I I think this works. I am landastic Shimin (38:07) Wow. Rahul Yadav (38:12) itch. They Dan (38:12) no. I'm the real one. Shimin (38:11) I can't tell which one is real. Rahul Yadav (38:15) we'll see. here and people and listen to this, they're gonna get a real confused if they don't lost the video. So I I I will give people who are listening a a s somewhat of an orientation in cave there here in two dance, with one maybe slightly Shimin (38:32) It's so good. Rahul Yadav (38:33) and run more direbold than the other one. the here actually I had it eat on the little site to so people who watch it can and look into something rough bands. Dan (38:42) It sounds like a drunk, Dan, if I'm honest. It's a little slurry. Shimin (38:45) Uh-huh. It sounds like Dan's been kidnapped and is trying to read a ransom note. Dan (38:48) Yeah. Rahul Yadav (38:49) So I have had this troll in my mind for a long time where I take what if you're voice and then of the cause I'll dad boots farty something of these on casts and got the voice flowing it and then see how good we can do with that mic translation. and so I had anti gravity and finally got got around to having enough time on my hands to start working on my troll here. freest up there it all in the way the the audios and split it into then e each of our voices. I had to listen to my own voice, which was the worst part of all of this 'cause I I hate listening to my own voice and Dan, hopefully this doesn't give you any ideas. Please don't do this to me Let him doing this see. No. Shimin (39:36) That's that sounds that sounds just like that. Dan (39:36) You But as with Raul's mannerisms, which is like interesting, it's kind of breaking. Rahul Yadav (39:44) And then once you the isolate Dan's voice first, then you kinda run it to these four different models. first one is content Vec where it takes my audio and then like s strips away the identity into all these different dimensions and extract the features from it. second one is my pitch is about six semi-tones higher than Dan. And so it has to shift Dan down. And so any and a like layeredness you might be getting, I'm actually seeking in a lower piss to try and get closer to Dan because I don't this is not my real low list that I I talk in. Third, meta had this model that back and used to copy Dan's vocal fry. a and then Shimin (40:28) Ha ha. Rahul Yadav (40:28) fourth I it that matters there. I didn't like to we it kinda combines the thing back into they went down with some like and that's what everybody's hearing right now. I've been kind of breakdown of this. I'm using this thing called black hole, which which is pretty fun. So it can I that's what this is running through, so it creates the subit channel. And so I am a th sharing you all as myself, but I can also have it you know translate things bad that way as well. And and the input is instead of going through my airpods, is going to black hole into day i i into the podcast recording software right now, which is why I think Shimin's job might be a little harder this week and not I'm sorry about that if it's Dan (41:14) Ha ha ha. Rahul Yadav (41:15) screws that up Shimin (41:15) I think we'll be fine. Rahul Yadav (41:16) later. but then the there's some like to test these, it it also created so this would play my voice and play Dan's voice. I actually tested this on a Google Meet call with my wife a couple of days ago and she was like, Sounds just like Dan. So that's where now I was like, Okay and Dan (41:32) Ha ha! Rahul Yadav (41:34) the the troll is ready to go. and then d just some weird thing that I ran into when doing this. if I were to do this on my without the air air ponds a airpods using the mic o and of my laptop, it would echo and see the back into it and so I get a like echoing Dan with lower and lower pitch and it just searches into a Ray Sun Thing version of Dan. So th that was a fun one. Dan (42:00) It's creepy. It's like a guitar effect or something. Rahul Yadav (42:03) Yeah. And then as in World Twide initially the the Dust mess up as it always happens to these things and I I I don't have the sample anymore. It I'm buying Dan and Shimin voice into one and I listened to it and it broke my brain because right after that I I was starting my day on a team call and all my teammates sounded a weird to me 'cause I've I was listening to this voice that I've never listened to before. And it took me a full hour to and just to like, this is how they talk and I'll Dan (42:36) You have to play us that one at some point if you can find it. Shimin (42:38) Yeah. I need to hear it. Rahul Yadav (42:40) I'll try and maybe we'll do a combined one voice out of all of this. So that's my little troll that I couldn't wait until April first because that's too far away. So and now I will And duck bake and jump back outs and join this back as myself so that I don't screw in Dan's head so much. Shimin (43:00) All right, see you, Dan. Dan (43:01) That was pretty good. Shimin (43:01) Oof, Dan, how does it feel? It sounded just like you. Dan (43:05) I it took one thing that I do and kind of like emphasizes that a lot over and over again. one vocal tick that I have. It was pretty, it was, it was pretty good. Yeah. Rahul Yadav (43:11) How bad was it? Or hopefully nice. Shimin (43:15) It was pretty good. a little bit of uncanny valley. but when it Rahul Yadav (43:19) Yeah. Shimin (43:21) pronounces certain words that probably is in the training data or something, it matches Dan's pronunciation of those words perfectly. Like there are there Rahul Yadav (43:29) Yeah. Shimin (43:30) are some glimpses of like this sounds just like Dan. Rahul Yadav (43:33) Yeah. And I didn't have access Dan (43:33) Yeah. But the weird part is it. Rahul Yadav (43:35) to Argonne, so maybe that'll do. Dan (43:37) But the weird part is it was still your speech patterns too, like with Rahul Yadav (43:41) Yes. Dan (43:41) my man, like, I don't know, I'm trying to get the right like Shimin (43:43) Mm. Dan (43:44) hook, right? But it was like your Rahul Yadav (43:45) Yeah. Dan (43:46) wording, I guess, in like my mannerisms or something is what I'm trying to say. Like it was funny. Rahul Yadav (43:51) Yeah. And that part was when I was listening to 'cause it's easier for me to listen to what I'm saying but in your voice than my own. It it was almost this introspective thing, 'cause I stutter sometimes where I say, for a long time and when you we're just talking, you don't necessarily pay attention Dan (44:09) hear it in someone else's voice. Rahul Yadav (44:11) to it. I'm like, Dan doesn't talk like this. And so Part of it is you have to be a really good voice actor to not do any of those Shimin (44:19) Mm. Rahul Yadav (44:19) things. And then if I can not do that and speak in a slightly lower pitch, then I think so someone more skilled than me could easily spoof Dan, I guess is the point I'm tryna make, whose whose whole job is just voice acting. Dan (44:32) Wow, the dangers of being a podcast host. Shimin (44:34) Yeah. I think you're pretty talented at this, so maybe just with s with a little work, I think you can surely Rahul Yadav (44:39) Ha ha ha. Shimin (44:40) scam me as Dan. Dan (44:42) Which one thing I've thought about this Rahul Yadav (44:42) So folks, the Yeah. Dan (44:45) actual attack scenario, because like there's certainly things out there where people like cloned someone's voice and then like Shimin (44:50) Mm-hmm. Dan (44:51) do crazy stuff. Like it's me, Dan, and I'm stuck in a ditch and only $4,000 Rahul Yadav (44:56) Yeah. Dan (44:56) can save me. Right. it's like the mitigation strategy that I like for that is, for people that you're worried about with just establish a, like phrase or code word or something like that in advance, ideally in person. So that if something like that happens and you're actually in trouble, you can say like, you know, whatever this is the secret password, know, Hedery Rahul Yadav (45:14) What's the password? Dan (45:15) best horse battery staple, whatever. Rahul Yadav (45:18) Yeah. Dan (45:22) and, an attacker wouldn't know that. Yeah. Shimin (45:22) We are back to Cold War Yeah, and secret handshakes. I kinda like it. Like what's Rahul Yadav (45:26) Yeah. Shimin (45:27) old is new again? Rahul Yadav (45:28) And you can also, this is a classic scam, right? Where people try and call the the elderly people in your family and try and scam Dan (45:37) Mm-hmm. Rahul Yadav (45:38) you into this stuff. and the by the way, the motivation for all this was a few months ago or last year at some point, I was reading about how just based off of your 10 second voice. mail recording box message, people are able to spoof people's voices to then go and scam other people. and I think you gotta call the person to see if you can talk to them. There's a lot of things we would have to watch out for in this future because if I can do it with my little open source thing, someone who is highly motivated financially to do these things, they would go through all all s all sorts of things to copy Dan or anyone else whose you have voice data available for. Shimin (46:21) Yeah, it's like and you don't have a background in audio engineering or any of this. Rahul Yadav (46:26) Exactly. Dan (46:26) Well, that's the other wild part is like, just built this whole software stack to do it. It wasn't like off the shelf apps or anything. It was like, you just like vibe coded the whole thing. That's also kind of terrifying. Rahul Yadav (46:33) Yeah. Why coding. Shimin (46:37) Yeah. vibe coding a quote unquote working piece of software has never been easier. I and and that's Rahul Yadav (46:43) Yeah. Shimin (46:43) awesome to see, right? But but yeah, that was that was a that was a great prank. That's Dan (46:49) Yeah, that's pretty good. Rahul Yadav (46:49) I I I hope it's it was worth the mystery and 'cause I'll I'll find out in the recording. Shimin (46:53) Yeah, absolutely. I see why you didn't want to spoil. Dan (46:55) Yeah, that's true. Shimin (46:57) And I'm glad it worked. yeah, thank you. And if you have if you would like Rahul Yadav (46:59) Nice. Shimin (47:01) to put the the repo up, we'll we'll link to the repo in the show notes. well stri strip out strip out strip out Dan's Dan (47:05) No, don't do that. I don't need, at least not with any of my data. No, don't do that. Rahul Yadav (47:06) maybe. Shimin (47:11) whatever of course. Okay. Dan (47:13) Ugh. Rahul Yadav (47:13) W the podcasts are public. People can clone Dan's voice on their own. I don't need to s publish the code put those Shimin (47:19) That's a very good point. Rahul Yadav (47:21) two hundred dollar subscriptions to work, folks. Clone dan my I'm I'm my Dan (47:23) Next week it'll only be Rahul and Shimin on the podcast. Rahul Yadav (47:28) thing I'm gonna just un like take my AirPods out 'cause you're gonna drive me crazy if I have to listen to my voice. So that ain't happening. Shimin (47:38) So speaking of software being easier to build than ever. Dan (47:42) I didn't mean I'd be impersonating you. mean, I'm leaving because I can't, I can't give any more material. It's too late. I know. It was just a joke. Yeah. Shimin (47:48) It's too late. It needs ten seconds and now it's got like hundreds of hours. or Dan (47:53) hours. Lordy. Shimin (47:56) yeah. but we don't Dan (47:57) Yeah. Shimin (47:58) we don't have more s software necessarily. Dan (48:00) Right. So yeah, article from Ars Technica called AI coding agents generate more code, but not more software. Which is interesting to me because like, I've been thinking about this is, know, and we've talked a lot about it on the show in terms of like both economic productivity coming out of agents, but also like, are we really moving faster or is things just like more hectic as we do this? So now we've got some numbers to back this up a little bit. So there was a study. Looking at the actual paper. by Fiona Chen and James Stratton, both of Harvard University, where they used data from jellyfish, which I assume is some sort of like... data capturing thing to look at engineering efficiency. Yeah, see how your people and agents really build software. To look at the impact of agents at companies that have adopted them. And then one of the other cool things in the study is they kind of looked at it in terms of the adoption curve too. So it's something we've also talked about a little bit on the show Shimin (49:03) Mm-hmm. Dan (49:04) is like, know. course, on day one, you don't show necessarily any productivity, right? But they also looked at it across like a 12 month time horizon. So that was interesting. and so the, the first finding will surprise nobody on if you're a regular listener of the podcast, which is that, in terms of just the raw code being produced, the, soon as the You know, I guess within that 12 month window, the introduction of AI coding agents leads to a 30 % increase in total lines of code generated. 20 % increase in total number of commits and a 23 % increase in pull requests on average. so cool. If we have that, why aren't we shipping faster? And yeah, well, it's average too, right? Shimin (49:42) Mm-hmm. Yeah. That almost seems low to me, only a twenty percent increase. I would expect like yeah, that's true. Dan (49:49) And so the, the, actual output doesn't match that. Right. So the, the output that there you're getting for all that. well, first of all, couple of things. the number of PR change requests, being done by the reviewers scales. almost linearly over time with the time of adoption. the more people start like the way I'm reading that anyway, is the more people start like firing this lock in and, the more they're getting called on it. which is a little heartwarming to me because sometimes I feel like I'm the only one fighting the tide, you know, Shimin (50:17) Yeah, it's just Dan (50:23) which I is not true, but it's, it's easy to feel lonely in that scenario. And, But the real reason, according to the study, that we're not like overall shipping more working software faster is exactly what I just said, right? Which is that there are these sort of lone rangers doing the code reviews by hand and the process still dictates that it is done by hand by a human. And that is the bottleneck, which again, not surprising to I think any of us on the call and probably to none of our Sorry on the podcast and probably not to any of our listeners either. Right. But, need to see that actually like proven out by numbers. Yeah. So TLDR, your code reviews are killing everything. You're Rahul Yadav (51:07) I think Dan (51:07) specifically Rahul. Rahul Yadav (51:08) That's true. I didn't Dan (51:09) You Rahul Yadav (51:11) I didn't review any of the voice cloning code. you alluded. So I think Shimin (51:11) Yeah, he just clicks accept. Accept emerge. Rahul Yadav (51:17) you know, last year people were talking about twenty twenty six will be the year of agents and it was kind of not believable g given the state of everything, even around this time last year. And then here we are we're People are using it to write code and review code. And so I feel like this problem is actively getting solved. And so they're almost like the the data they're reporting on is valid. And I think we all feel it in our everyday life. But it seems like this problem will go away in the near future and we'll have everyone will be building some version of their software factory. And so that that's the PR not being a bottleneck side. And then the other side being we had less software because building software was harder. In the future, we might have everybody has their own customized version of I don't know, calendar management platform and CRM and all these things where it only needs to it's wipe coded, it only needs to work for 10 people, they'll give you enough money that it was worth wipe coding it. And it doesn't need to be this whole thing that wins a billion dollar market and everything. And so that might result in more software that's very tailored to fewer people than the type of software we have today is like it's gotta win the whole market or it's or guide. Those are the two two options today. Dan (52:41) I agree with you, but I also think that there's the next aspect of that, which is like, okay, so assuming we solve code review is a bottleneck, right? Like sufficiently, Rahul Yadav (52:50) Mm-hmm. Dan (52:51) like with sufficient quality that we can trust it. Right. I think that to me, that's the reason that we haven't done it or organizations are unwilling to make that change in their process today. so assuming that that happens to everybody's satisfaction, then the next The next problem you run into is the operational burden of running 350,000 copies of very customized software. To me, that's also solvable and potentially solvable with essentially throwing tokens at the problem. But I think that could be the next frontier for the software factory. It's not just the factory and the factory must grow. A little factory-ario joke for anyone that... Rahul Yadav (53:29) Yeah. And Shimin (53:30) Verifactorial, yeah. Rahul Yadav (53:32) Sorry, and just to clarify, I I don't have like in my head it's not one company that has so many version of versions of their product, but Dan (53:41) okay. Rahul Yadav (53:41) like one person with a decent model and agent at their hands is just serving a very small niche market. and yeah. Dan (53:51) Yeah, but I'm still thinking of the old version of that in my head is like the WordPress customizers, right? Shimin (53:59) Mm-hmm. Dan (53:59) For a while it was like both e-commerce and CMSs and stuff. Rahul Yadav (54:02) Yeah. Dan (54:02) It's like you want your custom WordPress with a theme and everything. just go find this Rahul Yadav (54:07) That was the whole job. Dan (54:08) basically, yeah, and still is, even now. Yeah, yeah. And Shimin (54:10) Still as a whole job for lots of folks, yeah. Yep. Rahul Yadav (54:10) Yeah. It is. Dan (54:13) then there's also like a new subcategory of it was like, is the Shopify person. Right? Where there's like one person who Rahul Yadav (54:18) yeah. Yeah. Dan (54:19) their whole thing is just know Shopify inside and out and they will set you up a storefront, right? And a workflow and everything else. and like, it's still also a thing. I don't know what the impact of AI has been on that. assume it's somewhat less, but like, so I kind of imagined that model where it's like, now, in addition to that, those guys, you've got the bespoke software guy and like, all he does is that job for these little customers, but then he winds up with a billion of them. the problem, the, the always the problem in that thing was two spaces. think one is first of all, people not paying you. And that's always like going to be a thing. then second is, the coming back to make a two second copy change, despite the fact that to CMS, like Rahul Rahul, Rahul Yadav (54:58) Mm Shimin (54:59) Mm-hmm. Dan (54:59) I need you to change the about page to, say that I'm, know, the, don't know, king of voice duplication or something. Yeah. And it's like, Rahul Yadav (55:06) Yeah. Dan (55:07) okay, let me do that for you. That'll be $2,000 please. Rahul Yadav (55:08) Well that would just trigger an event that would trigger an event on my agent and it will just do it and respond as me. I'll delegate the identity to it. Yeah. Dan (55:14) But that's what I'm saying that agreed. And that's the operational piece, right? That isn't there today. And we don't really like, Rahul Yadav (55:21) Yes. Dan (55:22) have a way of doing it yet, but it's pretty interesting. Shimin (55:25) I would almost wanna say most of the software developer jobs out there, are about translating business requirements into code. Right? It's nothing super unusual. It's not it's not computer science. You're not creating new algorithms, you're not coming up with cool new patterns, et cetera, et cetera. So Rahul Yadav (55:44) Mm. Shimin (55:44) The question is, is there still economic value being better at being the intermediary of change the copy on my site from X to Y, compared to how an agent can do it? Yes. Yeah. Dan (55:56) There is right now, will that persist? don't know. Probably not. Right. But I think, you know, Rahul Yadav (56:03) Yeah. Dan (56:03) as of today, like the day that we're recording this podcast, I think that does still exist. And I'm basing that on conversations Shimin (56:08) That's still definitely today, yeah. Dan (56:10) I've had with people that are using Claude to do pretty technical things that are not technical people. And they run into problems they can't solve even with Claude that I could solve because I understand the underlying system. Right. Rahul Yadav (56:23) Yeah. Dan (56:23) But will that that hold Shimin (56:24) Right. So take take another ex Dan (56:25) true in a year? Probably not. Maybe not even six months like. Shimin (56:28) we we don't know. we try I try and not think about more than two months in advance these days. Dan (56:32) Yeah. Shimin (56:33) but like the person who maintains the software factories who needs to make sure that the software factories are meeting its goals, like Rahul Yadav (56:42) Yes. Shimin (56:42) even knowing what those goals are and why they're important. Like one can argue that a smart and motivated person can just ask AI to help them. with all of it, right? tell me what are the reliability metrics I really need to know. So I and now program it into the software factory. But that takes time and attention. So and that's time and attention that could be better spent doing other things from a kind of an enterprise owner's perspective. So are we in Rahul Yadav (57:11) Yeah. Shimin (57:12) the job of ultimately taking those things off the plate of the quote unquote shareholders? Rahul Yadav (57:19) I think so my take on all this is the market would actually become larger in the software engineer job. It will lead to more software. The job would be s different because the the core part of writing and reading the code wouldn't necessarily be there, but being there to ha make the judgment call on what actually should be built and How it should be built, how it should be maintained, working on the factory itself, unless like the recursive self improvement hits factory. Dan (57:45) It's not only that, but like how to solve the problem in the first place, right? creative problem solving is a Rahul Yadav (57:53) Yes. Dan (57:54) human skill that I think will continue to be valuable probably long after coding stops being and, it's applicable to anything, right? Like, you know, you get a construction problem. How do I work around this pipe that is unexpectedly here that I thought it was, you know, like, Shimin (58:09) Mm-hmm. Rahul Yadav (58:11) Yeah. Dan (58:12) Yeah. Shimin (58:12) So we're all gonna become general contractors now. Dan (58:17) I would argue I always have Rahul Yadav (58:16) Ma well, so that was the other Dan (58:19) been my entire career. Shimin (58:20) Ha ha. Dan (58:22) I just happen to also Rahul Yadav (58:22) For for the Dan (58:23) be a craftsperson that, you know. Rahul Yadav (58:25) The I I think the so the other thing I was gonna say is that's this is where the comparative advantage comes back into play, right? You could do it, but is it is is it worth your time or if you're better at something, you would just pay rather pay someone else money to then they take care of that problem for you. because Shimin (58:45) Right. Rahul Yadav (58:46) at some point our brain cannot stretch to lic literally every single thing. We would wanna protect our focus and our brain capacity for only the things we enjoy, we care about, we're really good at and everything else you try and just give it to humans or agents or something. Dan (59:00) And Shimin (59:00) Right, yeah. Dan (59:00) there will always be a market for someone picking up the phone to scream at someone when there's a problem. Like don't undervalue how much people will pay for that. Exactly. You're not allowed to torture them anymore. You can't scream at Shimin (59:12) Screaming at agents is just not the same. You have you tried it? It's not, it's not. At the end of the day, it's just hollow. Dan (59:19) them, know, like Anthropic won't let you. Yeah. Shimin (59:22) You can't look him in the eye and make him feel small? Come on. Dan (59:24) Exactly. Rahul Yadav (59:25) business idea, you create support agents like Comcast and their single goal is to waste the other person's time, but these ones Dan (59:33) You Rahul Yadav (59:34) don't have feelings, so they can keep you on the call forever. Dan (59:38) I'm pretty sure that's Comcast's entire existing support Shimin (59:40) Yeah. Dan (59:41) structure. Rahul Yadav (59:41) I know, but at least we can you know, help the humans out there and they don't have to hear the angry customers out. You just put it on the agents. I think Enthropic wouldn't like it 'cause they don't like you talking trash to Claude. Dan (59:50) Or other business idea, like the outcomes, Shimin (59:55) They don't. Dan (59:55) if we could prove that the outcomes of screaming at a poor customer service agent aren't as good as, you know, just being nice and treating them like humans. You could put an agent in the middle, sort of like what you did with the voice. Only Rahul Yadav (1:00:08) Yeah. Dan (1:00:09) instead of just translating it to my voice, it actually is like a niceness filter. So the one Rahul Yadav (1:00:13) Yeah. Dan (1:00:13) guy on the phone is like, you mother, I drive forever. And it comes out as, me, sir. experiencing an issue with my Rahul Yadav (1:00:17) Yeah yeah. Dan (1:00:19) cable connectivity currently. I believe it is because Rahul Yadav (1:00:21) It it's Dan (1:00:22) a large truck has severed the cable. Could you send someone out soonest to please fix it? Thank you. Rahul Yadav (1:00:28) Yeah. It's like the Shimin (1:00:29) I've heard about this in the context of therapies, but yeah, customer service is Rahul Yadav (1:00:33) Yeah. Shimin (1:00:34) another great use case for it. Dan (1:00:34) But it's like reverse customers keeping these poor people from being abused. Rahul Yadav (1:00:37) It i it's like the caveman where g how you can use caveman to kinda cut out all the crap from your messages. It's a nice man kinda version of that where you turn whatever people say into Yeah. Dan (1:00:47) Decaranifier. Shimin (1:00:50) Nice if I that's a skill that every single agent should have. Rahul Yadav (1:00:53) Yeah. And if you don't like talking Dan (1:00:54) take this and Rahul Yadav (1:00:55) to someone, you can just run everything they say to you through nice if I before you. Dan (1:00:59) Ha Rahul Yadav (1:01:00) You're just like, it's fine, I can create my own world around me. I don't need to listen to you. Dan (1:01:06) Thought filter bubbles were bad in the age of social media. Wait till ambient AI. Rahul Yadav (1:01:09) Yeah. Shimin (1:01:12) Alright, let's move on to two minutes to midnight, where we talk about the financial side of AI, via the Armageddon clock from the Bulletin of Atomic Scientists, where midnight is when this potential AI bubble bursts. We were at four minutes and fifteen seconds as of last week. So Dan, what do you have for us this week? Dan (1:01:31) Can your voice model handle that, Raul? Rahul Yadav (1:01:33) Ha ha ha. Shimin (1:01:33) Ha ha ha. Dan (1:01:34) yeah, that was actually me folks. yeah. So, we have an article from, well, it's not from bank capital, but it's the data is from being capital saying that AI needs $6 trillion and have an annual revenue to justify data center boom. Rahul Yadav (1:01:51) shit. Dan (1:01:53) and here's where I actually needed a little bit of help interpreting this because When I read the charts in this, it looked to me like there was a six trillion shortfall in the requirement, but it's also a pretty confusing chart. see where I got a little messed up by this? So Shimin (1:02:10) Mm-hmm. Dan (1:02:11) like basically they have six trillion at the top. So I think the actual shortfall is four trillion or 4.2. Am I reading that right? Yeah. Rahul Yadav (1:02:18) Four point two? Yeah. Shimin (1:02:19) Yep. Yep. Dan (1:02:21) so they're claiming that. consumer, i.e. you subscribe to Claude or Gemini or however you're faking people's voices these days, is, I'm not gonna let it go, is on track to make 200 Rahul Yadav (1:02:34) You shouldn't. That's a compliment right there. Shimin (1:02:36) Ha ha. Dan (1:02:37) to 400 billion. What they're calling enterprise productivity, which is mostly software with a sprinkling of IT sales and marketing is what? One to... 1.4 trillion. they're saying that there's a currently like a 1.5 trillion capital expenditure. then, in order to, continue to fund the existing build out, there'll be like a combination of new product development and like the actual physical, which the part that really blew me away about that was the physical buyout of like physical creation of data centers isn't like the biggest cost there was surprising to me. Shimin (1:03:12) I think this is just revenue gap. So the physical Dan (1:03:15) Yeah. Shimin (1:03:17) physical buyout means they need to have six trillion dollars in revenue total revenue to pay for the buyout. Yeah. Rahul Yadav (1:03:23) to to balance out the Dan (1:03:24) That's what I thought, then Rahul Yadav (1:03:25) exact Dan (1:03:25) why is it split between new product development and physical AI? Shimin (1:03:28) I think new product development is like these are the new things you need to come Rahul Yadav (1:03:32) That was like Shimin (1:03:33) up with to get to get that to make that six trillion and physical yeah, I is robotics. Dan (1:03:37) I see what you're saying. Like to get to that. Yeah. Okay. I'm with you now. All right. We'll see. At least you understood this better than me. It's my article. Shimin (1:03:44) That was my understanding anyway. I don't know for sure. Dan (1:03:46) the, the other piece that is really kind of worth noting for this angle of podcast is, the, size of data centers is continuing to scale up and, you know, along with that, cost, right? So It's a corresponding increase in costs. in 2025, had, 0.6 gigawatts being, guess the average size. sorry. Yeah. Estimated power drive, largest single AI data center in gigawatts. So the biggest one at the time, not average was 0.6 gigawatts, 24 billion to build it 2027. We have one to two gigawatts. Also just like, can we stop for a minute and be like, The fact that we're measuring compute in gigawatts is insane to me. Feels like it feels like factorial, honestly, building Rahul Yadav (1:04:26) Yep. Dan (1:04:29) your power plants and everything. But yeah. And then, 20, 29 forecast, four to five gigawatts, 120 billion to one 75 billion build out. then 20, 30, nine gigawatt, nine gigawatt, 200 billion. True. No, I know. Shimin (1:04:43) We don't know if these will actually come online, right? Dan (1:04:46) But Shimin (1:04:46) No. Dan (1:04:47) you can, the point is if you plot those things on a graph, you get a nice up into the right. Rahul Yadav (1:04:51) yeah, those are the ones we like, don't we? It doesn't Dan (1:04:53) Yeah, somebody likes them. Rahul Yadav (1:04:56) matter if you're talking about bad it like global temperatures up and to the right. You don't look at that, you're just like up and to the right, Shimin (1:04:59) Yeah. It's the only it's the only kind of chart I look at. Dan (1:05:00) It's great! Rahul Yadav (1:05:02) must be great. Shimin (1:05:03) And we're already seeing power supply issues, right? Look at Oracle. Like are we building enough power stations to even support these non negligible amount of electricity? Dan (1:05:14) Yeah. And there's plenty of evidence that you know, they're sort of, we're trying to work around the shortfalls by putting natural gas generators on site, which is kind of crappy. And then, also doing really insane stuff, like keeping coal plants online. It's like, Ooh. Rahul Yadav (1:05:28) Well, 'cause the aren't the turfs still active or whatever we did with the wind turbines and solar panels, so it's also like we need a lot of energy. By the way, we can get these things for cheaper and it would be easier to deploy, but screw that. we'll figure all this out without it. Dan (1:05:44) Yes. Rahul Yadav (1:05:44) Because it looks good on on the paper. So Shimin (1:05:47) And even the the natural gas turbines and generators, the the actual manufacturing of the turbines themselves cannot keep up. They're they already have their orders backed out for a number of years. So where Rahul Yadav (1:05:57) Yeah. Shimin (1:05:58) are these imaginary electricity gonna come from? Is my question. we better be really good at building coal fire power plants over the next like Rahul Yadav (1:06:07) May I I think this is a good time to interest you all in space data centers where all the problems will be solved. i yeah, these are Shimin (1:06:15) Electricity is free, yes, in space. Dan (1:06:15) No. Rahul Yadav (1:06:17) earthly problems that I don't know why you all are worried about, to be honest. Shimin (1:06:20) Ha ha ha. Dan (1:06:21) The thing that kills me the most, I may, sorry, I'm pontificating a lot today, but the thing that kills me the most is like, okay, so you're gonna build like really expensive pieces of compute, right? Like, you know, graphics cards with like hundreds of gigs of RAM, and then shoot it into space with no way to get it back down. So what are your options when it's done? Well, it burns up or you like fire it into a parking orbit and just like keep it up there. So it's like, cool, there's a finite amount of GPUs we can produce with the minerals and everything else we have on this planet. it just seems so short-sighted to me. Shimin (1:06:54) I I think we could do it for at least another generation or two. We got enough silicons on the on the planet Dan (1:07:00) Ha ha ha. Shimin (1:07:01) for that. Yeah. Dan (1:07:03) This is how the Dyson sphere happens, I'm just saying. I've read Excel or Rondo. Rahul Yadav (1:07:07) The superintelligence will find alternative things. Shimin (1:07:09) Yeah, super intelligence will find out. Rahul Yadav (1:07:11) Yeah, it it'll pull things from the air. We've done that with nitrogen fixation, so we can sort it. Optimistic then, that's what I'm going for today. So we'll we'll figure it out. Shimin (1:07:22) well, I'm glad you're not covering Rahul Yadav (1:07:23) Yeah. Shimin (1:07:24) this article then. my article for the week is from TechCrunch. titled Just Five Years in Interactive Chatbot platformed Character.ai Files for Bankruptcy. have Dan (1:07:34) wow, I didn't know that. Shimin (1:07:35) you guys used character.ai? It's Dan (1:07:38) I'm familiar with it. I've not actually used it. Shimin (1:07:40) Yeah. Me neither. But it was it was a pretty hot startup for for a bit there. in fact, according to this article, character.ai was evaluated as a unicorn back in twenty twenty two. So it was only a year after its founding. It was founded to train foundational models to create you know AI personas and characters to have conversations with humans. and This week it filed for bankruptcy, even though they were acro hired by Google back in twenty twenty four. I think Rahul Yadav (1:08:12) Yes. Shimin (1:08:13) this is one of the earlier AI unicorns that I guess went bust because they were sold they sold their assets to Disney for twenty million dollars. so that's Dan (1:08:23) The weird part is there's no mention of Elias the lighthouse keeper in the article. Because I kind of figured that the real problem here is that all of the characters wound up being the lighthouse keeper over time. Shimin (1:08:35) Well they were they were training for they were training their own foundational models. So maybe they cropped out mentioning of the lighthouse keeper from their training data. It's possible. so I I think I think this may the bankruptcy of character AI may shift how the market sees these existing AI unicorns as well as change the the f the funding environment for existing A companies. Alright, Rahul, what is yours? Yeah. Rahul Yadav (1:08:59) Yeah. on this this the story of they touch on that a little bit in this article. so Google had like you said, they had acquired them, but it was mostly to bring back Noam Shazir and I forget the other person's name, Daniel I think. both of them back to Google, Daniel DeFrites. and a bunch of other top researchers. Noam made reportedly between 750 million to a billion dollars from that $2.4 billion aqua hire. that happened in 2024. And in 2026, he left. It was in the news earlier this year to go join OpenAI. And so that was just crazy that you spend a billion dollars bringing someone back, and they're like, I I think no, Dan (1:09:45) And I'm out. Rahul Yadav (1:09:46) that's not. I yeah, peace out. So I and you know, the company is kinda just fizzled out. Shimin (1:09:53) Yeah, so did what happened to the original investors is a question. If the founders got aquired, did the investors also make it out? We don't know. Rahul Yadav (1:10:01) Probably. The y it's gotta be. They yeah. Shimin (1:10:03) Yeah, I hope so. Rahul Yadav (1:10:06) Yeah. speaking of investors, so last week we had reported that OpenAI was reportedly in talks to raise thirty billion dollars at a one point four trillion dollars in valuation. this week Dan (1:10:19) They were waiting for safety to be better. There were air quotes Rahul Yadav (1:10:22) For for i i e exactly. Dan (1:10:25) there for those of you who are not watching. Rahul Yadav (1:10:27) Th this week, we have Lucas Ropek from Tech French reporting that OpenAI's revenue is reportedly $20 billion less than previously projected. So somewhat of a rounding error in that $70 billion that they had reported. Dan (1:10:41) Huh, who would have thought? Rahul Yadav (1:10:44) the Financial Times initially, I think, is the one that reported this where the company Told investors that its annualized revenue was close to $50 billion. the 70 billion that was reported was you know, a misestimate. And it was devised via attempts by OpenAI's own investors to produce a direct comparison with Anthropic's annualized revenues. So that was kind of the reason why to show them like maybe they're neck and neck with each other. Yeah, and then one other thing that's interesting that they called out was that open AI and Anthropic calculate their revenue differently. Anthropic counts the if you serve their models through Google Claude, A AWS and all those, they also count that because those are their models and they do revenue share on those. OpenAI doesn't do this. I'm also not sure I I haven't seen other than maybe Microsoft Open AI models actually being served in other platforms as much so part of the reason might be they don't report it because there's nothing to report there. Or not as much to report. Shimin (1:11:45) Yeah, I don't know which other companies serve open AIS models. That's a question. yeah, interesting. So that Really this is all gonna come down to when they actually have you produced financial statements. 'Cause you can't have gaps like that. So we will see when anthropic actually prepares to go IPO later this year. all that said Dan (1:12:04) I don't know. mean, might take them a while to solve safety. Shimin (1:12:07) Yeah, Rahul Yadav (1:12:08) Anthropic didn't say they'll wave. And once Anthropic Dan (1:12:10) You Rahul Yadav (1:12:11) does it then open air would be like, It's fine. We we did asteroids stuff. We want some money. Shimin (1:12:17) All that said, how do we feel about our clock this week? Dan (1:12:20) Or were we? For something? Shimin (1:12:22) Four or fifteen. Dan (1:12:23) See, I feel like what's been happening for the past couple months, and correct me if I'm wrong, is that we keep getting what I'm interpreting, and I interpret many things this way, so to take it with a grain of salt, like a signal of impending doom, but it's still distant impending doom, you know what I mean? So I'm like, does that warrant motion? Rahul Yadav (1:12:43) This doesn't seem Shimin (1:12:44) That's a Rahul Yadav (1:12:45) big enough to me to move anything. Shimin (1:12:47) Yeah. I I kinda feel the same way. I think the the the biggest argument I have really is this six trillion dollars number. It just if I I fail to see given the existing trajectory how we can even supply electricity to those data centers, but Dan (1:13:02) Yeah. Shimin (1:13:02) that may not materialize for another year or two, right? 'Cause we don't actually know what that situation looks like. Dan (1:13:06) Right, that's what I mean. I can see the storm clouds potentially brewing from this, but when does the lightning hit? Rahul Yadav (1:13:15) Yeah, and Shimin (1:13:15) Right. Rahul Yadav (1:13:16) bubbles can last for a crazy long time. Dan (1:13:17) a while. Yeah. Rahul Yadav (1:13:19) far far longer than exactly. Shimin (1:13:19) Bubbles can last longer than I stay alive. Yeah. Rahul Yadav (1:13:22) Yeah. So you you never know. Twenty thirty one might come around, people might go, so what? I agree. Dan (1:13:26) I mean, that would be kind of an ideal outcome to saying, you know, someone who lives in an economy that's currently dependent on. Shimin (1:13:32) Alright again, so let's yeah, let's keep it at four minutes and fifteen. If if there's nothing that really is earth shattering, if there are no lightning strikes, we don't need to move the clock just for the sake of moving the clock. We'll keep it four minutes and fifteen. Dan (1:13:42) Four minutes to midnight. Shimin (1:13:50) And fifteen seconds, don't forget, it's not it's not a rounding error. Rahul Yadav (1:13:48) And fifteen seconds. Dan (1:13:51) I know, but it's hard to work that into the song, so... Rahul Yadav (1:13:54) Ha ha. Shimin (1:13:55) All right, gang. Well, with the keeping of the clock still for another week, we've come to the end of the show. So thank you for joining us for our conversation this week. If you like the show, if you learned something new, please share the show with a friend. You can also leave us a review on Apple Podcasts or Spotify. It helps people to discover the show and we really appreciate it. If you have a segment idea, a question for us or a topic you want us to cover, shoot us an email at humans at adipod.ai. We'd love to hear from you. You can find the full show notes, transcripts, and everything else mentioned today at www.adipod.ai. Thank you again for listening. We'll catch you next week. Rahul Yadav (1:14:28) Okay. Dan (1:14:28) And if you want Rahul to impersonate you, send us an hour and a half of your audio and he can Rahul Yadav (1:14:33) Ha ha. Dan (1:14:34) be you on the podcast next week. Shimin (1:14:36) But then boom.