Latest / Artificial Developer Intelligence / Grok Buys Cursor, MidJourney Goes Hardware, Hermes Agent & Evaluation-Driven Development

Transcript

Grok Buys Cursor, MidJourney Goes Hardware, Hermes Agent & Evaluation-Driven Development · Artificial Developer Intelligence

• Shimin (00:00) Hello and welcome back to Artificial Developer Intelligence, a weekly conversation show and study session about AI and software development. We go through hundreds of links and dozens of newsletters each week so you don't have to. My name is Shimin Zhang, and with me today are my co-hosts. Dan, he is standing in the frontier looking at the foundation of the human experience, Lasky. And Rahul, Gemini makes you command an army, Yadav. Hello guys. Dan (00:25) Yes. Shimin (00:28) How are we doing today? Dan (00:29) Hi. Is it just me or are middle names getting longer week over week? Rahul (00:30) Hello? It's AI is outputting more text. Dan (00:37) As AI writing gradually takes over the entire internet, it's like flowing into the articles we cover and therefore like impacting our middle lives. Rahul (00:42) Yeah. Shimin (00:48) AI already has the impact on our podcast. Dan (00:51) Yeah. Even even aside from the content. Shimin (00:54) Alright, on this week's pod, we will first have the news threat mail, as always. ⁓ we're gonna talk about the cursor acquisition and what Mid Journey has been up to. Also, Dan, apologize for calling it the pod. I meant the show of course. Dan (01:09) I didn't even catch it. You could have gotten away with it too. next up we're gonna have the tool shed where we'll be talking a little about Hermes Agent. Shimin (01:15) Yeah, and then we'll have post processing, we're gonna talk about whether AI is ruining our skills, Claude code as a video game and evaluation driven development and what is that? Dan (01:27) And then finally we will have our I hope it's a fan favorite. I don't know. No one ever writes me emails. They only send them to Shimin So someone write me an email please and tell me if you like Two Minutes to Midnight. But yeah, our our maybe fan favorite section where we talk about ⁓ where we're at in the AI bubble and ⁓ see where we're gonna set the clock to this week. Shimin (01:45) Alrighty then, let's get started with our first news item brought to you by Dan. Dan (01:50) Yeah, so cut what is it? I don't know. I always say a couple of weeks ago, but it feels like everything was a couple of weeks ago. It was like a while ago, right? Like almost a month ago. Elon had made a deal with Cursor and in that deal they were gonna provide or like I don't know exchange cert a billion dollars worth of services if I recall something like that. And then it also gave them the option to potentially buy the company for 60 billion at some time after their IPO. And sure enough, this past Tuesday, ⁓ which is the sixteenth of June They have announced that they will be doing that. So I guess it is gonna be a 60 billion in class A common stock deal. so cursor's only gonna be getting stock out of it. And apparently as of like last November, Cursor had reported one billion in annualized revenue, which is pretty insane when you think about like how small that company is, right? I they've been growing, but it's still like not that big. There's not that many engineers to have the a billion dollar footprint. So ⁓ pretty wild. And I for one am I excited about this? I don't know. It sounds fun to say it, so I'll just say it. I'm excited to see what Grok Cursor is gonna look like. Shimin (03:06) ⁓ so they have one billion in revenue and SpaceX is paying sixty billion for it. that's a sixty X multiplier. That seems high. Dan (03:15) Yeah. I think they're basically paying to get into the enterprise space, which is kind of what the C N B C article that we ⁓ got this from has hinted at a little bit is like Grok really has or ⁓ XAI slash spaceX really doesn't have a frankly they don't have that big of a consumer presence either, but like they don't really have any inroads in the enterprise space and ⁓ cursor is actually pretty popular in it. It was sort of like an early tool and a lot of folks have stuck with it. Shimin (03:42) Mm-hmm. Yeah, and I if I recall correctly, Cursor was for a while there, ⁓ touted as one of the fastest growing, you know, sass of all time when it comes to growing to one billion dollars. Right? Like it was like the hottest of the possibly hot unicorns. And to see them being sold relatively quickly and a little unceremoniously, I I do I do wonder if we're Dan (03:56) Uh-huh. Shimin (04:09) seeing starting to see like a consolidation phase from that initial explosion of AI tools. Dan (04:15) Yeah, it could be a early early shot in that direction for sure. but we'll have more data there in two minutes once once we get there. So yeah, I think that's really about it, unless y'all had anything else. ⁓ but figured it was worth covering 'cause it's, you know, some big news in the the AI space. Rahul (04:23) you Gawk cursor is gonna be great. What could go wrong? Grok cursor. Dan (04:38) Recursor. We need a better name. Rahul (04:41) No, it's perfect. Dan (04:43) Sure, Grock. no. Shimin (04:44) All right. Dan (04:44) Croc with a K. Yeah. Shimin (04:46) The second item I have, speaking of a crazy fast growing AI startup, the little little known player in the AI image generation space, Mid Journey came out this week that ⁓ they are going to start Dan (05:00) L little known? Shimin (05:02) That was sarcasm. Yeah, they they are one of the first I don't know, ⁓ consumer grade AI image generation companies. and what I always appreciated about Mid Journey is that as a little bit of background, they don't have a nice front end UI. Everything is done via credits that you then post your prompts into a Discord channel in order to generate an image. Dan (05:02) Weren't they like the first ba okay, gotcha. Rahul (05:10) Yeah. Dan (05:10) Mm-hmm. Shimin (05:26) And this is when this is like like the newest cutting edge image generation. like I always find that UX to be super, super crazy. But they came out this week announcing that they are going to I don't know if pivot is the right word, but they're gonna expand into creating these ultrasonic body scans. and they are planning to build fifty thousand of these. Dan (05:32) No Shimin (05:51) Body scanner spas all around the world generating billions of body scans by by twenty thirty one. this pivot was so hard, it like I found myself on the floor with like a black eye afterwards when I read this. Like, what the heck? What is going on? So this is where Dan, your middle name came from this week. Midjourney in their announcement blog said, you know, they were looking there they felt an obligation as people standing on the frontier to look at the foundation of the human experience and ask what do we want to be different? And how do we want to be different and what do we want to become? And they decided to do hardware tech. become a service company by creating spas where people can go to a sauna, go take the jacuzzi, and then also get this ultrasonic scan. ⁓ Dan (06:37) But it's gonna be cheap, right? Is what they're anticipating. Like it'll be like a hundred bucks or something. Shimin (06:41) Yeah, I think that's that's the idea there. Yeah. I mean a billion a billion scam per month is like that's like one eighth of the entire world's population, right? Like that's nuts. Dan (06:44) And it it's Yeah. That's it's pretty cool. Like the actual so they didn't even build the tech like the hardware tech behind it necessarily either. They licensed the scanner technology from another company that makes these like micro ultrasound chips. So each each chip is like capable of basically being an ultrasound. And like, you know, ultrasound is almost as useful as an MRI for some stuff. Like, you know, usually it's like much more localized. But like if I learned anything from watching what was that crazy ER show that everyone was talking about like a couple months ago? Yeah, there you go. Guy did like a they they didn't have time to get to an MRI machine or whatever. So he he like just uses an ultrasound and like spots the the thing. Yeah. So I mean, you know, maybe something to it. But Shimin (07:24) the Pitt? Love that show. Dan (07:38) I guess the way it works is pretty interesting. So you you like there's a tub of hot water and you stand in it and there's like a ring of these tiny like ultrasound chips all the way around the tub and then it slowly warm like lowers you into the warm water, which sounds a little creepy, honestly, that part. But as you're going in, the like ring is basically like scanning every, you know. slice of your body as you go in so it can like see what's up, I guess. so I don't know. It's not the worst idea in my mind. It's just kind of wild that like it's the same company. But I guess first we had Alberts and now we've got Med Journey. So yeah, what's next? Rahul (08:08) you Shimin (08:12) I think this is a I think this is a fantastic idea. Yeah. Rahul (08:16) The end. I agree, the news was surprising, you know, they call out that they are taking in terabytes of data each second. And they add this reference of like, if you converted that data into HD internet video, you'd need to watch 500 hours of footage for every one second of scan data. So it's like this insane amount of data that they're taking in per second. And it just wasn't. possible before but now it is possible and so this is one of those like yes AI and all that but given that we have so much more compute now and we will have so much more compute what are some other crazy things we can do that just weren't possible before so this was awesome to see Dan (09:03) Yeah. Shimin (09:05) Yeah, and they do have domain expert yeah, well they they do have domain expertise in analyzing AI images, right? They like I I could see how there is some and I hate to use this word synergy in this project Dan (09:05) And networking too. sorry. Rahul (09:12) Yep. Hehehehehe Dan (09:17) Yeah. Well plus networking too was gonna be my point, right? It's like in order to do like large scale deployments of stuff like you know, a big image generator, it's like you need to have like really beastly interconnects and stuff like that too, right? So or at least for some of it, I don't know. Shimin (09:21) Right, right. Rahul (09:21) Yeah. Shimin (09:31) Mm-hmm. Rahul (09:34) so when I read this my maybe this because I was traveling my mind went to TSA scanners you don't have to do the whole stand like an a make an error Dan (09:43) ⁓ Yeah, this that's millimeter wave radar for those. Yeah. Rahul (09:49) Yeah, but like you could also very depending on like how fast they can do it you could figure out a way in the future of applying the same technology for all sorts of you know security related scanning and everything too. Dan (10:01) Yes. Rahul (10:03) Yeah, there's gonna be some other crazy applications. Shimin (10:04) Yeah and The other really interesting thing is Mid Journey is completely private. they have no VC backers, they are a grassroot organization. I kinda think of them as like you know, maybe they are actually controlled by a cabal of evil people, but there's this other possibility which they are ⁓ like anonymous but doing good things and building Rahul (10:12) Yeah. Shimin (10:30) they're they claim to be building a community based research lab and that is a new model that I've never seen before, honestly speaking. Dan (10:37) Some sort of like sci fi trope come to life basically. Like the hacker collective out to do good in the world by building a weird med spa thing. Like, okay. Shimin (10:46) And then scan all of your body data. Like what could go wrong? So I guess to summarize our news items for this week, like we have two super early players in the consumer AI space, Cursor and Mid Journey. And one decided to sell to deeper pockets, and another one decided to pivot so hard that I'm still I'm considering calling a personal injury lawyer from the whip lash I suffered. what does this? So like Dan (11:10) I'm still spinning. Rahul (11:14) So this was Dan (11:14) It would have been funnier Rahul (11:14) a Dan (11:14) if my chair went all the way. Rahul (11:15) full pivot? That wasn't clear from the website. What they're saying is we're done with the whole image generation because it's been commoditized. Shimin (11:24) I d I don't think they're they're pivoting to completely the spa model, but they're clearly investing a a ton of their capital into this project. ⁓ and listeners, if you want to go visit, you know, I think they're planning for their very first spa to open in San Francisco in twenty twenty seven. I may book a trip. That sounds kinda like kind of a cool thing to do. so Rahul (11:32) Yeah. Shimin (11:45) ⁓ yeah, so I'm I'm wondering if we are gonna see a shakeout coming up. Like kind of reminds me of a old Winston Churchill quote like this is not the end. This is not even the beginning of the end, but perhaps this is the end of the beginning. Like there will maybe be some consolidation coming up. Alright. On to our tool shed, where Dan, this week, you've been reading the docsite of Hermes. Dan (12:09) Yeah, so I actually didn't even know about this thing. So maybe other people did. I don't know how long it's been out for. It seems like a bit, but there's there was kind of like a big chunk of news that came out around it in the past like five to six days, it seems like. So maybe it's a new newish thing. I don't know. Anyway. ⁓ Shimin (12:26) ⁓ I first heard about Hermes maybe a month ago and I remember going to a local and this is a Seattle area ⁓ AI hackathon where the organizer told me that yeah, it was all the rage in Seattle or in San Francisco a month ago. Dan (12:40) Yeah. Yeah. Yeah. Well, I guess that means it's dead now, so why are we even covering it? But yeah. Yeah. So the thing that immediately caught so I how did I find it? I was I guess it got it got posted to Hacker News just kind of with no context. And I was like, What is this thing? And like I know of the underlying company like nous research because they've done some cool stuff with like their own models and you know just like a couple other you know what else did they do? There's something else they did that was interesting to me. I don't know. The brain is all over the place today. But anyway, ⁓ so I was like cool, I'll give it a read and just see what's up. And the first thing that immediately stood out to me was that it at least proclaims to have a built-in learning loop. So not only does it have like you know, sort of agent memory, which like some of the agent stuff that we've talked about on here previously, like Pi doesn't actually have built in memory. Now, granted, it's pretty trivial to like make it, right? And make it exactly how you want versus you know, some sort of built in thing. But it is kind of interesting that this has it in. And then it also comes with like pretty much a smorgasbord of every possible com Shimin (13:35) Mm-hmm. Dan (13:50) Channel you'd ever want to talk to this thing on. It supports CLI, Telegram, Discord, Slack, WhatsApp, Signal, Matrix, Mattermost. I don't even know what that is. Email, SMS, Ding Talk. Don't know that one either. Faishu, WeCom, Way, Weijing, QQBot, Yambao, Blue Bubbles, Home Assistant. That's a wild one. Microsoft Teams, just in case you hate yourself, and Google Chat. which is pretty wild for you know, out of the box. Rahul (14:12) and more. Shimin (14:13) Yeah. Dan (14:15) So the notable things that kind of like I would argue like separate it from Pi is that Pi is like super stripped down minimal and that's kind of the neat thing about it. This is very much like a every single tool is included in the toolbox. It comes with this like boot menu thing that pops up when you first run it that asks you like what out of its pre-baked cool tools you wanna install and it has like huge lists of them. But it comes with memory, it comes with a bunch of skills out of the box that like, you know, they've sort of already worked on. And then the other sort of killer feature is that it has a cron capability, right? So it can schedule tasks, check in on them, and do the things that you'd want from like sort of a long-running agent. ⁓ so I I haven't played with it personally, but ⁓ I just found the idea of like, the self learning skills. So it has like not only does it have memory, but it also has this idea of like a built in learning loop. So it can create skills from the things you've asked it to do. which I thought was kind of novel and worth chatting about. So There you have it. Shimin (15:18) Yes. ⁓ you know, one of the nice things about AI is you can ask AI to take anything you've been working with the agent on and turn it into a skill. Like that is practically free. but this is the only one where I've seen that it is a first class object that does it automatically. ⁓ I don't actually know how frequently it does it or Dan (15:38) Mm-hmm. Shimin (15:41) you know, how ⁓ one doesn't know that, you know, the slash learn command should should be used, but ⁓ it is pretty cool. And I do want to also point out that they have a ⁓ memory provider feature. And this is something that I think we haven't really covered at all on this show. there is a whole class of external memory providers on the market now for agents. Dan (15:54) Mm-hmm. Shimin (16:05) Hermes supports Hongcho, Mem Zero, Hindsight, Holographic, Retain DB, Bite Over, Super Memory. and of course the default is open Viking. I've not heard of any of these. I don't personally trust an external provider with my memory, if I especially if I was to use my agent for personal sensitive tasks, but maybe I'm just a little old fashioned like that. I have asked Claude Co to take a look at my existing Pi agent usage and also go through the Hermes agent documentation and see if I should switch and like what benefits it will give me. And the biggest one is the memory. And I feel like and Claude actually recommended me against using Hermes just because I have to transfer all of my existing workflows over from Pi to Hermes. And also ⁓ I kick I already have a lot of these features built in on my local version of Pi agent. so I didn't make the jump. Dan (16:53) Right. Yeah. It's that that's the trade off. It's like you can have it either have it build it yourself or you can build it yourself, you know, with Pi. So but this is very much like I would argue kind of the opposite approach where like it just has tons of tools. Yeah, it's got memory tools, it has like web search stuff already baked in and there's like I don't know, fifteen or twenty search providers that you can use. Most of them are paid, but like if you already have a subscription to like one of them, it's pretty easy to just like plug it in. And then of course, like the thing that almost made me a little hesitant to bring this up is that then they also have their nous research like platform. And so if you run it on that then and you already subscribe to their subscription, then it gives you everything like memory, search, all this stuff all in one place, including the model runner and all that, you know. So it's like cool, I get it, but you know, I I at least I appreciate the fact that they did make it very pluggable too, in addition to their stuff. So Shimin (17:33) ⁓ nice. Dan (17:46) It's neat that it's open. They could've Rahul (17:46) And they're recommended ways to get the nous portal subscription through, like that's how you get the models, or that's what they recommend. Shimin (17:57) Wait, what is this news portal subscription? I've not heard of this. I d this did not come up in my research. Dan (18:01) It it does everything, I is my understanding. So I think it has some sort of memory product. I believe it'll actually host the agent for you and then it'll also is sort of like open router where you can connect it up to lots of different frontier providers or their own models too, I think. Shimin (18:08) ⁓ okay. Got it. So if Pi Agent is like the bare bones version of an agent harness and open claw is the open sourced kind of jankly vibe coded version, if you're a fan of open claw, I'm sorry, that's just a vibe I got. ⁓ this is more of Dan (18:31) I mean who's to say that this isn't also jankly vibe coded? This is twenty twenty six after all. but yeah. It's definitely like this is like the the plug in maximalist version, right? If if pie is like the you're just gonna go out into the wilderness with a K bar and make it all happen yourself. this one is you've gone out there parachuting in with a sixteen crates of supplies and a film crew. Rahul (18:35) Yeah. Shimin (18:36) Yeah. Rahul (18:37) is the crack head version of the major artist. you Shimin (18:55) Right. And and it's it's a SaaS right? It it already hooks up to a subscription that you can buy. So it's battery included. You just gotta pay them a little bit every month or so. Yeah. That that makes sense. Well ⁓ Dan (19:03) Yeah. Yeah. Or don't. Or or you can go through their pretty easy wizard and don't pay them anything, which is I think admirable considering so Shimin (19:11) Right. Yeah, listeners, if you've had a first hand experience with Hermes, write to us, let us know what you think of it and especially how it compares with some of the other ones. Dan (19:21) Right, me specifically. Shimin gets all the emails. I'm tired of it. Just Shimin (19:24) Yes, I will include Dan's personal ⁓ email along with his ⁓ work email at the show notes, just write to him directly at Rahul (19:32) We have his address too if you would like to send him snail mail. We're happy to Dan (19:38) The the Rahul (19:39) share it. Dan (19:39) weird part is like my email starts with Shimin at like I don't know how anyway. Shimin (19:43) Hmm. How do all right. Rahul (19:44) Hehehehehe Shimin (19:46) do we have anything else to add on Hermes? Dan (19:48) ⁓ come back in a couple of weeks and I promise I'll have tried it out on something. or if you send me an email about it, maybe I won't have to. Shimin (19:54) Alright, ⁓ let's move on to post processing Dan. first article is brought to you by Dan again. You've been busy this week, Dan. Dan (20:00) I know. I'm on I'm on a roll. All the links. so this is ⁓ article in Nature and it is called Is AI Ruining Our Skills? Early results are in. And I'm gonna add this part myself. Spoiler alert, they're not good. Didn't actually have spoiler alert in the real title, but missed opportunity. so they they were had a couple of different data points in here. I this is Shimin (20:16) Yeah. Dan (20:23) Really nothing super new if you've been listening to the podcast, but just ⁓ seeing more and more actual studies come out that are looking at this phenomenon. I think it's worth discussing. So the first study was they had a whole bunch of physicians that were like essentially experts at doing colonoscopies. So each each physician chosen for the study had done over 2,000 colonoscopies. And ⁓ they Typically, I guess are trained to spot something called an adenoma, which is like a precancerous lesion that can be pretty hard to spot, it seems like, based on the what the article said. So in the like sort of control group that wasn't using AI, ⁓ they were or before they were using AI, they found an adenoma in about twenty eight point four percent of all colonoscopies. So then they of course gave everyone an AI-based image tool that like runs while the procedure is happening and like helps them out. their rate stayed about the same with the AI tool. And unfortunately, and I don't know if this was intentionally part of the study or not, but the tool is intermittently unavailable. And so when it was unavailable, folks that had been using it, folks, physicians who had been using it. ⁓ their detection rate without AI dropped to 22.4%. So that's almost a six percent drop in skills. So that was the first one, right? So it's like your optical scanning ability as a human is not immune to this as well, right? Like I know we talk a lot about software, but like there's many other ways that it it can impact too. Shimin (21:41) Mm-hmm. Uh-huh. Yeah, I I would hate it be one of the twenty two percent or or the people who happen to have my images read when the service is down in quotations. That's the only way the IRB passed it. Dan (22:01) The the six per I see of the yeah. Yeah Yeah. Rahul (22:10) Or you used up your Claude credits for that day and was like, sorry guys. Shimin (22:13) Right. Called us unstable that morning. Dan (22:14) Yeah. I mean token maxing is over, so it applies to doctors too, I guess. Well yeah. So ⁓ in the second study, ⁓ they this one's a little closer to home. So they took fifty-two software engineers, basic coding task. half of them were allowed to use AI. They were given a quiz afterwards. The average score was 50% in the in the AI group versus 67% in the non AI group. And the thing that stood out a lot was that like the parts of the quiz that they scored worse on the AI group was on conceptual understanding of the produced code. So I think how many more data points do we need to bring up about cognitive debt? But it's here it's here and it's real. So ⁓ and there's a nice little poll quote in the bottom that I thought was worth bringing to the to everyone, which is people need to manage the competing dynamics of relying on generative AI and staying mindfully vigilant. That's by Rinta Kalia. which I think is really true. It's like you can use these tools, but you need to be aware of how you're using them and what the impacts they could be having on you. Both professionally and personally. So not saying don't use them, but just be mindful. Shimin (23:26) Yeah, and and there was this additional article where they talked about a group of accountants who have been using an automated non AI accounting system continuously for more than a decade. And then when the tool was taken away, the accountant have forgotten how to do several routine work tasks. And that part got me thinking like how much of these tasks we should be able to hand off to AI entirely? Right? Like there are certain things we've forgotten how to do, like ⁓ Dan (23:51) Mm-hmm. Shimin (23:51) writing script and that's not important anymore. ⁓ sure, yes, cursive. I've forgotten the name already. That's how long ago I've been asked to write cursive. ⁓ Dan (23:55) I mean cursive. Yeah. Okay. Yeah. Well, I it's like cursive versus like calligraphy, right? But I guess either one they've they've both kinda gone away. Shimin (24:08) Right. Yeah. So i I think it's it's right. Yeah. Like you should see my hair right. It's notoriously bad. and I guess it's especially confusing right now 'cause we can't quite tell what is still needed and what isn't. Like we don't know if the skills that's being lost is a really important one that we should never be able to off Dan (24:11) Or just handwriting it all. When was the last time you did that? Yeah. Shimin (24:31) source to AI and and which ones we need to be to in control just yet. Dan (24:36) Yeah, that's fair. Rahul (24:37) I'm reading this book about dementia and Alzheimer's and stuff and one of the things, and we know this, it's not news from the book, is if you don't use it, you lose it. And people who suffer dementia are people usually who, after retirement, don't challenge their... Dan (24:37) Well Rahul (24:59) as much or in general don't challenge their brains as much and then if you don't use the brain as much then you end up losing it because there's not you know much to pull from and this reminded me of that where over time if we give AI more and more agency Shimin (25:09) Mm-hmm. Rahul (25:20) it will impact our jobs but also the long term it might accelerate how quickly people get dementia at an old age there are things like exercising and stuff you can do but if you're not using your brain as much at the end of the day you're gonna get dementia sooner so that That's something definitely concerning. then, sorry, good. Shimin (25:46) Mm-hmm. I was just gonna say, luckily this episode is brought to you by leet code problems. Spend forty five minutes doing leet code problems every morning and you will not get dementia. Leet Code problems. Rahul (25:53) hahahaha ⁓ Hahaha ⁓ Dan (26:00) no. Rahul (26:02) Four out of five grandmas recommend leet code problems. The fifth one is busy playing video games, which is how you actually prevent it in old age. And then the other, you know, unlike the... Dan (26:06) Yeah. Rahul (26:15) actual AI and Edward Note, one of the arguments that we continuously have for when to use AI and when not is, and where do you draw the boundary is anywhere where you need a judgment, you shouldn't put AI there because it doesn't have judgment and you to pair it with human judgment not. place it but that human judgment at the underlying assumption there is that human has that judgment and it is continuously being practiced because otherwise similar to you know anything else again if you don't use it you're gonna lose it over time if AI is relying on a human but the humans are not really able to help we might you know almost like make it more obvious that it doesn't really matter very human. judgment is needed or not because it's just not there even when you need it. So if you play it out long term that's definitely a big concern too. Shimin (27:13) Yeah, well speaking of playing video games like the grandma to stay sharp, we have a article about that by Rahul. Rahul (27:19) Ahem. I do not know this person's name. They're a product engineer and an indie builder. The website is Provi.me. The ⁓ article is, Claude Code is a video game. And we've seen instances of this in the news. think even Steve Yaghi had it in one of the... Dan (27:38) Okay. Rahul (27:39) News articles recently were used like, know, I'm thinking about this late at night and about what else I can do with my agents and all that. And it's interrupting his sleep and everything. And that's the same thing the author's talking about. It's three hours past bedtime, a quick bug fix turned into a refactor and then that refactor turned into a brainstorm, then a feature branch is running parallel and a lot of time passed. Their reason for that is because using cloud code feels like playing a video game where you have this like one more turn kind of feel where you give it a prompt, you get something back and then you have to respond to it and it's continuously this back and forth but the end result being you're able to accomplish something in the makes the world change, the real world change instead of similar to playing a video game where you change the world but it's in the video game environment. And then you know that's just with one agent if you're running a bunch of agents in parallel that really makes it you know you have to like spread your attention across different things and if one agent is keeping you up so long then Imagine like running a whole fleet of agents and everything and so it gives you this feeling of continuously accomplishing something Something related to this that I realized while reading this was if you look at Sa saw B2B SaaS I don't know about the consumer SAS stuff, but B2B SaaS was not addictive until AI came along. The closest thing you got to like addictive B2B SaaS products where maybe people are just like checking their email all the time to get that hit of there's something new or Slack messages. But yeah, like no one logged into their Salesforce or HubSpot or Pick whatever to be like, I'm going to get a hit from this. No one cared for that. ⁓ But all of a sudden now Dan (29:28) Blackberry much. Shimin (29:40) Mm-hmm. Rahul (29:42) I see this real sort of an addiction to workplace SaaS products, which is all AI driven because it's making people feel like they're accomplishing more. it's still very much because these are LLM models. We've talked about that, how they're up. for engagement and so now your workplace SaaS products have engagement built into them which makes them feel like playing video games like this article is saying which makes it addictive and I don't know where that will take us in the long term but I hadn't seen addiction to SaaS products before this. Shimin (30:19) Yeah, do you guys agree that using an coding agent feels like playing a video game? Rahul (30:25) I think so. Dan (30:26) I can see where the like just one more round kind of thing would translate to like just one more prompt feature or whatever, yeah, turn whatever. So yeah, I can see that aspect of it translating. So I guess I agree with the core premise. Shimin (30:32) Yeah. ⁓ Dan (30:40) I the thing that I think is unique to the the like usage of agentic engineering to me is it's f a little bit flipped, right? So when I'm playing the video game, I kind of feel like I'm like wasting time sometimes. Like, I could be doing something real instead of just whatever. But then I'm also like sometimes I just need to turn my brain off and, you know, play some sleigh or whatever. But Shimin (30:54) Mm-hmm. Mm-hmm. Dan (31:02) The the part that's that I have with LMs is reversed and it's kinda funny is like when I'm a a step away from the computer for like just a minute sometimes even, right? It's like, I gotta go like, you know, grab a water or something. And then in the meantime, the LM has finished whatever it was doing, has fire done the next thing and is firing up like, you know, a slightly controversial tool prompt where I have to review it and it'd say, Okay, approve. then Like I feel like I've wasted time by like stepping away for a minute, you know what I mean? So it's kinda like this like I don't know, antithesis in that respect. But but I do I do think the one one just one more turn, man. Just one more I'll just I'll just I'll just beat this level, you know, and then I'll go to bed. yeah. Shimin (31:34) Mm-hmm. Just Yeah. Yeah. Rahul (31:45) and then setting things up to be done while you're asleep or you're running chores and stuff. Dan (31:51) Yeah. Shimin (31:51) Yeah, like if you I don't know if you guys had experience of setting up like gold farming bots or automated scripts for your video game characters growing up. ⁓ I'm showing my old World of Warcraft OG version ⁓ player ⁓ disciplines here, but that very much reminds me of like leaving an agent on and give it a go and wake up in the morning to see how far it's gone. Like it it's it's very much reminiscent of like waking up at five before school starts. Dan (32:00) Ha. Shimin (32:19) To see how much ⁓ gold I've made in the World Warcraft auction house. And I also do think running multiple agents does remind me of StarCraft, like the article mentioned. Like this idea of you're paying maximum attention, you're always clicking through the tabs, and sometimes there's a fire. Yeah. Who knew my APMs will come back? Dan (32:37) That's true. Yeah. For those of you that that haven't played StarCraft extensively, like the the really like pro players and you know, even bad players like I guess will eventually learn this like me. you basically set up a bunch of hotkeys that you're cycling through like your base, your army's forward position, maybe a defensive position or two just to make sure the other guy isn't doing anything wacky. Like secondary base, like Rahul (33:04) you Dan (33:05) production, you know, every mining facility you've got. And you know, like so it's just like click, click, click, click, click. And there's always a task each time you go through the the loop, essentially. So yeah, it is funny. It is pretty similar. Rahul (33:17) And what was it like six, seven years ago when Alpha Star competed against the top Starcraft player? So I'm sure they learned ⁓ a lot of just not just like can we beat a human at Starcraft lessons from them that got applied to these products. Shimin (33:22) Mm-hmm. Dan (33:29) Yeah. Shimin (33:34) Yeah, even the problems are similar. Like the reason why I stopped playing StarCraft was the games were too intense. Even though they were like twenty to forty minutes long, I like am pumped in adrenaline, like my hands are shaking after a match. Sometimes I'm very upset because I lost when I shouldn't have. And this kind of reminds me of being overwhelmed by having too many sessions concurrently happening. Dan (33:57) That's why I should just play big game hunters against the computer in Brood War. Just saying it's nice and relaxing, takes a couple hours, like, you know, you wind you wind up with whatever equivalent of a Protos carrier fleet is. Anyway, sorry, we digress Shimin (34:03) Yeah, just Just kick their ass every time, yeah. Rahul (34:13) Before we move on, so the author created a repo called agent dice which you can, I think it's just skills that you can plug in. where you just lean into the whole, yeah, this thing is ⁓ like playing a video game and then it rolls a die each time a turn ends and the chance grows the longer the conversation's running and then when you land a natural 20, the stop hook's going to prompt the agent on like, reflect on what happened, you extract the patterns that you saw in the con- and it creates an artifact based on that that improves the system going forward. So they've leaned into gamifying it further and threw a dice into it. Shimin (34:54) Mm. Yeah. Dan (35:00) I thought you were gonna say when you when you roll a twenty it basically like fires a kill switch and you can't use the agent anymore for like five hours or something to like force you to step away from it. But Rahul (35:05) hahahaha Shimin (35:12) that would be healthy. I'll be too healthy in in the world in the age of AI. Alright. ⁓ okay. let us move on to my post of the week. this is a post from Decoding AI magazine. they are a sub stack magazine about AI. it's titled How Evaluation Driven Development EDD works by Paul Easton and Alejandro Aboy. Dan (35:36) All right. Shimin (35:36) ⁓ this is another you know new methodology about software development in the age of AI, similar to spec-driven development and VS ⁓ verification and spec driven development that we spoke about ⁓ a couple of weeks ago. the problem is fundamentally when you have an AI feature system, it is very easy to break what has already worked. So if you change a prompt, your refactor tool. an old feature may regress and you may not know it. And this is especially hard when the output of the system is non-deterministic, right? Because you won't be able to catch that regression every time. And the second problem is when you're developing a new AI-based feature, it is brand new, so we have no way of testing and see how good the feature is. So their solution and this blog post does have a little bit relies pretty heavily on opik an open sourced ⁓ AI feature observability telemet telemetry app. the idea is to you first treat every single feature like a hypothesis. And so based on this hypothesis you make code changes and after a code change is done you then run through a offline pipeline where ⁓ you have Claude Code agents come up with imaginary scenarios using Opik ⁓ of course you can swap out Opik with any other kind of scenario generation tool. It could just be another AI that's trained on your ⁓ existing data. And you run a set of evaluations on your new code base. only when your feature ⁓ does its hypothesis. And does not cause new regressions to you actually open the pull request. that is the I think heart of the methodology. It makes a lot of sense to me because I I've been thinking a lot about you know what to do with all these AI power features that you have no evaluation harness for, right? Then you have no idea how good they are. You have no customer feedback, so are you even building the right thing? and this is one way to solve that problem. Kind of similar to how we had ⁓ the age before tests where we just wrote spaghetti code and now we have testing and regression tests and sometimes even test driven development. This is kind of the eval driven development part of our next age. The article also goes into different types of data sets. persistent hand-built evaluation sets that you know are the gold standards that you use to catch regressions, as well as on-demand synthetic data sets used to evaluate a feature that you're currently working on. ⁓ I find that dichotomy interesting. and since we're in the land of code, they also have two different evaluation types. The evaluator could be either for code metrics that you know look for deterministic linters, kind of code metrics, or you can use ⁓ AI as judge, LMS judge to score for subjective things like completeness, accuracy, ranking quality. also a helpful dichotomy to have. I think altogether this seems like a really powerful new way of doing software development that is worth keeping an eye on and maybe try out in your own workflow. At least I try to always build in evals as a a foundational part of any AI features that I'm building whenever possible. Dan (38:49) You mean AI driven or AI assisted development of? Shimin (38:52) AI driven. So if if the if the feature uses AI in some way, if the feature is non-deterministic in some way. Yeah. Because if it is deterministic, then the old unit test, regression test framework still serves us well. You know what this reminds me of, actually, Sidebar this reminds me of when we went from Newtonian physics to quantum mechanics. We went from a nice deterministic world to something that is non deterministic and fuzzy. Dan (38:57) Mm-hmm. Shimin (39:18) And we need to have new techniques and new harnesses to work around it. Rahul (39:22) I like the aggression setting. Set it to max aggression. Bring in, bring a joker here. Shimin (39:24) Wait, expla ⁓ explain to us what is the aggression setting? Rahul (39:28) So you have these two modes where you can do a manual quick check versus automated experiments. The manual quick check is similar to like you want to just do a lightweight check. It's not going to have any database. It's not going to set up experiments or anything. And then you have Shimin (39:32) Mm-hmm. Rahul (39:49) to which is automating judgments. So when you're shipping new functionality, it creates a whole data set. The judges are going to score different items and then create experiments to make sure that they can catch differences in everything. You can pair all of that with an aggression mode. it picks how much of a jerk you want your reviewer to be and ⁓ like how every serial you want it to be and you can go from happy path and I click the things and with work looks good to me ship it to like no why does it have this like you know one millionth of an edge case that is really not going to maybe do much and so you can tweak it Dan (40:13) Ha ha ha. Rahul (40:31) across that spectrum to get some really adversarial reviews as well. So that was a, it's almost like simulating real life because you see reviewers in real life, PR reviewers at both spectrums as well where you have the people who are like, yeah, I at it, it looks fine, just a bit, whereas someone who just like really goes through everything and it picks even the small things and you can pick which one you want to go for. Shimin (40:58) Yeah, except the nitpicker will burn more tokens. So it's even worse than the real life nitpicker. Rahul (41:01) True, as long as you have toe cans. Dan (41:02) Ha ha ha. Shimin (41:04) All right. ⁓ Dan (41:04) Potentially lead to over editing. I don't know. Shimin (41:07) Yeah, and I worry if if it's too aggressive it causes the whole system to drift a little bit into yeah, some unexpected or unnecessary ⁓ specs areas of the spec. Rahul (41:17) Yeah, and they do call out like you don't want to go full adversarial necessarily because you don't want to like, yeah, like you don't want to prematurely optimize and also just have, you know, waste time. Dan (41:23) Never go full adversarial. Shimin (41:26) Yes. Rahul (41:34) So you kind of have to anchor it to previous trace. Shimin (41:38) makes a lot of sense. Alright, shall we all move on to our favorite segment of the week? Two minutes to midnight, where we take a look at the financial side of AI and make our best guess on how far we are into the AI bubble. the clock is based on the ⁓ Armageddon clock by the Bulletin of Atomic Scientists, and we are currently at five minutes and thirty seconds. Dan (41:44) Yes. Shimin (42:03) ⁓ closer to midnight, the worse it is. So Dan, why don't you kick us off? Dan (42:07) All right, so we're gonna lead things off with ChatGPT's market share has dipped below 50% for the first time. So this is coming to us courtesy of TechCrunch. up until around January, ChatGPT had commanded over 50% market share, but by the end of May, it has fallen to 46.4%. ⁓ and the article attributes that to the rise of Gemini, which is at 27.7%, and Claude, which is at 10.3%. and then everything else, Grok Proplexity, Deep Seek, and Meta AI have less than 5% market share. the thing that was interesting about the those like graphs they show, if you want to scroll down a little bit for folks on on YouTube, ⁓ is that in terms of the actual like recent growth leading up to May, and granted these, you know, are lagging ⁓ the current month by a little bit, but Gemini had some optick, but if you look at ⁓ like visually, Claude seems like it's growing quite a bit. Granted, Gemini's, you know, still at 30%, so like Claude growing is relatively smaller, but it's interesting and I wonder how much of that had to do with all of the sort of recent back and forth with administration, both around like the fable stuff, because that's brought even more attention to it. And then previously the run-in they had with like the Department of Defense or whatever they're calling it these days. so the other thing that I thought was worth calling out in this article is like less about two minutes to midnight, but just something we've talked about before, which is that users are increasingly willing to switch between assistants. and that data was courtesy of I think Sensor Tower was the the provider for it. and then the other thing that's kind of wild is growth rates have decelerated in terms of both spend and overall growth. So they're claiming that that is likely a sign that the market is maturing, ⁓ even as the absolute numbers continue to climb. Shimin (43:48) Mm-hmm. Dan (44:00) And then the other sort of really telling thing, which is kind of comes on the back of what we talked about how was it like a month or two ago, right? Where you'd I think you'd brought the scenes that like developing markets particularly are using AI a lot more than the United States. There's like a lot more haters in the US than than other countries. So Asia recorded the first download decline of three point three percent in Q one of twenty twenty six. Shimin (44:25) huh. Dan (44:26) ⁓ and then the last stat that I'll just drop just cause I thought it was interesting was that ⁓ thirteen percent of Anthropics users are paying for a subscription plan. which is both less and more than I thought. It's kinda funny. I don't know how it manages to be both, but yeah. Shimin (44:38) Ha ha ha. No. Them filthy casuals. Rahul (44:41) Are these only consumers or like businesses or something? Dan (44:45) Yeah, I believe this was like not Enterprise. but I I could be wrong. Shimin (44:48) Correct. Yeah. still though, overall like looking at their numbers with ⁓ Chat GPT having 1.1 billion monthly users, Gemini with 662, and Cloud with 245 million. Like, we're talking 2 billion, assuming they're non-overlapping, which is definitely not the case, but it's a good idea of the scale of the whole thing. We're talking like one in every four persons on Earth. the TAM doesn't get much bigger than this. There's no Dan (45:13) Yeah. Ha ha ha. Shimin (45:19) Ten X from here on out. Yeah. Rahul (45:21) Just read the SpaceX IP address Shimin (45:24) Ha ha. Dan (45:25) Damn numbers for their IPO. Yeah. ⁓ yeah. That's true. Rahul (45:27) The time can get bigger if you just keep expanding your ambition. Shimin (45:32) Yeah. All right. my news item for this week is ⁓ Nvidia is in the process to try and raise over twenty five billion in bond issuance the first since twenty twenty one. ⁓ to put this in perspective, ⁓ Nvidia currently has something like eight billion dollars in bond outstanding, so they're raising ⁓ They're gonna more than double their total outstanding debt after this issuance. And this comes at an interesting time because Nvidia also ⁓ recently dropped something like five percent over the last five days. So it seems like the market demand for Nvidia stock may be hitting a plateau or even a slight dip. So but the The monster must still be fed with more tokens, so ⁓ they're now moving towards debt issuance. And that's not a good sign in my opinion. Dan (46:28) Although I'm I'm still pretty I always forget which one. Bullish is like you like them, right? Yeah, I always get bull I don't know why I get bull and berry confused, but I do. Shimin (46:33) Mm-hmm. Rahul (46:33) Near. Shimin (46:36) Think about bears are always sad. That's how you know. Bears sad. Sad bear. Dan (46:40) Yeah, okay. The bulls are angry. I don't know. Anyway, ⁓ I'm still bullish though, because I actually think that their move into ARM stuff with Microsoft is pretty smart. And it will be pretty interesting to see how that goes. Cause like, you know, we've we've sort of hinted at this last week too, with the article that Rahul brought in where it was like the writing might be on the wall for like having a personal agent that's running on your hardware. Shimin (46:44) Yes. Dan (47:06) And is just calling out to these other APIs. And I think if that's the case, having a machine that can run a halfway decent LM on board is kinda smart, you know. Especially if it's relatively like all things considered power efficient. So Rahul (47:22) And Nvidia has like half of the revenue from AI when they say, you know, the industry has done half a trillion, like 250 billion or so that is purely Nvidia making money. So if you look at it from that perspective, they're the only one who are making the margins and the profit and the money. is not as concerning as let's say Oracle or Meta or similar to. Dan (47:48) But if they're but they're raising debt though, right? If they're printing so much money, why are they raising debt? And then I'm sure there's probably some like corporate y reason for it, but like I don't know. It does it does concern me a little bit. Rahul (47:56) Yeah. Shimin (48:00) It's the same question I have of why is Alphabet raising debt? Y your your money printing machine. Why is Facebook raising debt You know, like it it's a little concerning. Rahul (48:04) Yep. Shimin (48:09) Alright, until our last article, brought to us by Rahul Dan (48:12) Ed Shimin (48:13) Ed is back. Dan (48:14) Yeah. Shimin (48:14) Where's Ed? Rahul (48:15) Where is your ad at? Edzitron got exclusive access to OpenAI's, the financial documents that they submitted. And this has been independently verified by Financial Times as well. So there's not too much, any like he's making any numbers up. And he doesn't even, didn't even comment much on the numbers. He just like stated them as plainly. as you can see them and leave say his commentary for the next week. Thanks to note there was a in 2024 they had five billion in loss and that shot up to 38.5 billion in 2025. 2025 is when they did the whole conversion from nonprofit to for-profit. So there were probably like one-time losses that came as part of that. Their operational costs have been exploding. They went from 12 billion and changed to 34 billion in 2025. And they spent close to 200 billion on R &D. The revenue growth has increased obviously in 2024 they did close to 4 billion, 2025 they did 13 billion. And there's this like tight interdependency that when Microsoft had published one of their quarterly reports, they had like one one customer was the largest like. consumer of their computer and stuff and then later it was like yeah open AI was basically you know I will not fully draw the circle but you know what I'm referring to and they've gotten some money from SoftBank and Microsoft but that was not even a billion so whatever we're talking about here so yeah this kind of like they're planning to go to IPO Shimin (50:07) Mm. Rahul (50:12) financials obviously don't look neat here, but we just saw the SpaceX going to IPO on Vibes. Elon, to his credit, does have a great retail following, which is what the IPO was successful off of. So we'll see how people feel about OpenAI and can they also write the retail labor. Shimin (50:34) Yeah. is Sam Altman as big of a Giga Chad as Elon Musk? I don't know. Dan (50:39) Ha ha ha. Rahul (50:42) One other thing here that I did ask Gemini about is in both 2024 and 2025 they have this line item that says, net loss attributable to non-controlling members capital. I'm like... What the, what is that? And this goes from like, I think it was four billion or something in one year. And then, or no, it was four billion in the second year. close to four billion in the first year as well. I do not have enough financial acumen to be able to tell what that is, so I will read what Gemini said. It represents the portion of a subsidiary company's losses that belongs to minority shareholders rather than the parent organization. So four billion goes to that as a loss. Shimin (51:25) Mm. Interesting. Ed also pointed out that OpenAI has just over fifty billion in assets with almost half of that in cash. So if you compare the fifty billion in assets to the f I don't know, fifty seventy billion that they spent in expenses last year, does not paint a great picture of their cash flow, yeah, or their run way right? Rahul (51:36) Yep. Dan (51:49) They're run rate. Yeah. Shimin (51:52) So Rahul (51:52) And they didn't raise, I guess they had gotten some like compute commitments, and they had the whole like, yeah, it's close to 900 billion or whatever valuation, but the actual money they got was not that much compared to that. And a lot of it was just like, yeah, we'll give you compute. And if you had some certain targets, we'll give you some more. But they didn't get all the like cash on hand or something. Dan (52:13) Just totally way outside my wheelhouse, but I immediately want to start like modeling that rubber to understand like where will these lines intersect? Come back then for some fireworks, probably. this is just gonna be Shimin (52:26) Yeah, this probably also explains why they're going IPO pretty soon 'cause they probably need that cash injection this year. Dan (52:31) The cash, yeah, I know. Otherwise yeah. And unfortunately the only signal we have right now is really the the SpaceX IPO where like they were up quite a bit and now it's kinda like tapered off a little bit. I think they're still over the initial list, right? But yeah. But it's definitely some of the initial fire has ⁓ chilled. Shimin (52:48) By a significant amount. Yep. Rahul (52:54) No, Shimin it came back. They started at 160 and they're currently at 156 so they're compared to the initial price. It's currently Dan (53:05) Yeah, but it was like over two at one point, I guess is what I'm referencing. It was kinda like boop boop. Rahul (53:08) Oh yeah, yeah, yep it did and then this last week it just went down into the right. Dan (53:15) Yeah. Shimin (53:16) Yeah, so Rahul (53:18) Yeah Dan (53:18) Down into the no. Shimin (53:20) ⁓ all that said, how how do we feel about the clock this week? Dan (53:24) I know it's probably super boring to I don't know, what's worse, that we flip flop every week, or that it's boring to be like we need to wait and see because it's about to get super exciting. But that's really where my head's at is where my head's at. That's right. Come back next week for more puns that you can shake a stick at. Rahul (53:35) We didn't see it. Hehehehehe Shimin (53:39) Ha ha ha ⁓ I I think we got three kind of bad news in a row this week. You know, between Nvidia needing to raise bond. Open AI is losing more money than we thought and at the same time is losing market share. Like that is not comforting for me. Dan (53:59) Mm-hmm. Yeah, that's fair. Shimin (54:05) I I would actually push to move it forward to like five ten, five fifteen. Dan (54:10) Okay. How about just five? Shimin (54:11) Okay. Five works too. I d I I do I do feel Dan (54:12) That's too big of a jump. Still pretty far away. We've been much closer. Rahul (54:17) Yeah. Shimin (54:18) Yeah. But like originally I could've I could think like OpenAI maybe has it till like November or something. They go IPO in like October. But now it feels like they're actually burning a lot of money. so I'm my gut my gut says ⁓ five sounds good. Okay, let's do five. Dan (54:26) Mm-hmm. Rahul (54:31) you Dan (54:33) All right. Shimin (54:33) Five minutes of this. Rahul (54:34) find out all this within the next four months. All the IPOs are likely going to happen before the midterms. Dan (54:37) That's true. Shimin (54:41) Yeah. Cool. That's exciting. ⁓ and Rahul (54:43) And people, you know, if they don't do it, they'll spend all their money on Black Friday deals. Who has money to buy stocks after that? Shimin (54:52) I'm excited to use for our November fifteenth ⁓ episode where we digest all the happenings from early November of this year. Dan (55:00) We w we gotta get Nathan back for one of those too. It'll be great. Rahul (55:03) Hehehehehe Shimin (55:06) But as as things stand, Winda put the clock at five. And as always, with the clock being set, it calls for the end of the show. So thank you, listeners, for joining us ⁓ for our study session this week. If you like the show, if you learned something new, please share the show with a friend. You can also leave us a review on Apple Podcasts or Spotify. It helps the people to discover the show and we really appreciate it.