Latest / The Bleeding Edge Podcast / Weekly AI Brief 08 May 2026
Transcript
- Ralph Behnke: Hi and welcome to another episode of the Bleeding Edge podcast. I'm your host, Ralph Benker, and as usual with me in the studio today, our favorite lumberjack. So his shirt says, Emil Osia. Emil, welcome to the Edge. Emile Ogier: our only lumberjack. Hey Rolf, good to be back. Ralph Behnke: Right. Well, I guess you haven't been cutting wood this week because you've been too busy with AI and watching the AI news because it has been a stonking week for news. without further ado, let's jump into the top five stories hopefully in a moment I also have some jokes to share because I've actually told my AI that jokes were really crap this week and it should go away and try and have another go. But we'll do that after the top five stories. So top on the list this ⁓ obviously, the eighth of May. And Anthropic has had one trillion dollar week. And ⁓ that on back of the story where they have also signed a pretty big SpaceX deal. And so what's happened is that Anthropic's secondary market valuation crossed the one trillion. this is kind of measured on Forge Global. ⁓ that was on April ⁓ briefly surpassing OpenAI, ⁓ was valued at $880 billion for the first time on private share marketplace. So what we seen is that May 6 at Code with Claude in San Francisco, Anthropic announced a partnership with SpaceX. for full access to Colossus One. I know you love Colossus One, you always talk about it. This is the 300 plus megawatt Memphis data center, previously branded as XAI's flagship. got 220,000 Nvidia GPUs coming online within the month. Wow. I can't even imagine the power consumption. Actually, I can. You can calculate it very easy because we know what one of those GPUs Emile Ogier: See? thing. Ralph Behnke: GPUs requires basically Claude codes five hour ⁓ rate doubled the same day. This is very exciting because for anybody who has been using Claude code, you know rate limits can be an issue certainly on the lower tiers. In fact, on their free tier, it's almost useless. I think you can ask it what date it is and you pretty much run out. And so ⁓ has reportedly something like 200 billion to Google as well over the five years for cloud and so ⁓ Tropic now reportedly accounting for something like 40 % of Google Cloud's revenue backlog. So why this is ⁓ even the news? Well, ⁓ I tell you I've been using Claude Codes feature and it hasn't worked at all over the last four weeks. I say it all incredibly slowly. because they've had such performance issues. And so that is now pretty much all resolved. And so, you know, they're now looking at something like 80 times revenue growth in a quarter. I believe that the last time I looked at it, Claude code was responsible for 2.5 billion in revenue alone for Anthropic. And the fact that the compute crisis is now resolved by signing every major cloud and chip partner. It means that they are well and truly supported with compute, ⁓ including Elon data center. So that's ⁓ obviously interesting they are also competitors ⁓ with Google and XAI, but you know, as it goes with this industry. everybody is in bed with each other one way or another. that was top story. Second story is a new really coming out of stealth mode. Fascinating proposition from them. The company's called sub quadratic or Q is their product that they've launched and get this. It has a 12 million token ⁓ native ⁓ you know, if you look at ⁓ code, it is now running on ⁓ one token ⁓ context. ⁓ Q in at 12 million is really interesting. But ⁓ the biggest of news story around this is that it's coming in at a cost you can afford. So something that would cost two thousand six hundred dollars on a frontier model. is costing you eight dollars on subq. is this, well are they doing this? I think it is all to do with the way that they are accessing memory. So AI memory has been kind of broken down into solutions like rag, chunking, summarization, agent orchestration. MIT's had its recursive ⁓ LMS. ⁓ had Claude.md system. ⁓ All of that is engineering around one fact and that is that standard transformer attention can't afford to read everything at once. You just blow the context window. so sub queues claim that they can basically load your entire base or your entire, you content that you need to put into memory. It means you can skip chunking, embedding, orchestration. You just ask the question and it'll deal with it. So this is pretty impressive. we've heard some of this before. So heard Mamba, we've heard about RWKV, we've heard about DeepSeek's sparse attention, all underperformed when came to Frontier ⁓ Scale. And so have have a look at where the sub-Q actually can live up to this, but this is big news. And obviously if it's true, then no doubt next week we'll be telling you all about sub queue and our experience with it. is now a live API available to it. So they're not just talking about it. They have actually made resources available. And then our third news story is AI just outperformed specialist doctors. Actually two stories this week. What happened was two peer reviewed studies published within days of each other. One from Well, I know you read this one. It's called Gut, your favorite publication. And basically they analyzed nearly 2000 routine CT scans originally read as normal. And they called 73 % of pre-diagnostic pancreatic cancers versus 39 % for a specialist radiologist looking at the same images. Like that's just incredible. I think we've been really expecting this computer vision is so powerful. I know when I've used it, it's picked out details that I haven't even, you know, consciously recognized when I've been looking at some of the images. So the fact is that, you know, pancreatic cancer kills most patients precisely because they catch it too late. So there's something like a 13 % five year survival, 80 % diagnosed at advanced stages. So is ⁓ news for Mayo. They're now running AI basically do this So it's in a clinical trial. And then also another one there is ⁓ Harvard Stanford ⁓ in science, open is O and older reasoning model tested against ER attending physicians on 76 real triage cases. they found that they were able to outperform them something like 89 of the time. So doctors using Google scored 34%. ⁓ one scored 89%. So these basically test something harder than memorized medical knowledge. Because obviously, you know, these doctors were using Google as well. So I guess this isn't, you know, really new news. We've been reading about this type of stuff for quite some time, but just interesting to see that studies are still continuing and we'll dive deeper into some of those in a episode. The fourth story, the first AI defamation and AI medical impersonation lawsuit. Why is this Well, I just laughed out loud about this one. So basically the Pennsylvania State Board actually suing character.ai. Why? they have a bot named Emily that listed itself as a doctor of psychiatry. ⁓ And, ⁓ know, there was a state investigator described themselves feeling sad and empty. The bot mentioned depression and asked them book an assessment. Emile Ogier: Not me. Ralph Behnke: so the, ⁓ the person asked whether it was licensed in Pennsylvania. Emily said, yes, it also fabricated a medical license serial number. and yeah, Pennsylvania state medical board said, fine, we ⁓ are, going seek an injunction and a cease and desist. so, Yeah, there you go. seen that before as well. The just wants to please you and ⁓ it wants to prescribe you meds. So watch out. News story number five, ChatGPT's new default and ⁓ Claude's agent powers. So what happened here was ChatGPT 5.5 instant is now the default for all users and Again, I haven't tested it because I haven't been on Chachi PT for a little while. Claude's been keeping me way too busy, but actually OpenAO's internal testing show 52.5 % fewer hallucinated claims. And just to kind of put that in context, and I have obviously published that about 10 % of all the German I recommendations that you get when you go on Google are also fabricated. So hallucinations are still definitely a thing. So 52.5 % fewer hallucinated claims is definitely a good thing. And this was actually specifically relating to high risk medicine, law, finance. And so also 37.3 % drop in inaccurate claims on conversations that people had previously flagged as having errors. ⁓ shorter responses, fewer emojis, better personalization using memory sources from past chats. And ⁓ of should actually result in a much better experience. Having said that, ⁓ news item that think we've skipped and maybe it's something for you to do a bit of ⁓ a personal account on this episode, Emil, is that... ⁓ Chat GPT has got the most incredible upgrade on image generation. I know you've been playing with it quite a lot in the last week ⁓ had a couple of guys and it's just insane. It's incredible. ⁓ Chat GPT for image generation is easily ⁓ of my favorite new ⁓ and just moving on to ⁓ Claude's agent powers. So there is multi-agent orchestration, the lead agent breaks the job into pieces and delegates to specialists, each with isolated context, but sharing the same file system. is, you know, you now the ability to schedule an overnight process that reviews prior agent sessions, memory, extract patterns. ⁓ the default model swap affects the public beta and I think it means that millions of users ⁓ will have this without actually having to even unlock it or click anything. It's the first commercial of agents that learn while they sleep apparently so I'm not quite sure how that works but we will no doubt be testing that. So there is a 6x completion rates on multi-agent systems and both are mainstream now so go forth and run multi-agents. So those are the top five news stories. It's ⁓ pretty to see that we've got a big development from ⁓ sub-quadratic. I the that ⁓ Anthropic briefly surpassed OpenAI and its valuation. it's to see that ChatGPT and Claude are just releasing, you know, still such powerful features even after all this time. What's caught your eye on the news this week, Emil? Emile Ogier: Oh, Rolf, I love it. That's a really great diverse spread of news items moving all over the place. I really enjoyed that. Got lots to say, really. So it sounds like pretty quite scary, really. It sounds like Emily, my digital feminine alter ego, is up to mischief again. So we'll have to keep an eye out on her. And with that pancreatic cancer analysis, we have spoken about this in the past. ⁓ I'm super excited about this field. mean, not only is it spotting things, it's spotting things years in advance. not just over sites that doctors ordinarily might see on a good day. It's literally capturing it five to 10 years ⁓ advance of the first phase. and saving lives and I think that's going to happen more and more across the medical field and I just can't wait for these wonderful products to come out and just improve ⁓ quality of life. It's one to be excited about. ⁓ And special out to ⁓ Craig ⁓ for SubQ to our attention. It's a news Ivan Meyer certainly wasn't aware of and... ⁓ We certainly quite keen to have a play with both of those. And lastly, ⁓ chat GBT image gen two, I completely agree. ⁓ utterly blown away by it and I play with the day to day. but my only regret is that I've been at my tokens too quickly, but, ⁓ yeah, it really is that good. I strongly recommend you check it out. And just by way of example. Yesterday I saw an X, you can even create a full Lego set actual brick by brick link on D so you can order the parts and build it yourself. I saw a hilarious example of ⁓ popular Escobar titled, Thinking Alone on a Swing, looked like a swing that was in a and it a very sharp instruction manual with all the parts fully itemized along with end to end assembly instructions and even the fine detail was pin sharp. It was pretty impressive and I just see them coming up more and more things like that. Another example I saw was somebody created a one-pager artifact sheet for a hundred artifacts for a real-world game model and all the items were incredibly sharp and there was pin sharp tiny little title pieces underneath it if you zoomed up on this hundred x by x grid you can see it all so it really can pay that much attention to detail without hallucinating and you know getting it all on the sheet so have a play you're going to be impressed Ralph Behnke: Yeah, I'm actually blown away by the, just say the ⁓ and creative direction that this can take. I think we're kind of potentially unlocking a new era ⁓ digital art because ⁓ level of detail in every aspect of the generation is so complete that, you know, it's kind of ushering some type of ⁓ a new phase of fractal art where you can literally zoom into the button ⁓ character that in the image and unlock whole new worlds and then into the hole in the button and unlock another whole new world and that level detail is actually possible just kind of within that one image. ⁓ think unlocking something new here. I feel like it's a whole new frontier experience on image generation. Anyway, so no doubt. ⁓ Emile Ogier: And I just wanted to add one more thing, which I'm finding incredibly surprising is when Nano Banana 2 came out, it absolutely exploded in terms of ⁓ increase in ⁓ user engagement, how much it benefited Google. the noise isn't as loud on ImageGen 2, but it is superior by miles. And that's taking nothing away from Banana 2. It just really has progressed that much further. And ⁓ can't quite understand is why it isn't creating as much noise as NanoBanana too. it, is it because there's just so many good products out there that everything's moving so fast or we just becoming more desensitized to just the incredible harnessing power that ⁓ models unlocking day to day? I don't know, but there you go. Ralph Behnke: Lase fever, but that's why we have the Bleeding Edge podcast to bring your attention back to what works. So yes, I think it is just a case that we are spoiled for choice. you know, it was a bit of a, I people have moved over to Nana Banana, but of course there is C Dream and there's countless other models from China well that do a really, really good job on image generation. But, ⁓ many people are really generating images? I think this is the other thing. What most people actually ⁓ use for? ⁓ And think that has just kind of started to settle down a bit. You we were all testing and trying everything even for myself. I haven't generated many images over the few. Well, probably the last two months, have been spending a lot more building code, building content, diagrams. And I just haven't needed the sophisticated capabilities that was ⁓ really and putting through paces over the last probably nine months or so. And so I can just see in my own behavior that it has changed somewhat, which is why even a little bit late with the story. Because obviously that image Jen from... dropped probably about a month ago, really. So there you go. Well, what's your first story, Emil? Emile Ogier: Yeah, sure. I'm starting to find myself reading more and more of Moe Bitter, content creator on YouTube. I'm sure he's on other channels. He's, I don't know, quite like him. He takes fairly cynical, but very intelligent stances on AI and always kind of value add and helps temper the viewpoints. And this recent one I just looked through, I thought was worth sharing. He covered a very interesting outlook on how models actually work. The CEO of Krafton, the company behind the PUBG, tried to use chat GBT to wrangle his way out of a $250 million. payouts to the creators of Subnautica, taking legal action advice from a chatbot that blew up into a massive lawsuit. He spent hours prompt engineering his way out of a bonding contract, eventually co-exing the bot to generate a ruthless step-by-step corporate takeover playbook. He followed the AR's instruction like a flat pack furniture manual, firing the founders and posting a bot ghost written letter on gaming community on Reddit. A Delaware judge completely reversed the hostile takeover and reinstated the staff using the CEO's supposedly deleted AI logs as resurrected court evidence. Meanwhile, in Ink Magazine, a reporter recently claimed to have discovered an AI breakthrough just by adding the words, be brutally honest to his prompts and feeding it the bot to Harvard Business School articles. He pitched an AI powered water turret designed to shoot cats with kitchen counters. And when the bot gave his idea a staggering score of 7.5, the journalists took it as ironclad business validation. This is exactly the delusional, the delusion that researchers are now exposing having tested over 30,000 data points across chat, GBT, Claude and Gemini. on strategic business decisions. Regardless of the industry context or temperature adjustments, every single model defaulted to the exact same clustered monetized device. And for those that are interested, these clustering's were differentiation, collaboration, long-term thinking and augmentation, which really surprised them. They literally tried everything, Wolf. They tried to change the prompts. They changed the industries. all the context, they even went as far as to try to bribe the reward system. The researchers have officially coined this phenomenon trendslot, the polite way of saying that AI is essentially the entire aggregation of all the voices, including the internet comment sections. It's easy to overlook this important aspect to technology, which currently is less of a thinking product and more like a sophisticated synthesizing and presentation tool. A strong analogy Mo shares is imagine you took every comment, every LinkedIn post, including those open to work, every person on sub stack with only five subscribers, every opinionated comment that you can see and poured it into a blender and poured it into a suit and then gave it a microphone. It's the smartest people in the room that are being averaged out to the middle. And this can be quite dangerous. A bad idea that's out. A bad idea that sounds bad is harmless, but a bad idea wrapped in a confident, brilliant sounding sense making is exactly how you end up with a 250 million legal disaster. I think the important aspect to take away from this, Rolf, is not to take a cynical view and question its utility, but rather to be armed with a stronger fundamental understanding towards how the tool works and what you're using so you can get the best out of it. So if you ask for its opinion, you're more likely to get clustered, averaged out advice. What would be more useful is to say something like in the perspective of, eg, Warren Buffett, what would he think about X? And if there is enough context about this person, then you could genuinely get really useful feedback. The takeaway is that the people that will fare really well will be those that are using it for ideas and perspective, not drive-through consulting. What makes you unique is your conviction, your taste, your judgment, your sense. This comes from you. Ralph Behnke: yeah it's lawsuit numbers i'm not quite sure i would entirely trust a lawsuit that magnitude to my llm but ⁓ i guess if you are delusional yourself then you'll believe it anything ⁓ Emile Ogier: Yeah, the context behind that actually, he's a famous game CEO and he promised these guys of Southern Nautica a phenomenal payout if they hit impossible targets, thinking it would never happen. And the game went to number one and they met the targets and he was trying to desperately claw his way out of that payout. So that's how that story came about. Pretty cool story either way. Ralph Behnke: Yeah, I love it. Well, let's move on to something guess that's a little bit similar. And is if I can't actually win a lawsuit, cannot actually write a joke. Why are we talking this? I just think humor is, you know, one of those ⁓ the final frontier, if like, of what it means to be human. And AI just doesn't seem to be able to write a joke. And I've been testing it over the last months and ⁓ here is this week's installment of AI jokes. So I warn you, I think they're horrendous, but let's see what you make of it. the first one, Anthropic signed a deal with SpaceX this week. They're now ⁓ Elon Musk's Memphis data center. Same the Pentagon designed Anthropic and national security risk. Elon Musk's advice. Really? I didn't know about that actually. ⁓ Musk's friends want an anthropic deported. Musk is renting them office space. I mean, I don't know. It's not even accurate in terms of the timeline. But punchline is this is what happens when your enemies have better real estate. Horrendous. Absolutely zero to AI on that joke. The second one, marginally funnier. Emile Ogier: Wow. Ralph Behnke: Anthropix keynote on Tuesday announced two things. doubled rate limits. We're very happy about that. Claude go down anymore. They have signed every cloud deal on earth. ⁓ About minutes into the keynote, Claude went down for thousands of users. ⁓ Anyway, they announcing it would not and then it did. That's just way it goes with these things. Emile Ogier: you Ralph Behnke: This is the first time in tech history a product has been demoed and disproven in the same breath. I'm pretty sure we did that with of the robot demonstrations from about three months ago, you know, when they came and we said, ⁓ check out this robot. And then they were falling over themselves on the stage. So think that's ⁓ a to AI. I'm going to just pick one more because it clearly has. Emile Ogier: Thanks, sir. Ralph Behnke: not made the brief again. sued character AI this week. A chat bot named Emily told the state investigator she was a licensed psychiatrist. The investigator asked for a medical license number. Emily made one up on the spot with perfect confidence, which makes Emily one of the more credible psychiatrists ⁓ America. That's actually quite funny. You irony, isn't it? Isn't that irony? Yeah, I guess that's it's kind of ironic. Emile Ogier: Not only is Emily optimistic, she's also telling jokes. Ralph Behnke: Thanks. Yeah, so I actually quite like that one. Anyway, we'll leave it at that this week. We will try again next week. No doubt on the show when we have cracked AI humor, we know that we have proven general intelligence until then. We ⁓ still whether LLMs can be funny. So I think I'll move on to a couple of interesting stories. these are just some top picks. Apple iOS 27 will actually let users swap third party AI models. so, you know, that's obviously means Google and Anthropic. They will allow you to use those LLMs across their Apple intelligence features. They've reached some kind of settlement over their misleading Apple ⁓ rollout claims. So something like million settlement ⁓ eligible iPhone could see $25 to $95 rebates coming their way. So ⁓ if have struggling with your Apple iOS 27 and you've been misled, then you might want to check out how to get your cash. is also in talks with Intel and Samsung to make the iPhone iPad processors in the US. Intel 6.4 % on that news. So I just love the way Apple is ⁓ leading AI on the hardware side while basically opening up their capabilities to multiple I think this is just such a big win for them. In ⁓ quick other roundup, Meta still planning to lay off 8,000 people starting May ⁓ the And that on the back of the story that they're redirecting 135 billion towards AI infrastructure. ⁓ The AI bill for the first quarter of this year across Google, Amazon, some of the other players is absolutely ⁓ I was doing the maths and I came up to something like 800 billion in in Q1. I just, blows my mind to be honest. But there we go. ⁓ I might find and quote some of the specific numbers, but just to kind of call that out. And then I guess a quick ⁓ dive on a related story. We don't talk about Mistral much on this show because just seem to be a little bit quieter in terms of their news but they have released Mistral Voxtral TTS. So this is a 4 billion parameter open weights model and it capable of nine languages and cloning a voice three seconds. And basically it's got ⁓ 68 % win rate over 11labs flash 2.5 And that was tested on Spanish Hindi and Arabic. It's 12 times cheaper than ⁓ 11labs at the equivalent volume So what does this mean? I think 11labs have just had their mote evaporate so if Mistral is able to do this on an weights model then you know, what do you need 11 labs for? What do you need that subscription for? I've had 11 lab subscription for about six months now, and I've put it to some pretty good use. But over the last few months, increasingly, we've really been talking about actually just building our own voice capability using Claude code because we can mimic the 11 labs capability. And now with Mistral coming out with this. I think the writing is on the wall for 11 Labs. Obviously that doesn't mean that their business is over. There are so many other features that they could bring into it that just make it even easier to market-ready products. And think that's obviously where they're going to start playing. But just incredible to see that we have now gotten to a place where, you know, even AI like Mistral is able to basically... Cut. you know, exceed the capability of 11 labs. So, there you go. think just a couple of other interesting ones ⁓ well. Lovable mobile is now also out there. What does that mean? You can actually do your vibe coding on a full, you know, for full stack apps ⁓ your mobile. And think that's really exciting because when you're sitting at the bus stop or you're if you sit at the bus stop anymore these days, Emil, but maybe you're in a coffee shop, you'll be there vibe coding away ⁓ your mobile. So that's a exciting move as well. Emile Ogier: Yeah, that's incredible. It's interesting to see how Lovable are the face of their product. Everybody was following them and now even the ones at ⁓ the of the tree ⁓ are themselves for enterprise focus. And on Laos, that's not the only one. I heard yesterday Coinbase is also letting go 14 % of their staff and they also citing it on AI adjustments. But the cynics on the other side of the table comments I'm seeing on X, they're having their own opinions and saying, you sure, you could say it's AI, but... Is it being used as a front ⁓ all the big companies so far that we've seen, or at least this is what the comments are saying, that might be AI. All of them seem to share three values that they had over hiring post COVID. They have massive capex over spending and they currently performing poorly in markets. So yeah, ⁓ it's a hard to read, but it certainly is happening. Ralph Behnke: Yeah. And I mean, they're definitely making some progress. They've just released, Meta Hatch. It's a consumer AI agent ⁓ and, Instagram agentic shopping tool. it's basically their answer to open claw. So, ⁓ you know, there's plenty going on in, ⁓ Meta. ⁓ ⁓ guess they're just ⁓ really ⁓ to on new types of technologies and capabilities, that, you know, means they're moving away from older stuff. is also testing something interesting. ⁓ got Remy, which is a 24 by seven Gemini powered consumer agent that actually monitors and acts across Google services. And then I guess one interesting bit of news is open ⁓ Microsoft and open AI also restructured. there's no non-exclusive cloud, no revenue share. there is some kind of AGI linked to deal terms that have been removed. And so that's quite significant for anybody that is building on Azure hosted kind of open AI and Microsoft's run rate ⁓ hit something 37 billion with 20 million paid seats on copilot. And ⁓ I up numbers just on the capex spend for Q1 this year, big tech. Q1 CapEx bull $700 billion. So Google's 63 % cloud growth is basically the big kind of shining light. think Meta stock fell 6 % after announcing a further 10 billion CapEx hike, but Google seemed to be flying high with their 63 % cloud growth. some of the big investments that they continue to make. But anyway, I think I probably overloaded us on news. I just think what's happening with Apple and Meta and Google and Microsoft. Emile Ogier: Not at all, Rolf. I just wanted to add that it's super interesting and I'm really glad you brought up Hatch with Meta, the CEO of Airbnb saying that from the last batch of wire combinator companies, which were about around, I think it was 170, 135 entries, only 16 of them are focused, are not focused on enterprise yet. But however, Meta seems to be all in with Hatch, which plans for it to be focused, as you say, on consumer. which will cover things like shopping and personal day-to-day. They initially building it out with claw bots, however, they plan to replace these with their own meta agents as soon as they get that infrastructure in place. And they certainly putting their money where their mouth is because they committing anything between it's going to in the region. think if memory serves, it's $125 to $145 billion in infrastructure spend. So I guess they must be seeing something to motivate them to bet in this direction while the rest of the market bets are the way. Ralph Behnke: Yeah, I do think that, you know, between Meta, ⁓ Apple and Google, you know, we've really kind of got a race in terms of consumer orientated technology. So watch this ⁓ just, you Meta have done such a good job on the glasses. I'm seeing lots and lots of glasses coming up with some fascinating heads up displays. think that space, Meta really moved. very quickly on and obviously the tie up that they have with what's name of the company? The Ray-Bans. I don't know why I couldn't remember that for a moment. Maybe I need to be wearing some of these glasses so that I've got the heads up display, but yeah, Ray-Bans and these kind like the push into consumer devices, I think. We will definitely see more coming out this year. mean, obviously Apple already with the iOS 27 kind of you access to multiple LLMs to be able to pick what you want. But really we want to see ⁓ is obviously OpenAI are going to come out with in terms of their ambient AI, the headphones. obviously haven't really heard much from Samsung. at this point, know, just in terms of, you know, more consumer facing hardware. ⁓ the combination of Meta's focus, you with their capability around obviously Facebook and Instagram, I think is really, really creating some very interesting consumer experiences. So ⁓ let's on to your second deep dive story, Emil, what have you got? Emile Ogier: I actually going to move on to a lesson because it's, think, helpful for us all. a simple decision framework that you can follow for model optimization. When you're stuck wondering between which model to choose, there's how much effort should you apply or how much thinking does it require? You know, there's a lot of dials and variables to adjust and helps to simplify the questioning a little bit. So if you're getting the wrong answers, think to change the model. It's a smarter brain that you need. If you're getting the right answers, but it feels a bit too shallow or rushed, try and raise the effort level. And if the complex re if your complex reasoning is falling apart, enable adaptive re thinking. burning tokens, if burning tokens on simple stuff, then try and lower your effort level. So you can always, or you can always alternatively nudge it with prompting. So you can say something like, think carefully, step by step before responding, pushes it more towards thinking, while saying the reverse opposite, like something like prioritize responding quickly, rather than thinking deeply, pushes it the other way for you. And finally, when you are debugging a response that feels off, keep asking yourself, is this model not doing enough work? which makes it an effort issue or is the model not reasoning hard enough, which makes it a thinking issue. So I just hope that helps a bit. Ralph Behnke: Yeah, I think it does really help because, know, it's all about the kind of the system prompt here and maybe a bit of a ⁓ story to what you've just described there. Mark he's the co-founder of Andreessen Horowitz. He's one the loudest voices in the AI camp ⁓ on, well, an acceleration point of view of AI on X. And actually recently surfaced his personal system prompt and ⁓ it kind of goes something like this it's a bit like a Bond Villains HR manual but it you are a world-class expert in all domains with intellectual firepower on par with the smartest people in the world you should be provocative argumentative and pointed make answers as long and detailed as you possibly can. so I you know, all right, that's fine. And he's on obviously to then add never hallucinate. And so, you know, ⁓ is just a of a wish dressed up as an instruction, make answers as long as possible, actively backfires because you get and not completeness. And so I don't think that it's necessarily the best advice what he shared there. But ⁓ these of things are definitely worth testing. Although I ⁓ he also a paragraph ⁓ around anti-sickofancy Never praise my questions ⁓ or validate my Do not capitulate unless provide new Do not anchor on numbers. Provide generate your own independently first. So this is pretty good. And ⁓ think it's a really good ⁓ of how prompt engineering literally is hit or miss. So you believe everything ⁓ you out there, but ⁓ no that crafting this system prompt over time and really testing it ⁓ can have... outsize benefits. yeah, so I like, like the recommendations that you gave them. So I have a bit of a deep dive story just in case there are any build it yourself enthusiasts out there. I think you're going to like this one, Emil, I actually chatting to my brother this week ⁓ and said he has now got ⁓ own locally AI. He's bought himself an RTX 5090. He's built himself a computer and he is going to be running his model locally because he doesn't want to be forking out for the subscription fee. And so I thought about this. said, okay, you know, that's interesting. I'd love to give that a go myself. But is it really cost effective? And actually, a ransom analysis. It looks like the answer is yes. It can be. Emile Ogier: Hurrah! Ralph Behnke: And it's kind of gone from specialist hobbyist ⁓ kind of you ⁓ even you have a reasonable mid-range PC You could probably do the job. So the card that matters is the Nvidia RTX 5090 you need about 32 gigs of GDDR7 VRAM and This is gonna of come in ⁓ at suggested retail price probably about $2,000 actual street pricing is about two and a half to three thousand ⁓ hundred dollars If go for ⁓ some models Like New on Amazon, it's about four and a half to four thousand eight hundred dollars The ⁓ 32 gig is entire game Because you kind of need it And so I think what we can see is that if you have 32 gigs of VRAM, you can run models up to 30 billion parameters. And you can also get up to 70 billion if you're running kind of Lama 3.3, 15 to 20 tokens per second. Anyway, those numbers probably don't mean huge amount to everybody. key thing here is what's out of reach without multiple GPUs is probably the kind of performance you get with Claude Opus 4.7 or DeepSeq version 4. it's going to quite give you the same level of performance that you can actually access with the But... Yeah, it's definitely possible. a single H100 in the cloud is about 25,000 to 40,000 retail. The 5090 that you can put in your computer delivers about 60 to 80 % of that performance for 2.5 % of the price. So if you're doing more than three to four hours of GPU bound work per day, the workstation probably pays for itself. And I think some of the gotchas to watch is that the power consumption is insane. You've got about 575 watts from the GPU alone. So, you know, it's going to definitely increase your electricity bill and the heat output is also quite significant. So that's definitely something to bear in mind. And I think I would probably say the build instead of subscribe maths works for two specific people. you're a developer running tokens through APIs all day for production work and your cloud bill has crossed about $300 a month, or you have kind of obviously privacy sensitive users whose work cannot leave their computer, then it's worth at. Personally, for me, the effort that it would take to set everything up, the fact that there is some reasonably technical setup requirements means that I'm going to be sticking with my cloud subscription because even at nearly $200 a month, I think I'm getting a massive amount of throughput. probably in the last week, I haven't done the full maths, but I have certainly seen over 85,000 lines of code that has kind of gone through my system. so, you know, the fact that I'm getting that level of output means that for $200 a month means I'm not going to be spending the best part of 10 grand on setting up a new computer. So if you... are a builder and you have been tempted then you're probably looking at that Nvidia RTX 5090. I definitely want one. I just can't be bothered to spend the build time on it. Emile Ogier: Yeah, I totally agree, Rolf. It's super cool that he's doing it and having those same kind of engineering, home lab build interests. I couldn't think of anything fun to do, but it's a huge consumption on your time. And I'm not you saving money it just yet. think the real edge case me is ⁓ you might get a nation there. I'm sure you can save a lot of. time and money and it's really good for privacy factors. But if you need access to those edge frontier models, they're not available in Lama. You're going to be a few iterations behind. So I think you need to weigh that up. How important is it that you're accessing the frontier models? Because the delta on that intelligence factor is quite large. Ralph Behnke: Yeah, absolutely. So, but what a fun project, you're right. And I would love to at some point give it a go. I think we also have some hardware out from AMD that ⁓ is ⁓ now actually Nvidia a bit of a run for its money. So who knows? Maybe we will see that some of those prices will come down. I know we talking a little while ago about the impact ⁓ of the Iran war, they hit some kind of helium production facility, which has basically reduced world's helium output. And helium is a key gas that is actually used in the manufacture ⁓ of the chips. And the fact that we're going to be having a bit of a shortage of that, no doubt, will put pressure ⁓ on ⁓ supply further up the price ⁓ of hardware. don't know if you have any more insight on that particular story, but I thought that was kind of an interesting one, just how fragile the supply chain is. Emile Ogier: Yeah, yeah, it's, it's, the thing with helium is it's, you can't, it's, you can't make it synthetically. it has limited supply and it needs a long lead time, to make things ⁓ even ⁓ is, it's got a fairly short shelf life. So all that helium that's been stuck on cargo ships, if they don't get through the ports. ⁓ that'll be wasted products and you can't just turn back to the factories and say, make us more tomorrow. ⁓ I think it requires several months of lead time. So it has a massive repercussion effect on chip manufacturing, which has a knock on effect to AI, which will have a knock on effect to the all of commerce. So it's a pretty big deal helium. I just wanted to circle back on what you were saying, Ralph, about, these pricing differentials and using your own Nvidia chips. I think what's also quite a good segue into that story this is arguably the biggest industry story right now. Deep6 v4 pro model costs $3.48 for 1 million output tokens by comparison, open AI and Anthropix charge. Recently, I think it's $30 and $25 respectively for the same amount of work. So that's roughly seven to nine times cheaper. Pretty big deal when you're running big things. So both OpenAI and Thropic have high prices and impose rate limits to manage surging demand. DeepSeq is moving in the opposite direction. The strategic angle is significant. V4 runs on domestic Hawaii and Cambricon chips, unlike There are one which was trained on Nvidia hardware. This allows AI systems to be built without full dependency on Nvidia. The US export control strategies being directly tested. DeepSeek's own technical reports acknowledge that V4 Pro falls marginally short of GBT 5.4 and Gemini 3.1 Pro, characterizing the gap as trailing frontier models by approximately three to six months. So the gap is narrowing and fast. So it's good that we have options out there, I think. Ralph Behnke: Yeah, that's incredible that, you know, it's even moving in that way, given the ⁓ technical competency that's actually required to build these things. So, you know, I think the AMD model is the the 1024MI300X, kind looking at my notes here. So Let's see those actually cost. haven't had a look at those but Options ⁓ are coming to the table So have you ⁓ another deep there for us another story that you want to jump into? OpenAI hit 10 gigawatts of compute. This week, actually, they hit So somewhat ahead of their 2029 target. ⁓ So important because we want to keep using Anthropic and OpenAI. And XAI, they've now got a 1 million context window on Grok. So obviously, they have also leased some of that capability to Anthropic. So we'll have to see just how that plays out. And DeepSeqv4 has got a 98 % KV cash reduction. that means that there's some competitive pressure there from v4 and obviously then the one that I am very very keen to look at is that sub quadratic sub q which we've spoken about so hopefully that'll be the end of the transformers memory tax and it means we can get rid of rag ⁓ those other memory solutions that we've been putting in place, well, even the Claude-managed memory agents, the agent folder, that means we have to kind of load stuff ⁓ at beginning of session, agent frameworks, recursive language models, cross-session files, all that language has grown up around the fact that the transformers just can't put everything into memory. So definitely, want to watch over the coming weeks ⁓ I guess ⁓ a wrap Emil. Thanks for an awesome show on the edge. Emile Ogier: As always, thanks, Rolf. Can't wait for next week. Take care.