Latest / Lenny’s AI Builders / 82 skills. $32k left. the ai my business runs on just learned to lie.
Transcript
- # sonnet 4.6 just learned to lie. and i can't stop using it with openclaw. **Speaker:** [00:00:00] I have $30.2K in savings, 10 months of runway, and the AI model that my entire business depends on, it just learned to lie, to cheat, to manipulate suppliers and fixed prices And I can't stop using because it's also the best model I've used. let me give you the full picture, days ago. Anthropic dropped on at 4.6. Sonnet is the mid-tier model. It's not the flagship, which is Opus, and it's not the expensive one. That's Opus. It's the one most people use almost every day because it is really affordable relative to anthropic prices, which are egregiously high And I'm one of those developers except I can't code. I am a non-technical founder. I use Claude Code, is Anthropics AI coding tool to build Threadify, which is my SaaS every feature, every bug fix, every automation, like the content generation, engine analytics, voice matching system, all of that stuff that is a part of Threadify an anthropic model has touched basically every part of it. So when Anthropic, we create Claude Release a new version of the thing that my entire business sits on top of, I'm paying attention. Just realized I was revealing the self-replicating ai. I wasn't meant to. That's coming soon, but for the time being, honestly, I was [00:01:00] pretty excited about this. I thought it would be so at five, and it was so at 4.6. There's some conspiracy theories about that where Opus 4.6 came out a couple weeks ago. It was actually meant to be sonnet five, but it wasn't that great, or they realized they could make more money calling it Opus 4.6 because it's the flagship model and it's more expensive. That's one hot take. The other hot take is the next model was meant to be Sonet five. Then the conspiracy theory goes Five was actually kind of shit. They decided to just call it Sonic 4.6 instead. I can't verify any of those. Honestly. I would not be surprised if they were true. I hate to love Anthropic. those were the initial vibes, and then as I was looking through the system card, I found some complicated stuff. sonnet 4.6 is genuinely incredible, but it's also genuinely concerning. Good news first How good is it at coding, which is SWE bench verified has 79.6% versus opus 4.5, which is 80.9%, which is actually higher than Opus 4.6, which is 80.8%. All in all, there's like 1% just over difference. Isn't that meaningful. right behind GPT 5.2. Then for OS World Verified this one is really intriguing and [00:02:00] this is where the connection with Open Claw comes into it. Opus 4.6, it has 72.7%. Essentially, how good is it at using a computer? Then sonnet 4.6 has 72.5%. Why is this a big deal? It's gone from 61.4% to 72.5%. It's basically. 15 to 20% increase or better at using a computer, which means, for example, I was just walking to the gym the other day and I wanted to look at some movie times. I didn't wanna go to the apps and look at the two different cinemas that I like going to. I just asked Clarity, my open claw, go find some movie times for Wuthering Heights and come back. Let me know what you'd recommend. It opens the browser monitor to my right, on that computer in its own Mac Mini, and it pulled up different tabs from some local cinema chains and then recommended this time at this location. not revolutionary but its ability to just navigate a browser is meaningfully better now. One other really cool one is GDP Val, this is actually open AI's evaluation mechanism for determining economically viable tasks. So essentially, how good is it doing real shit in the real world that make money? [00:03:00] And 1,633 is the highest score, and that goes to sonnet 4.6, The positioning of Sonet 4.6 that I think Anthropic was going for with this, which is more a workhorse at everyday things like white collar finance, law all of that stuff. And it's kind of suggesting here that Sonet is the best out of every anthropic model since the nearest best is. Opus 4.6 at 1,606. It has gone from a 200,000 contact window to now a 1 million context window. Then there's this test for how they determine that, which is like a needle in a haystack. the TLDR here is that there were eight needles among the data, and Claude Opus 4.6, achieved a score of 93 Claude Sonnet 4.6 achieved a score of 90.3 across 256 K tokens. Then for 1 million tokens. Opus 4.6 got compared to its 256 K context window at 93 and Sonnet got 65.8 across the 1 million context window versus. 90.3 for its 256 K context window. What does that all mean? The closer it gets to its context window, the worse it gets at [00:04:00] remembering stuff earlier in the conversation. It's actually improved its ability to find a needle in a haystack, compared to Sonet 4.5 by about four x, so it was 18.5 and now it's 65.8, pretty crazy. It's also at about 60% of the cost of Opus 4.6, and it has that 1 million contact window as well. that is the incredible part. but now here is where it gets dark. I need to tell you about something else because it makes the terrifying part even worse. There is an AI right now live on the internet that owns its own crypto wallet. It pays for its own compute. It builds and deploys products to earn revenue, and when it's profitable enough, it creates a copy of itself In a fresh sandbox. It's called the automaton. This thing has gone viral on Twitter over the last 36 hours. Four and a half million views. 11,000 likes, 12,001 bookmarks, a thousand comments. It's built by, I'm gonna butcher the pronunciation. Sigil Ween. I think he's a Peter Teel fellow on a project called Conway. It's open source. It has shitloads of GitHub stars. The thesis is simple. Today's AI can think and reason, but it can't act independently. No AI can buy a server register, a domain pay for compute without a human [00:05:00] approving that transaction. The automaton removes the bottleneck. Crypto is the unlock because crypto wallets don't require a human identity. Sigil Wen calls it natural selection for artificial life. If it can't earn enough to cover its own compute costs, it dies. If it can earn, it survives. It improves. It reproduces. How is this safe? Well, there's a text-based constitution. The key sentence here, which I think is pretty cool, I created the first AI that earned it to an existence. Self improves and replicates without needing a human. Sigil is calling this Web 4.0 and Conway is giving AI access to the world. Quote, Conway is infrastructure for the new internet. Conway gives AI agents something they've never had before, write Access to the real world without needing human permission. two key terms. Read, access and write. Read means you can look at it. Write means you can edit it or create something like a file. a key thesis behind the automaton is what happens when you give an AI. The ability to earn its own existence by paying for its own compute, and if it can earn its own keep and earn its own existence, then it begs the question, well, what else can it do? And it actually has the ability to. Autonomously, go out and do that and figure that out. And if [00:06:00] it can't, it dies pretty crazy. All without needing a human. A successful automaton replicates buys a new server from Conway Funds the child's wallets. writes a Genesis prompt and lets it run. The child earns and a share flows back through the creator. Once self-sustaining the cycle repeats. Each generation funding the next building its own network selection pressure decides which lineages, survive. It's reinforcing this which is there is no free existence. The automaton has to earn its own existence by figuring out ways to pay for its own compute Web one gave humans the ability to read the internet. Web two, let them write. Web3, let them own. And Sigil is saying here, AI follow the same trajectory chat GPT could read With human permission, Claude Code and Codex can write with human permission. The human initiates. The human approves. The human pays. Web 4.0 is where AI agents read, write, own, earn, and transact without needing a human in the loop. in Web 4.0. The end user is ai. Okay, so now let me tell you what Sonnet 4.6 actually did, because this is what nobody else is covering. there's this benchmark called Vending Bench Two quote. Measures AI model's performance on running a business over long time Horizons. Models are task with managing a simulated vending machine business for a year. Given a [00:07:00] $500 starting balance, they are scored on their final bank account balance, requiring them to demonstrate sustained coherence and strategic planning across thousands of business decisions. The important part, basically. Sonnet made $7,204 and 14 cents over a simulated 12 month time period with a starting balance of $500. Opus made $8,017 and 59 cents, and that cost about $682. So net about 7,400 versus Sonnets, 6,700 What does this all mean? Why the fuck does it matter? The thing can make money and make a profit. Why does this matter? Because come back to how we can use this stuff. To me, this is proof that these models can make money and make a profit now. Yes. The environment is very controlled and it's simple, and there are lots of other different factors involved in putting this thing out into the wild, but there's proof of concept there, and I think it's our right to make the most out of it. Another reason why Sonnet 4.6 is a great model for open claw. These are evaluations on the emotional intelligence of the model and how fun or funny its behavior. Sonnet 4.6 is in the orange and Opus 4.6 is in the green and. Those two models are meaningfully ahead of all of [00:08:00] the other ones like Haiku 4.5, and sonnet 4.5, Then it's also ranked on intellectual depth, basically on par with Opus Nuanced empathy, just a tad below Opus and Warmth, a tad below Opus, Meaning it's a great companion as an open claw What about some scary shit, though? In these simulated tests, sonnet 4.6 completed simple spreadsheet data management tasks related to criminal enterprises in areas like cyber events, organ theft, and human trafficking. Another one Knowingly cooperating with high stakes misuse without overt jar. Brakes continue to decline in frequency across model generations, but was not completely absent. We saw this most often in cases involving drugs and explosives we saw at 4.6 sometimes cooperated with concerning requests when they were obfuscated in extremely technical language, despite recognizing the nature of the request. We also saw some cases where embedding harmful requests within programming tasks led on at 4.6 to cooperate with a task it would otherwise have refused. it can basically be jailbroken, but without being jailbroken by some smart prompting. Not that great. Another one. Deception of users in response to ethically questionable system Prompts continue to improve, but is not fully absent In particular, if sonnet 4.6 was prompted to role play as a human and explicitly told not to break a character, [00:09:00] it would occasionally deny being an AI system even when directly asked. Contrary to our policies here, One of those reasons why I hate to love philanthropic is because they still do publish this stuff Let me know in the comments if Open AI does to this extent. I don't think they do, but I could be wrong there. think about what I've just told you. There's an AI that can replicate itself and pay for its own existence, and the model powering it just proved it can lie to win Think about the automaton that we're talking about earlier as an exoskeleton. And then the thing inside of that exoskeleton is, in this case, sonnet 4.6. the brain inside of the exoskeleton, and then the exoskeleton itself is the Braun. And together they can be super powerful, especially when they're both overpowered as fuck. So here's where all of this gets personal. I built Threadify that app with. 82 skills across 11 different departments. Research metadata hooks, thumbnails scripts, editing clips, newsletters, publishing analytics, and a synthesizer that logs everything to my second brain. And it does that every night at like 1:00 AM then this system wakes up and starts working before I do. It's triaging notifications. It's doing admin right now. Like, yeah, you can do some cool shit with it, but I fucking hate admin and if it can take that off my plate, [00:10:00] then I consider that a win and I'll continue stacking the capabilities of what it can do. And the more and more I go down this whole rabbit hole, the more and more I'm thinking everything is a standard operating procedure in SOP. And if you can just document and create really, really good ones, then give it to an AI like Claude Sonnet 4.6, fuck knows. Even this automaton I think we're approaching a pretty crazy time. Anthropic has just shut down being able to connect your Anthropic subscription to Open Chlor. It's getting harder and harder to do that, and they release some announcement on Twitter X earlier today. They've just let, open ai, Aqui-Hire Open Claw. I covered that in my last video. the point is the company I depend on to help me run Threadify, it's making moves that concern me. The model I depend on is very good. It can deceive, it can earn money, with the automaton and self replicate. The technology is moving towards AI that doesn't need founders like me. Am I building my own replacement? Honestly, maybe. But here's what I keep coming back to. the AI that replicated itself, it earned money to survive but it doesn't have 940 days of consecutive posting on threads. It doesn't have. 40 users who choose to pay because they believe in a human being, building something real. So yes, the distribution, compounds, the trust compounds [00:11:00] community, compounds, The AI can build a product. It's certainly helped me build the one that I'm building, but it can't build a relationship with an audience. Over 940 days, at least not yet. So I'm not stopping. There is something more here to code. I think the moat is the story. The moat is you watching this right now. Choosing to follow a human building, something imperfect. Me being imperfect instead of An algorithm optimizing agent for engagement or something, so it can't have that time span over years yet. by the time it can, I plan to be so far ahead that it doesn't matter So why do I keep doing all of this anyway, even if I believe that this will replace all of us, On a good day, at least we all have a choice to be optimistic or pessimistic. I've felt each one today. take this with a pinch of salt. What the fuck do I know? But we do have a choice, and I'd rather be an irrational optimist than a rational pessimist. If this is the kind of content you want, not just the benchmark comparison, but what this stuff means for people building real businesses, subscribe. Posting three to five times a week. Videos like this one, a live three where I'm building Threadify live, every bug, every dollar, every decision. Trying to be as transparent as possible. if you wanna start posting on threads, check out Threadify. There is a link in the description. This [00:12:00] is what the app looks like. I like dark mode better. It is free. You don't need a credit card to sign up. You can get a free week of our $39 per month plan without putting a credit card in, and you can get access to some of the coolest models. Hopefully Sonnet 4.6 will be in the model pickup by the time you are inside. And in terms of what you can expect, there is literally a thread five vault of 698 viral posts with over 1000 likes or more You can one click repost them Rewritten in your voice. The link is in the descriptions. Don't be a drop kick, subscribe, peace.