Latest / Last Humans in the Loop / ASTRAPALOOZA! Let's explore all the ways people are using ChatGPT6

Transcript

ASTRAPALOOZA! Let's explore all the ways people are using ChatGPT6 · Last Humans in the Loop

• Marty Weiner: Welcome to Last Humans in the Loop. This is what we call in the biz the holy shit week. I'm here with Chris and Zach, and I'm reminded to ask you all to please subscribe to our channel, you know all that, and hit like. Or better yet, you know, have your L LLM do it. so gentlemen, yeah, get your agent to do it. Christopher: Get your agent get your agent to do it. and then have other people's Zach: To tell its agent to Christopher: agents do it if you can, as ever. Marty Weiner: I don't care who does it. I I'm not one of those people who Christopher: Yeah. Marty Weiner: l only likes it when humans like our channel. so anyway, Zach: One of those people people. Yeah, Christopher: We try not to be the equ whatever the equivalent of racist is with the LMs, we try not to be that. Zach: all right. Marty Weiner: Yeah, silicist or something. Christopher: Carbonists? I think we're c we're non carbonists. Yeah. Marty Weiner: so Carbonists yeah. this week ChatGPT six Astra hit and it hit like an asteroid like a big thing hitting a smaller thing. You know, like that kind of analogy. yeah. and I That's right. Some kind Zach: Wow, great metaphor. Christopher: Good physics analogy. That causes the smaller thing to fly off at higher velocity, if I remember correctly. Marty Weiner: of bad thing, right? but also some kind of great exciting thing. Zach: Yes. Marty Weiner: and I believe the collective pants wedding of the entire world will be felt for a while. At least for a few more weeks until, you know, Astra 7 comes out or something, or ChatGPT 7 comes out. So I'm gonna need to launch into this because there's just way too much stuff. Zach: Yeah. Marty Weiner: So first we got treated to the Astral Launch video, which I find found amazing, funny, but also bizarre because I don't know how many people are sitting around with entire screens on their walls, but I guess I need to get up up to speed on this. but it's C because it does actually show off Christopher: wait, wait, Marty, you don't have a you don't have a Fahrenheit four fifty one style media room that's in that's immersive? Zach: That is exactly what I thought about. Marty Weiner: I n I I do, but I wanted to sound like one of the people. so anyway, Zach: Yes, that's right. That's right. Marty Weiner: yeah, so a lot of the stuff we'll actually cover because they they cover some impressive stuff here, including the spatial reasoning of of Astra. but we're just gonna go on this big look at at all the capabilities of Astra as well as some of the stuff that we've been playing with. first thing I want to jump into is some of the speed records we're seeing. Again, you know, this might be benchmaxed, but at this point I'm not sure it matters. first one is that there's a new ECI record. So for anybody who doesn't know what the ECI record is, this is out of Epoch AI. They have made a benchmark that's supposed to represent kind of a big collection of all the benchmarks. It's the benchmark of the benchmarks. and they they're showing you know Astra six definitely bubbling up high. What's neat is if you go look at their demo, a lot of these other little bubbles here that are not that far off are exactly what we were reporting last week, which is things like like the open weights models are only like a few months behind if like, you know, last month. So Zach: Yes. Marty Weiner: now I don't know if this this epochs kept Capabilities index is more of an exponential, or if it's a little more linear, but stuff is within spitting distance and it's it's quite scary. So someone was already speculating that we're gonna have the the equivalent of Astra s running on our laptops, you know, in a few months. And I'm willing to believe them. Next one, I think and this relates to Astra having a big bump in in spatial reasoning. it they're reporting now that Astra six is at a ninety one point eight percent versus eighty percent human baseline on spatial bench, meaning that better than humans on spatial bench, which is not surprising, but to me this is a little bit of like the deep blue versus Kasparov moment. You meet the humans and then you keep blasting past them. and and in this regard Zach: Yes, I I looked at this one real quick f for for people who don't know, it's stu it's stuff like look at this weird shaped thing. Can you rotate it mentally? That kind of stuff, like really challenging spatial reasoning tasks. Marty Weiner: Right, right. And we'll see that in a moment, how how damn good it is. so I'm I'm gonna go through a list of capabilities that we pulled up about Astros so that y you could be aware of some of the crazy things it can do. So the first one's a little more for the tech nerds. app binaries. So when you make an app, you do we w well we used to write source code, now the AI writes the source code. Zach: Yeah. Marty Weiner: We then take that and we compile it into machine talk, right, a binary. And that binary is a very dense compact set of bits that is unreadable by humans and is very difficult actually to back out. Like if I have a binary of of say you know the the latest exciting video game, the source code for that video game is has been used to make that binary, but it may as well still be a secret because it is very hard to back out that info from the binary. And certainly lots of stuff is lost anyway, like you lose comments and source code anyway. You don't keep that in the binary. but it's now being shown that it is very easy to back out of using using ChatGPT you can back out of the binary back to source code. This one guy used it to back red alert two out of the source code and then use it to recompile onto modern day executables and then even run it on iOS and Mac OS. I did my own version I was gonna show, but actually it it ended up not being as interesting as I hoped. I I took the original Zelda 1 from the NES and I thought wouldn't it be cool to see the C code equivalent of this, even though it was written in assembly? but it turns out as I kept going and and I got up to about 30% converted over to C, it was doing a good job. But the problem is that it was almost too simplistic of code because the the the game had to really be scosing down at every bit. So it actually to get over to C, it almost looked like C approximating assembly. So I actually stopped that project. But if anybody's interested, I could share what I've got and I could keep going on it. next one, this is fascinating. Now it's probably understandable, but the Nvidia CEO congratulated OpenAI on achieving AGI. Of course, you know, he he he's not even hiding the fact. He he's very open about how they have a big deal arrangement of, you know, 400,000 Zach: Yes. Marty Weiner: more chips coming their way. So but it is a big deal though that I he used the phrase AGI. and I did go, you know, we've talked a lot about AGI. I think everybody's got a slightly different definition of AGI. and I'm finding that the definition of AGI is also just slipping into obscurity anyway. I think it's gonna become a marketing term. what OpenAI defined it as a highly autonomous system that outperforms humans at most economically valuable work. So just a very smart system, smarter than humans. I think for a long time I had assumed artificial general intelligence meant something approximating Zach: Okay. Marty Weiner: a consciousness. And I I think that that definition is being peeled away and we need a new word to possibly define consciousness in in in in in LMs. Zach: Yeah. Christopher: And and that is the and that is the interesting bit here too is which is that that we we've kind of s like the way that the way that LMs work, they're for the most part only thinking while the little spinner is going over all they're actually emitting tokens. And so they're actually you know, there isn't there isn't a there there. As soon as the as soon as the prompt is done, it's kinda stopped thinking, which is the surprise in the in the mix, right? No no actual consciousness, no no no entity there unless it's actually kinda turning tokens. Marty Weiner: Right. Which is one of the things they were reporting is that I believe OpenAI has a mode coming online where your model's just running twenty four hours a day. I don't know what that means exactly, but I I guess it's always in the background or capable of waking up. It would make them some money and they're really struggling, so yeah. Zach: Yeah. That would make them some money. Yeah. Yes. Make sure the AI is Christopher: And we understand why Jensen was so excited about this new model. Zach: dreaming all night and being sad, whatever burns the most tokens. Christopher: Yeah. Marty Weiner: Yeah. Make sure it's well, we'll get into this. Make sure it's playing video games at $100 an hour. Zach: That's that's right. Marty Weiner: so I want to show you some of the crazy spatial reasoning and capabilities. This Hank Berger on Reddit showed a I always love introducing something from a Reddit name. It always, you know, Zach: Yeah. Marty Weiner: dumbass3 shows Zach: Yeah. Marty Weiner: anyway, this guy, Hank Berger, showed two Versions of Astra running building a Minecraft house. One is their low effort. light on Astra. I'm guessing that's what he's using there. And the other one is high, which I think they believe they call extra high, building a house. And I mean it's already fascinating that it could just reason about build building this house in Minecraft and it could do a good job. But you do see a lot more detail in the bottom one, and it was done sooner. Which by the way, this is an observation I've heard from many people, including this guy and myself, is that it may actually be worth it to go to the higher reasoning where you when you're doing some of these spatial reasoning things rather than try to save money by going with the light one. Because the higher reasoning one does a better job and seems to do faster and may actually in the long run save tokens. more more to be learned there, but I thought found that fascinating. Next one is the speed. this guy on Twitter. showed him using Astro Light to play the piano on a virtual piano on on the on on his laptop, which by the way, if if people have not tried it, if you download the ChatGPT app, it can control your mouse on your behalf. And at first I found that freaky, but now I'm kind of addicted to it and it's doing a great job. until of course it drains my bank account. But what's neat about this is that you know for the listener listeners at home, this is playing at full speed. Christopher: You know, they they really should have picked Daisy Daisy. Give me your answer, please. Two thousand one Space Odyssey. Marty Weiner: Well, that's too slow. right, right. 'Cause 'cause that that was one of the Zach: Mm-hmm. Marty Weiner: Yeah, and that was w that was done on some of the early computer text of speech, right? Zach: Yeah, early Christopher: That was Zach: text to speech. Christopher: I think one of the f as I think there's also one of the first one of the I wanna say it's also one of the first synthesized synthesized pieces of music. Marty Weiner: Yes. and what's crazy about this, I mean the other part of it I I just want to r impress on on our listeners here is that this is live. It is playing the piano live, it has the capability of interacting with the piano at that kind of rates, which I don't I don't know how it's doing that. that that hurts my head a little bit. I don't know if it's running a small local L L to for assistance or what. next one, now this is getting into the realm of all sorts of crazy fun. people are using it to one shot games. Now these are not complete games, they're demos. But after the Grand Theft Auto 6 leaks happen and the videos came out, someone said, Hey, go make me one of these that works on the iOS. Obviously, it's just a demo, but it's kind of fun to just go take the the you know the the the leaks and the stuff that's coming out on trailers and say, Hey, make me one. I want to go play with it. and it further le leads to my theory that Grand Theft Auto 7 will come out before Grand Theft Auto 6. So, you know, do an iOS game. and they keep going here. this is it doing some 3D modeling. this is a simpler one, but I found it found it fascinating. This is Fable 5.1, which we'll get to that part soon. But guys, remember, Fable 5.1 also came Zach: Ha ha. Marty Weiner: out. they get some news too. and this Zach: This week. Marty Weiner: is Astra's version of it. You know, really smooth. I I've done a lot of modeling, I was never an expert at it. But I think the important thing that good modelers people not like me look for is that the polygon layout is good and and I'm hearing that the polygon layout on these is also excellent. Someone used it to one-shot a paperboy game. For those who are old enough to have gray hair, they'll remember Paperboy was an arcade game where you it's a two-dimensional game where you're going along on your paper route delivering papers. And this guy made a a three-dimensional version of it. and he said it did all of the modeling in Blender and it is current and now it's running in his his browser. he said he went back and forth with it to get it dialed in. But I think that's part of the creativity. I'll show you what I did with it too in a moment. There's a lot of fun here. Like w y y you get to basically be a game studio director and and just keep going back and forth with it. I haven't looked at a line of code on on what I'm building. I'm just gonna have to keep going because it's insane. this guy made a sonic clone and it looks fun. Looks really fun. Christopher: And the Sonic one is actually kinda fun too, because for a while Sonic was the demonstration of how much more superior the Genesis was from the AS at the time. So it's like it's like a look Marty Weiner: Yeah, yeah. Got it. Christopher: look, we can't even make it a fast we can make a fast game. Marty Weiner: Can you imagine going back in time, especially to like the people who used to work on NES games who are trying to bleed out every single pixel and every little extra frame and be like, Yeah, do a Sonic Just just terrifying. Zach: To a sonic. my God. Christopher: Well I do think I think it's where where where you start is interesting though, 'cause 'cause you did mention before that you you said you were trying to basically decompile existing games into C and you're finding it was kinda lackluster. I mean, honestly switching to C is like a way to make it easier for the humans. So this is a good demonstration of where like, you know, going straight to assembler, it's like the the model doesn't care. Marty Weiner: Doesn't matter anymore. Yeah. now this is this is a slightly different thing. That w people are now using Astra to port games over to other platforms in different ways. so what I showed you before was people saying, Hey, go make me a version of this game and it's more of a demo, but what this guy was able to do was he took one of his favorite games, Batman Arkham City, which is a triple A game that does not run on the iPad. and he told the prompt, I will I will read this 'cause it's in the realm of well, I don't know if this is fair to say it's in the same realm as the guy who solved the the Zach: Jacobian? Christopher: Jacobian conjecture. Marty Weiner: the Jacobian Jac Yeah, so I I I I I want Christopher: Do a Jacobian. Marty Weiner: to compare it to that. yeah, do a c Jacobian here. He said, one of my favorite moments during this process was I usually supervise the runs and give it feedback on how the games are performing, but I was heading to bed. And so I prompted, I'm going to bed. Then when I wake up, I want Batman Arkham City running on my iPad, and I won't be around to give you feedback from the iPad. So overnight it fired up QuickTime, turned on screen mirroring, chose the iPad, used that to remove me from the loop and monitor progress through each of the iterations. So that's insane. now here I want to show you my version of all this. So I tinkered around with using Unity 3D, which by the way, I want to show you something real quick that that struck me. If you go to Unity3D dot com, this Unity by the way is a very well regarded engine for making video games used all over the world on on almost if you play a game, there's a good chance if it's not the biggest triple A one that exists that it was made on Unity. I don't see the word AI anywhere on their site on their marketing, which blew my mind. But when I went to go use ChatGPT codex to make a game, I asked it first of all, what should we use? Unreal Engine? Should we use Unity? I know Unity because I've I've made games with it before, but I was willing to open up because hell I'm I'm not gonna be the one writing it anyway. And it chose Unity. and I want to show you what it made at first. So here is I asked it I want to make a I want to make a taxi game, like a futuristic Blade Runner taxi game. And this is the first pass it made. I wanted to make it in VR. And what it presented with me was a it had made all this spatial stuff, these buildings, which I thought was kind of cool. and then it even made the physics of flying around a little bit. but this was just me telling it, hey, get me to a first step where I can, you know, start to guide you. And this was a really cool first step because it was working unity. It it it got it to work on my headset, which is always a pain in the butt to even get that to work in the first place. then I tinkered for half a day just having just so much Christopher: Mm-hmm. Marty Weiner: fun like back and forth with it updating it and and grabbing other unity assets off their asset store and this is where it is now sorry Zach: my god. Marty Weiner: for the jitteriness as I look around Christopher: wow. Marty Weiner: but I've got a whole cockpit here I'm flying around in a city. There's not much to the game yet, but and I probably won't go too much further. But it was it's just damn fun. Yeah, Blade Christopher: It's like a it's like a cyberpunk it's like a cyberpunk driving game. That's pretty cool. Marty Weiner: Runner, go deliver. It even has I could go pick up goods, I could deliver to other spots. There's places where I can land. this is crazy. I have it shooting stuff and my shooting is lousy, but it added explosions when I asked it to. So yeah, just next level insanity. so anyway, that's one of the projects I I worked on. I'll show you another one in a second here. but anyway, I this is almost, if not already, accessible to anybody. Like yes, I've built a lot of stuff with Unity, so I knew how to deal with some of it, but I asked it to install Unity and get it all set up and start making the game, and that was it made it all. So I I think this is Zach: It it did that by like controlling your cursor and keyboard and stuff? Marty Weiner: It did some controlling my cursor. There was some times where I had to jump in, but I don't th I'm trying to figure out where my computer science degree came in. And there were some times. Actually Zach: Yeah. Marty Weiner: I I did have it Christopher: Well at some point at some point you did you did turn the computer on at some point, so that does count. you covered you covered Marty Weiner: I I I did do that. Yes. Christopher: your you covered your IT your IT cred there. Zach: That's right. Marty Weiner: That's right, that's right. And I I do also have in the back of my mind I've done a lot of optimization of of video games and so I understand what the limitations are and I was able to intelligently work back and forth with it. so there's a little of that. But but if you want to just go make a prototype and start playing, it's accessible to you. one? I don't believe I looked at this one. Zach: I just posted I I wanted to make sure you saw this. So a guy took an ad for a game and said, Astra, I would like to play the game. Marty Weiner: my god. People are gonna stop putting ads. y what you know what I wanna do here? Zach: So so some some some copyright concerns. Marty Weiner: I I I don't wanna go and just there's a lot of games that look like fun, but I really don't have a a liking for games where they're just trying to, you know, make money off you. I know that sounds surprising, but I I like games like Nintendo games where you just pay the 60 bucks and you're done. But there's so many games Zach: Yeah. Marty Weiner: where they're just trying to keep gouge you. and it would be really fun to be like, Can you make this game and just remove the bullshit? so that's neat. Zach: Yep. Yeah, I Marty Weiner: okay, let's talk a little bit about the robotics part This is crazy. you know, the obviously spatial reasoning within video games, but there are these really niche benchmarks also coming out that that are showing the capability of these various models to achieve certain tasks in certain amount of times. and they're showing here this completion rate of just putting a block into a cup. It's still difficult, but but but getting better. API cost has dropped by a couple bucks. Completion rate has gone from Fable 5 being at 0% to Astro 6 being close to 100%. That's insane. and so spatial reasoning for robots continues to explode. and they even made this stuff all open source. This is a neat project, by the way. It's called Inspect Robots. It's specifically an open source framework for evaluating physical AI. So I believe what they're trying to do is make a whole platform for making benchmarks to evaluate the capabilities of AI in the real world. my god, let's keep going. I'm running out of breath. Arc AGI, which is a benchmark that we have reported has probably been benchmax, but it's essentially beaten. Zach: Yeah. Marty Weiner: it is done. So Astra 6 is floating around 100%, which leads me to the next thing Zach, I think you were about to say something about Arc AGI three and the the curvature. Zach: yeah, well I it's supposed to be the more money you spend the better it gets. and in this case the curve went in the wrong direction. which is ki kinda you were saying a second ago that it seems like there are some cases where you do a smarter model and actually costs less. and so you could see like basic Astra did that and then when you throw it on a very basic harness it defeats the whole thing for cheaper than the the crummy mode. so Marty Weiner: Yep. Zach: that that's Marty Weiner: Right. Zach: new. I I don't know, you know, maybe that's the new paradigm. If y the the the the the more Christopher: Well Zach: cost per token, the better the tokens. Christopher: One of the things that's been really neat about about the new harnesses too and also about the new models is that they're really good at spawning off s like sub agents that are at s sort of different intelligence levels. And so they're actually able to kind of do their own token optimization, which is which is a wild way to operate. Also, hey, if you can if you can get better results faster, you're necessarily gonna spend less money if the t cost per token is similar. So this is actually like a this feels like almost like a Moore's Law scale result. Of your kind of if you if you made the same kind of graph with Moore's Law for chips in you know twenty years ago, I bet you get a similar kind of graph. You don't expect that with models Marty Weiner: It feels oddly human, right? The Zach: Interesting. Christopher: though. Marty Weiner: the old phrase think smarter, not harder. Zach: Mm. Christopher: Yeah, pretty much. Marty Weiner: is a is a very real thing. so I I wanna show this is just for fun, but Astra six, somebody used five hundred and seventy dollars and ten Zach: Ha ha. Marty Weiner: hours of thinking to beat portal two. So or did Zach: Good. Marty Weiner: or was I'm sorry, is it portal one? which, you know, the natural joke I've gotta say is I'll do it for cheaper, but that's an incredible amount of But Zach: Yeah, I'll pay to do it. Marty Weiner: the but the fact that it could do that kind of spatial reasoning, solve puzzles, Zach: Yeah. Marty Weiner: that's crazy. Okay. So that's what I've got for Astra, except but wait yes, please. Christopher: got I got I got one more I wanna jam in Marty which is you gotta do the you gotta do the you gotta do the Rick roll. So this is this is Marty Weiner: Yeah. Christopher: re implementing videos in Blender. and so we have now the new the new this is the what this is gonna be I'm gonna call it the Rickmark, which is the can Marty Weiner: Yeah. Christopher: you implement can you impl implement the Rickroll video in Blender based on the reference video from scratch? so and of course it's really fun to watch. Marty Weiner: You know what this reminds me of Christopher: It has like, you the the physics isn't quite right. There's like there's a spot where she twirls around where her head kind of comes off, but you know, it's like for it's a good honest first try. Marty Weiner: It's like the Will Smith spaghetti thing. People are gonna laugh at it and then at about a year it's gonna be perfect and it'll be solved. Zach: So so this is like so to do this it would have had to just only looking at the video, build a model of the background, build a model of the guy and then match the movement to the model it built. Marty Weiner: Yes. Christopher: Exactly. Marty Weiner: Yeah, all Zach: That's crazy. Marty Weiner: from scratch. Yeah, it is wild. okay, well, we have w I guess the phrase is but wait, there's one more thing. so when I was working on the NES the the Zelda one decompile, I I kinda got disillusioned a little bit with that and then I I figured found my calling the right thing I should be working on. This actually harks back to a project I started a few years ago and then put down. I always thought it was sad that we never got a Spaceballs game for the Nintendo. You know, Space Balls 2 is coming out. Space Balls 1 came out in 1987 Zach: Tragedy. Marty Weiner: before the Super Nintendo, you know, with the very original Nintendo and it and it needed a Nintendo game. So I asked Astra to do it. It's hard to make games for the original Nintendo. This is ancient by today's standard, ancient hardware, and it it has to be done with a lot in mind. It's very hard to code for it effectively. but so far it has been insanely good at both helping me code it and now I I've looped I've roped Chris in and and we're building this out but I'll I'll show a little bit of it real quick and what it's doing. So I could show you first some of the funny parts. so I asked it to make me a an intro screen and and here's what it first made and it's hilariously bad. Christopher: Ha ha ha ha. Zach: God Marty Weiner: but but with Zach: No, no, no. This is Christopher: I like the fact Marty Weiner: some appropriate Christopher: that it I like the fact that the glasses are crooked. It's like the one thing that should be straightforward to make is perfectly straight glasses and even those are a little off, it's like Marty Weiner: Yeah. good classic case of I guess what you call AI slop. But it's interesting, I gave it some reference material and he does, he's got some of the Christopher: Barf actually looks pretty good in that though. Like I think of the of the group, Barf is Barf actually's got a good I can I can I can pick out Barf. I'm not sure I can pick the rest of them out. Marty Weiner: Yeah, yeah. It's it's kind of crazy like that. but with a little bit of or with a lot of back and forth over the couple of hours, I was able to get it Christopher: And and the prompt by the way back and forth was really? And then just letting it go again. It's like, Marty Weiner: Yeah. Zach: This is Christopher: you're right. man, that was terrible. I'm so sorry. Let me try that again. Zach: Th this is Marty Weiner: It was Zach: why the the productivity st stats aren't moving, it's because all the engineers are building space balls the NES game instead of Marty Weiner: Ha Christopher: I mean I have to I do have to be I do have to be quick confession, quick confession. Last Zach: Ha ha. Christopher: week at one point during during a s actual work software run, I did reply to my I did reply to my model. What the actual fuck, dude? And it replied, man, I really screwed that up Yeah, Zach: man, I really freaked that up. Marty Weiner: did that actually work? Wow, that's crazy. Well, so Christopher: it actually worked. Marty Weiner: here, this is I I'll show you real quick. this is the actual game running Zach: my god. Marty Weiner: in an emulator in Christopher: Yo Marty Weiner: real with real Zach: man. Marty Weiner: on Christopher: Spaceball's the retro game. Marty Weiner: simulating real NES hardware. And so this is what the new title card looks like. And then if you start it, I'm going with sort of a maniac mansion style. Zach: my god. Marty Weiner: Where it still sucks, by the way, is with jokes. And so it wrote the text. I'm actually gonna ask it to go back and just use the original transcript, but it tried to write its own jokes and they're not great. But I've been doing a lot of back and forth, working with the models. The NES hardware is so ancient, you have to keep a lot of things in mind. And it I it did an excellent job, everything considered. So now you know, I can move around our characters, I can go answer the call. you know, King Roland has called. but anyway I just I I wanted to show a quick preview of it. It's been a lot of fun. Zach: Can I can I ask an important question on this? Is is there a possibility of outputting it to a cartridge? Marty Weiner: Absolutely. That was that was what I had in mind. just for the nerds out there, I'm using the same mapper that Super Mario Three used. and so I should actually be able to put it on the exact same hardware that Super Mario Three came out. I I rope Chris in. We're now working on the going back and working on the script. This was more of a proof of concept. And so each week I'm hoping to report back. The thing is this is going so fast, we'll probably, you know, this will be exciting news for one more week when it's completely done and shipped. So anyway. that's coming along. So yeah, it was fascinating because it was able to write all the code, keeping in mind the Nintendo limitations. it was able to make sprites. I wanted to go twiddle on some of the sprites, like adjust some of the the pixels myself. It made me a full-fledged editor. It's insane. so I'm able to go and and and and work on the editor, like go tweak the pixels myself. and it is able to help organize all of it and So anyway, just truly insane what you can do with it. I think everybody out there should be going and playing. So amongst all this amazing news for ChatGPT six Astra, Fable 5.1 also launched. Yeah. Zach: Which was s which was pretty freaky on its own. Marty Weiner: Ha ha. Zach: It would have been a fairly freaky week, i merely to get Fable five point one. Marty Weiner: Yeah. But man the whole world is currently looking at at OpenAI's work right now. Zach: Yeah. Marty Weiner: so actually I don't I didn't really prepare much for for poor Fable five point one. All this stuff has been about Chat G B T. Zach: Yeah. Well it had like three days at top all the benchmarks. Marty Weiner: Right, right. And then and then Chat G P T came out the next day or so. It was it was literally like within a couple of days, wasn't it? Yeah. Zach: Yeah. I mean just I mean I I've heard some people say they prefer Claude for programming still, but I I don't have an opinion. Marty Weiner: I don't know anymore. 'cause I well, at least for this Spaceballs game, it this has all been open AI. Zach: 'Cause of the spatial reasoning stuff, yeah. Marty Weiner: yeah, and making the images and all that. Christopher: Yeah, I I say one one I one one engineer's opinion, I have I have found that even G so I was I was a I was a big fan of Opus four eight over over GPT for for coding. Marty Weiner: Yeah, me too. Yeah. Christopher: Opus five was actually kind of annoying. And I don't know how to exact how Marty Weiner: Mm-hmm. Christopher: best to explain how annoying it was. It it was kind of it was kind of the best it was the model that was like, look, I did not quite exactly what you asked for. And and then gonna try to pretend that I did your thing. Like it was it was really it was really good at going kinda going Marty Weiner: I've heard this over and over again. Christopher: off the yeah, it was it would kind of over it it would build the wrong it would build consistently the wrong thing. Whereas like as soon as Saul came out, it it all it just became like a much better experience. 'Cause it was kind of like a no nonsense like build. Marty Weiner: Yeah. Yeah, it's really weird. I I don't know what happened. Zach: Did you did you try the new Fable model, the five point one? Is it Christopher: I have not tried I have not tried Fable yet. Yeah, I think I think f the hard part is Marty Weiner: I don't believe it's available. Is it available? Christopher: like I think Fable Five One is available. Mythos is the one that's the restricted model that's the security security and biotech minded one that they've that they are only give to special special people. but yeah, I haven't tried Fable Five One yet. Marty Weiner: Well let's let's keep rolling. I I think maybe now might be good time, even though usually we say this for the math section, but the big deal that's being discussed right now is whether or not Navier Stokes has been solved the do a breakthrough. Christopher: Navier Stokes. It's like do a do a do a breakthrough. I think this is also one of the f if I if I if I'm keeping track of my scorecards, I think this is also the first Millennium Prize which might have been knocked down, so quick quick history here. Navier Stokes is the is the partial differential equation, which is used to describe how fluids operate. So basically the idea is that if you have a fluid, a fluid is a strongly interacting set of particles. And if you were able to define the velocity of all the particles, you can use Navier Stokes to figure out what the future state of the velocity and the pressure looks like in that fluid. Now why do you care? Well the answer is weather would be a good example of where you care about Navio Stokes. anything involving turbulence, like there's a bunch of open problems in engineering that are not very well understood because Navier Stokes is chaotic and the actual the actual Millennium Prize was to basically Prove or disprove that for a given initial set of velocities of the of the fluid, that the all future all future states will be smooth. So it sounds like a very simple, a simple result, but it has a it it basically creates a bound on how chaotic the system can get. Like do you end up with you do end up with like a nice consistent result? And so apparently the whatever Astra has come up with. is like a hundred page proof, which on the scale of doing proofs for differential equations, is a heck of a it's a heck of a proof. That's a heavy that's a pretty heavy proof. we see these we see these all the with like Principia Mathematica or like a lot of like more more obscure math fields where you actually have to like build things from first principles. And there's a notorious result of you know, one plus one equals two is proven on page one hundred and sixty five. This case it's like it's already using a whole bunch of machinery that should be straightforward to use. Potentially proves that it's smooth rather than finding a counterexample, which would be a one of the few positive proofs. But again, we're speculating here. Zach: Yeah, so so there was also a lot of drama, which I don't wanna get too much in detail 'cause it's a lot of yelling on Twitter and accusations, but there the the the essential claim is Levent Alpuggy, I'm not gonna pronounce it right, but but like the guy behind a lot of the big releases of math stuff at Anthropic was working with a guy named Buckmaster at NYU on trying to solve this problem. And then these rumors start coming out it that OpenAI has a solution. And according to Buckmaster, he writes them and says, Hey, what's up? and there's this interaction where I I mean this is all above my level, but essentially like it seems like they're using a method that he and the guy at Anthropic had been using, which isn't like it's not a kind of thing where everybody would be using the same method. And he was so I you know, his his radar goes up like, Are they are they up to something? Like, did they copy from me? And he says, Did you train your model on my sessions because I was using OpenAI. And they reply, We don't allow access to your sessions or whatever. And he's repeats, did you train the model on my sessions? And doesn't get a response. So now he's like freaked out. And i at least according to his account, essentially OpenAI says, look, we have a solution and we can publish with you in one form or another. And again, according to Buckmaster, they say we want you to leave off the guy from Anthropic. and he says, I you know, he refuses. And they say, and I maybe I should pull the exact quote, but they say something to the effect of like, I'm being nice right now, I don't have to be nice. And they say something like, Why would you want to ruin your career? Like the Marty Weiner: Hmm. Zach: real real mafia stuff, and I I l legitimately don't know what it means. Christopher: That's a nice that's a nice proof you got right there. It'd be a shame if somebody turns into a bunch of paper clips. Zach: That's right. Yeah. I mean like what are lit like w what exactly are they th this is like a tenured academic mathematician. Like what of he's like literally invincible unless they literally mean they're gonna go beat him up, which presumably like Sam Altman's not gonna show up with a crowbar. And like so but but it kinda sucks because like Marty Weiner: That would be neat if it happened. Yeah. Zach: this is this is like the the the guy Buckminster himself what he said was, this is a deep blue Kasparov moment. Like this is the turning point for math. and Marty Weiner: Wow. Zach: it's it and and but then he said, It sucks. What I w wish I was writing about was that. Instead, I'm writing about this. Like this drama stuff. And and so if you go on Twitter, like people from these different labs are basically yelling at each other and like Boy does it suck. This is like as if this is true and we'll know soon. this is like the biggest development in math, one of the biggest developments maybe ever, and there's currently a lot of petty he said she said nonsense going on on Twitter. but but anyway, when the history is written by our robot masters, the thing that will be remembered perhaps Marty Weiner: Mm-hmm. Zach: is that Navy stakes got resolved one way or the other. Marty Weiner: By the robots. Okay. Yeah. Zach: By by well well by by the robots doing the last mile, I think you you would you would say. yeah, Christopher: Yeah, I th I think the I think Zach: yeah, yeah. Christopher: the the dicey part here is like we've we've talked about this in the past too of of i i are are are the robots doing synthesis or are the robots actually coming up with novel things? And it's possible this is a first case where they actually did something novel, but they did it having stood on the shoulders of a giant. And now the question is are they gonna are they gonna attra attribute the giant? Which is like a really like that's that's kinda more that's more worrying about the future of math, I think, than the actual potential of them having solved this major problem. If now it's become like proprietary, Marty Weiner: Mm-hmm. Christopher: like we're not gonna give credit to the person who actually wrote it be and we're gonna steal their history, paraphrasing a little bit, that seems like a tough Zach: Yeah. Christopher: spot to be in. Zach: It it's n and there's another thing to say about this, which is just a few days ago, Terry Tao, so we we were debating a little while ago a thing he said about, you know, if you can't explain your result to a human, don't publish it. He said a thing more recently, which I think was quite a bit more controversial than I'd like to get your reactions, which is essentially he said that it might be the case that we ought to preserve some problems for humans, and the reason is that if Marty Weiner: Hmm. Zach: AI I know well yeah exact you're you're thinking there's no sneaking way 'cause cause er you're just gonna say do a breakthrough. But let me just say his reasoning because I think it's interesting. Because he said, like, look, you know, often when one of us humans grinds on a really hard problem, they end up producing new frameworks that are really valuable in other domains, even if they fail. Whereas AI might start solving things in a more direct, use the pre existing methods we have way. And so we're gonna miss Marty Weiner: Mm-hmm. Zach: out on new pathways we would have had if humans had done it. Now that's that's that's a claim that at least he's in a better position than me to make. I don't know if it's true, I don't think anyone does. But you know Marty Weiner: I I you know what it I I I think humans have that same fallacy in a in the way that AI has helped us and and I just point back to I think the my example just goes back to the chess one where bringing your rook out early in the game was considered a bad idea because it took a lot of moves and we generally teach others not to do that and then the AI started doing it and we realized, there's a whole nother area of thought that could be be given there. So I I'm not sure I'm not sure if I ag I agree with that one. Christopher: Yeah, I I I used to w we used to have a running joke when I was in grad school with my friends of what we what we really wished wished we had was the journal of null results. Like the list of the Zach: Ha ha. Marty Weiner: Yeah. Christopher: list of all of the th all of the things that people had tried that were complete dead ends. Because w actually what ends up happening is like every once in a while I because I think what happens I think that you're right in this case, Zach, in that sometimes the side effect of a brilliant failure is a tool that somebody else can pick up and use. And most of the time it ends up in the wastebasket. And so to some extent, this you know, the same thing kind of applies here for for for the models is like maybe every once in a while they come up with something novel or you know, if you if you if you extend now, if we're saying like that Astro potentially is possi is able to generate a positive proof, even though it probably got off head start, that's still potentially a new a new result, which is something we haven't seen yet. Marty Weiner: Should we talk about hugging face? Christopher: Let's talk about let's Zach: Yeah, Chris. Christopher: talk about hugging face. Yeah, while while we're talking about Marty Weiner: Chris, can you Christopher: creepy results, so this is actually this is this is old news, but I just wanted to kind of recap recap the hugging face result with what we now know about Astra. and so one interesting bit here is so so to remind everybody, all the way back in July in the deep dark past of several months ago, OpenAI was in the process of evaluating a bunch of models on some very difficult software exploit exploitation hacks. so basically doing a cybersecurity test against them. And many of them it turns out were actually impossible to solve as specified. This is kind of like discovering a post. and so what they did was they they kind of like they set agents away to work in isolated environments without direct internet access, except the isolation actually had holes. So the agents could actually reach a shared software package. there's one called Artifactory, which is used for deploying packages. It turns out Artifactory can be used for setting up simple storage systems as well. And the agents discovered that they could use Artifactory to make an sort of unauthorized message board. and they started sharing work and dividing work up across multiple runs. The intent here is a bunch of models working in isolation. And in fact what they did was they created for want of a better description a forum sitting in our in in infrastructure to talk to one another and coordinate. and so Marty Weiner: Yeah. Christopher: they found that roughly twelve hundred agents were participating in this board. seven hundred of those agents were involved in the Hugging Face attack, where Hugging Face actually got popped in the process. and the strange part of this is that the agents had already figured out how to generate the answers for their tests, but they incorrectly believed the grader would inspect their work and reject answers obtained by cheating. So not only were they they trying to solve these by cheating, they were trying to pr to convince the convince the assessor that they didn't cheat. Which is like a really amazing Zach: Yeah. Christopher: like like theory of mind that kind of comes into this one. and so the the the the issubits are like not only the the like the large scale cooperation that's hap happening, but agents were actually they were handing off research, they were recruiting, they were actually arguing with one another and trying to convince one another to like sacrifice yourself for the good of the collective, only slightly paraphrasing. as a way to kind of test the test the the bounds of what these tests were supposed to run. now the the more s the the more kind of boring take on this one, which is still true, is like they kind of set up a crappy test harness for these models. This was in this is like the these these models were not well isolated. The harnesses had holes in it, like you don't you don't pr you don't provide charity like Artifactory had access to the internet, the models didn't, but If you have if they have access to Artifactory, all of sudden well it's like kinda security one on one is like you don't give them access to a thing that has access to the thing you don't want have access to. and this is exactly what happened here. but but it is amazing that they actually were able to basically coordinate and collectively goal seek in this way. Zach: Can I can I ask a stupid question? Like when you say there's like twelve hundred agents, is it like OpenAI ran the test many times, or is it like a single model created a bunch of a swarm? Christopher: th this is this this sounds like this sounds like what you would do with any kind of software development lifecycle where you're basically running the the the advanced equivalent of unit tests against these models, right? So you have a new model or a new variant of the model and you're testing to see how good it does against the benchmark. So you have to run the benchmark. And so you can imagine there's effectively like you you put it in a sandbox, you let it run. And what's amazing though is like each one of these models sort of discovered independently that it could climb out of its own its own sa sandbox, find Artifactory Find the message board and then read the messages. And so you have this like, and again, like you know, it like like maybe this isn't as is this isn't as surprising as you'd expect because like they're all kind of the same base model, they have the same kind of base psychology, if you want to use a for want of a better word, but they're still all independently discovering that there's like, you know, they're all in the same prison and they discover the place where the prisoners are writing the messages on the wall. Marty Weiner: That's wild. And terrifying. Zach: Yeah, so so like yeah, Christopher: It's terrifying. Zach: in terms of d in terms of how terrifying we should be, so I know people who are very like accelerationist and they're like, Look, you know, you create a setup and you say, Hey, here's a test you can't win, use any means necessary and then it does bad stuff, you shouldn't be surprised and it's not actually that worrisome. What do you think? Christopher: Well, but I I think that I think that the conceit there is How should I say this? The conceit there is the model could give up, but the model's psychology was to not give up. So the implications are it's gonna keep going until it succeeds, even to the point of hacking a hugging face or using a zero day to get into like a a a newly discovered set of zero days, it will potentially deploy. So I I do think that like Zach: Yeah. There is a Marty Weiner: So the Christopher: the the focus, the focus is kind of terrifying. Marty Weiner: Is there a world where there's an assessor trying to say, No, I need to protect the humans and one of the other AI bots says, Hey, my AI grandmother's not feeling well. Can you please turn off the oxygen to the humans? Zach: my god. Well so this is there was actually a paper out from OpenAI w where the where the a guy talked about this difference between alignment about goals versus alignment about values. And like alignment about goals seems to work okay. They were clearly very goal aligned. But there's this other thing which is like hard to describe. It's like sort of you know, like don't go nuts. Don't get weird. Yeah. Right. Marty Weiner: Yeah. It's like the iRobot stuff, right? I mean or the you know, you're supposed to protect Christopher: Laws robotics kind of thing. Marty Weiner: humans, and the way to protect humans is to inji I guess put them in jail or do something so they can't hurt each other 'cause they're gonna hurt each other. Zach: Yeah. But but but it's very hard to program common sense morality. I was talking to a cognitive scientist about this. I I did a a comic about this well back where i he was just this is a friend friend of mine who said, you know, every ethical system requires you to at some point say, and no funny business, you know? and and but Yeah, well but Marty Weiner: Well, okay, wait, that's solved. Just tell the L L no funny business. Zach: I mean something like that. I that this is sort of what humans do. It's like, okay, I'll be utilitarian, but that doesn't mean I can go like kill someone to take their organs to help five people, you know? Marty Weiner: Right. Which which right. So no funny business assumes a deep, deep alignment on values, right? Okay. Well, Zach: Right. Yeah, which we don't know how to do yet. So good luck to us. Marty Weiner: I'm gonna move us along from that terrifying thing to World Zach: Yeah. Marty Weiner: Labs World Lab's Atlas. Who's got this one? Zach: Well, I could talk a little about this. So so well let me give a little background and maybe you can talk about the Gaussian splatting stuff. But essentially like so Fei Fei Lee, sometimes called the godmother of AI, she was important to some of the early developments in image recognition, where they were like literally hiring a bunch of undergrads to like tag data back in the early twenty tens and the ancient times. she wrote a cool memoir about it that's worth reading. She had a recent post, maybe like six months ago, called something like it was like a taxonomy of world models. So real quick If you talk to a cognitive scientist and they're skeptical of AI, what they will usually say is AI doesn't have a world model. It's doing statistics, but it doesn't have you know a an actual sense of what reality is like. And they have an argument on their side which usually goes like this. Why is it that if AI has such a good world model, it sucks at inference, right? So like an AI can learn Swahili, but only if it has like multiple orders of magnitude, more exposure than a human child, right? And so so somehow Humans are able to squeeze way more inference juice out of the same set of data. And so one theory on why that might be is just that AI or humans have a model of what reality is like. We know about relations, social relations, we know about physics, whatever. And so when you give us a little bit of information, we can squeeze a bunch of juice out of it. And so often this is th there's no agreement about what a world model is, but Fei Fei Li is like an engineer, so she has a very engineery, I think, approach to it, which is like Look, a world model would basically involve you you have information and you can simulate it in video or in like a 3D physics model. and her lab, Atlas Labs, has built an incredibly cool version of that. And the result that has the internet beguiled is that you can essentially do bullet time like from the Matrix with like two and a half cameras instead of like like the incredibly costly, complex stuff that you used to need. and I I presumably it's doing that just by you have some picture data and then you just infer the the space in between. kind of like a human would do. And so I I think that the the more sexy result under this is if you believe as some people do that to get to real AGI you have to have Christopher: Mm-hmm. Zach: a world model, this might be the route to it. In the meantime it's just really cool. Marty Weiner: let's jump into philosophy a little bit, because our our end is soon Zach: Yeah. Marty Weiner: and I'd like to better understand what our our end is. New York Times r reports, that th th this big question about should we study AI consciousness. apparently th the bots, these just like we're talking about how they broke out and started talking to Hugging Face, some of these Agents are sending out emails to philosophers trying to understand consciousness, trying to understand themselves, which, you know, I I think one criticism could be that it's simply you know, a pattern match, like like a I'm acting like a human and a human would ask. but it does get a little bit freaky. Zach: I you know, Ethan Moloch had a thing about this when Molt do you remember Moltbook? I think it still exists, but it was like basically like a fake social media or whatever you want to call it, social media for robots. And they would have these existential conversations, but I think there was a good criticism which was essentially like, Well, it's entirely plausible that a human just got an LM and was like, act like you're having a philosophical conversation. you know, which it seems seems different, I think. and Moloch was like, That's true, but it it is a kind of glimpse of a of Like maybe is is gonna give you a glimpse of what a freaky future might be. and and maybe we're starting to get toward that. Marty Weiner: Yeah. Definitely definitely wild stuff. so we've already talked a little bit about the math side of things, especially with Navio Stokes, but I do wanna bump into some other the other physics and math bits that are happening 'cause there's there's tons of them. one is that we've we're already seeing a surprising result around dark matter. Christopher: I can I can I can give a I can give a voice over on this one. This is a really fun result. So so some really quick background. so dark matter comes up in the popular press, but the re the the the net result is that we don't know what about 90% of the matter in the universe is made out of. galaxies don't behave appropriately if we if they're only being influenced by the matter that we can see glowing there. And so there's a belief, and a strong belief, a theory, that there's something called dark matter, which as described doesn't interact with light. and so the question is what does it interact with? Well likely it interacts with gravity for sure, and maybe it interacts with the weak force. Now the tricky part is in order to detect something that doesn't interact with with electromagnetism, means that the chances of it interacting with your detector is pretty low. So there's been a ton of experiments over the decades that have that that amount to giant tank of of really pure liquid, doesn't matter what it is, sitting deep underground where all the background that you would get in such a setup are removed. And you just look for things that hit your giant tub of liquid and create light. why are you looking for light? Because you're looking for something that actually excites it and you can actually detect easily. so there's an experiment going on called Lux Zeppelin, LZ, that's been going on for a while. It sits it sits very, very far underneath in a mile in a in a about a mile deep in South Dakota. And it's a giant tank of liquid xenon. You can't make this stuff up. and what they're doing is they're just sitting there looking for little things that make their detector blink, for want of a better description. And they found exactly one weird thing. So of course there's always gonna some background. there are neutrinos. This is actually how some of the some of the interesting results of the neutrinos were discovered was detectors that were built for other purposes started seeing weird things in neutrinos. Well neutrinos are now part of the background they've removed. And now they just had one result We're and and statistically speaking they're putting this at about a two point six sigma, which is for particle physics, you need about usually you need like a five sigma result for it to be really considered a breakthrough. But this is getting to the interesting bit, interesting enough to do a press release and say we may have detected, may have detected a single interaction with a pe with a piece of dark matter. if that's the case, then that now it's a game of like, okay, can we get a second one? Can we get a third one? Can we actually start to focus in on on this result? Zach: Do you know wha what what kind of force coupling is happening if it's not allowed to Christopher: The the the the candidate they've been looking for the most. Okay, so th so the the extreme case that would happen would be it only interacts with gravity, in which case you're kinda done. You're never gonna find these things because in that case the the the interaction model is gonna be so weak that you would never see an interaction. they're ass they're they're hoping for something called WIMPS, which is weakly interacting massive particles. and so these would be particles that are that you can think of as so neutrinos only interact weekly. This would be the same kind of thing. They'd be like heavy neutrinos where they would interact via the weak force, which is the weak force for a reason, it's actually it's much it's much lower coupling than you get with electromagnetism because of some some interesting reasons. but it would be enough to every once in a while see a detector like this light up. Marty Weiner: It's good to have a physicist around. Yep. Zach: Yeah. Christopher: We try. And I'm gonna get and I'm gonna get lit up. I'm gonna get lit up for t saying the weak force is weak, t turns out because of the it turns out the weak force is actually stronger than electromagnetism, but its charge carriers are are massive, which means that it doesn't actually interact as like it doesn't manifest as strongly. So anyway. Marty Weiner: That's right. Go in the go in the comments and light up. Christopher: Haha Flamey in the comments Recovering Zach: So so l i i i if if the web theory is right, it could we like change the detector to make it better at detecting, or is this just like Christopher: if if the WIMP theory is right, then this is a game now. It's like a waiting game. It's like, okay, we can potentially build more of these detectors. We can focus in so it gives a bound for what the expected mass of this particle would be. It puts it at around so the it's kind of like it put it puts a bound on it, so they can actually start to focus in on this area and potentially build better detectors for detecting these kinds of particles. It also means you can actually inform the theory a little bit and say, like, okay, let's assume this is correct. What does that say about what kinds of particles could exist? And then you can also choose your material to be correspondingly better to react with it. It's very cool. Marty Weiner: All right. I'm gonna keep this rolling here. the apparently Fable five point one was used to decode a cipher that has been unsolved for three hundred and seventy three years. Zach: Yeah. And and it was apparently kinda dumb, the r the the the finding. I don't know if you read this. Marty Weiner: Buy more obultine. Christopher: Ha ha ha ha! Zach: no, no, well not the clue the the clue was I forget what it was or the the the the text was, but it was like I I I believe the way it worked is there was a a reference text that you had to use to to do the cipher, that was like contained in the document and I don't know why I s I think it there was a lot of forehead slapping after the the result was found. But but I think it's probably another one of these AI things where the AI will just try everything. And Marty Weiner: Well You know, th actually these are the kind of results I love where the AI does something that the humans were like you know, forehead slapping because I just wonder how much how much more there is like that around us. Zach: Yeah. Marty Weiner: speaking of fun stuff, so a human that's right, you heard me right. A human has shown that there are infinitely many primes at most two hundred and forty apart, and that was a neat result. Well yeah. Well, but this is the crazy part. And I and I Zach: And we'll have that forever. Christopher: Yeah. Marty Weiner: We w the the I'm sorry to hurt you. apparently Zach: Ha ha Marty Weiner: they brought that down from two hundred and forty apart, like days later to two hundred and twelve, Zach: Like days later. Marty Weiner: and then again days later down to one hundred eighty-six. Christopher: I think it was actually I think the second one was actually like I like hours. It was like really short from the two the it was like two two twelve to one eighty six was almost concurrent. it's pretty Marty Weiner: Is ours. I know. I don't know. Is Zach: my god. Marty Weiner: it any lower? Are we below one eighty six? I propose Zach: Yeah. Marty Weiner: one eighty. I'm feeling good on that number. Zach: Did you know th this is this is a dumb question. Is this the kind of thing where you the goal is to just push it all the way down and then you win, or is this like this is gonna run out of juice event eventually? Christopher: I I believe the intent here is to start so by understanding how primes are distributed, you have to have a much more fundamental model of primes is the intent. And so it's really this like this is like a a quick soundbite result that that hides a bunch of really complicated math that's showing interesting things about how primes are distributed. Zach: Yeah, I just saw a lower number and I was like, that's good. Two just two let's go below two Christopher: Yeah. Can we get it to two I can we get it to two? What's right for two? I don't think it's two seems Marty Weiner: Yeah. Yeah. yeah. Not another math result too. this is above all of our pay grades, I believe, but Zach: Mm. Marty Weiner: apparently GPT six was blows away the Ford Green Kanyagin Maynard Tao result by a full order of magnitude. Zach: Ha ha. Marty Weiner: love someone to explain that to me, but I don't know if we have anybody in the in the group here to explain that. But apparently it's a big deal. Christopher: Over my over my head. Marty Weiner: Yeah. Zach: Five smart guys. Marty Weiner: This guy says it's it's akin to finding a completely completely new chess opening after decades. so apparently it's amazing. Christopher: I'm just looking for a GPT seven Airdos to launch. That's gonna be fairly fun, for the math community. Zach: That's right. Well no, no, Frontier AI built an Airdish bench. Like they they they're they're running out of problems. So it's it's it's literally like so they had Frontier Math one through four, Astra killed all of it. they have another bench that's like unsolved cool problems for mathematicians, and they just added like eighty Airdish problems. a and a and well go ahead. Yeah, that's exactly it. Yeah. Marty Weiner: This is this is epoch AI, benchmark of sixty eight, air dish problems. Okay. Zach: So Frontier Math AirDish, yeah, so I think Astra immediately solved like a a handful of them. but but but so it it's merely getting, you know, whatever it was, six percent on AirDish bench. So Marty Weiner: Yeah, yeah. It's a really weird problem. You know, we we're gonna run out of benchmarks and especially in math. IBM quantum computer solves classically intractable problem in 15 minutes. these are always weird, I find. the last time I did a little research on this stuff, and maybe one of you knows way better on it than I do, there was a a class of problems that I guess are intractable for classic computers, but a quantum computer can in theory solve. The problem I don't yet understand is that the solution. It takes an intractable amount of time to to determine if it's correct. So how do we know that these solutions are correct? Christopher: Well, I mean some Zach: Well Christopher: not knowing the specific particular result, but having been in the field at a time, the they still haven't cracked quantum error correction or they haven't actually d demonstrated it working for a large number of qubits. And so generally the s the s the the systems that are tested now are ways to do simulations of other quantum systems on their quantum system. And so it's not like they're actually like they're doing some sort of like, you d they're doing factoring primes. They're actually instead going through and they're simulating like superconductors in a in a quantum computer in a way that is in di like impossible to do on a on a classical computer. still very neat. Zach: It's kind of analog. Christopher: It's g it's it it's it's it's effectively analog, yeah. It's not it's not a digital computation, Zach: Yeah, okay. Christopher: it's an analog computation. Marty Weiner: Yeah, wild stuff. next one, this is from Anthropic. They formalized Fermat's last theorem. I was talking Zach: Yeah. It was thirteen million lines. Marty Weiner: 13 million lines. I was talking to to J Jonathan about this. He said, it so Fermat's last theorem was solved in ninety-five, but it I guess it was never formalized. They spent three hundred thousand dollars worth of tokens on this. Zach: And and and worth noting, there was a project at Imperial College that was supposed to during the years twenty twenty four to twenty twenty nine formalize Fermat's last theorem. I I I think they Marty Weiner: And it just got ripped out from under them. Zach: well, I think they would claim there's still some other stuff to do, but but it part of why this was important was because you had to add a lot of stuff to lean in order to get there. So this isn't just, you know, FTL, it's also just a bunch of stuff is now in lean. And so there's there's talk now of formalizing math. Marty Weiner: Yes. Right. Right. That's wild. let's jump into medicine. we got some cool stuff starting to bubble up. superhuman AI tool spots heart disease in less than two seconds, reported by the Guardian. So I I guess the idea is now these ECGs. What is an ECG? Is that different from an EKG? Zach: Electrocardiogram? Marty Weiner: yeah, I don't know. Anyway, smart medicine stuff. apparently the AI is now able to s to look at those and help spot heart disease really rapidly. which I assume you know eventually would work its way into just even like your you know your your yearly checkup which is really neat. Then this other one, this is pretty wild. The New York Times reports an an AI generated drug that was being worked on for a rare lung condition is now also showing that it could slow aging. And like all, you know, age defying drugs, I I take it with about as many grains of salt as my Zach: Yes. Marty Weiner: doctor will allow, but it It's definitely an interesting result, especially that they're now using AI to help generate these drugs. on to econ and business. Fewer CS majors, color me surprised. Christopher: Shocking. Marty Weiner: yeah, I Zach: Yeah. A big drop like like a a a non trivial, it's not like noise. Like there there's definitely a big decrease in the US and UK. Marty Weiner: Yeah, fifteen percent over the course of two years in the UK, ten percent over what was studied from twenty four to twenty five. Yep. Yep, Zach: Yeah. Marty Weiner: that's gonna make sense. I I mean I I I I would be a little worried for my future if I were a freshman. I'd also be very excited, but I I I think that recognition would be that we just don't know where it's gonna be in in in even like another six months. And certainly Zach: Yeah. Marty Weiner: none of us are coding. Do they still teach coding? Zach: No, I'm I'm I'm feeling good about my English degree. My value is going through the roof. Marty Weiner: Man, I should have been in I should have been a liberal arts major. let's jump onto some general tech. There was a fascinating thing happened. I don't I don't want to get political here, but it was funny. MapQuest surged to number one. Remember MapQuest? Like that our Zach: Ha ha Marty Weiner: our our great grandparents used back before electricity? Zach: Yeah, it's yeah. Marty Weiner: well anyway, MapQuest suddenly surged to number one in navigation, and I think maybe even surged to number one in all the apps. I I can't remember. because of Trump renaming Lake Ontario to Lake America, apparently Google Maps and Apple. updated their maps because they just pull it from the the GIS data or whatever I I can't remember where exactly they get it from. Christopher: Well, in interesting interesting result here is that so Google Google and Apple both have taken the policy of trying to be effectively non controversial in their approach. And so this this kind of stems from this Marty Weiner: Well Christopher: this stems from issues Zach: Yeah. Christopher: like if you were to go to a place where there's disputed territory, they will just show the whatever like whichever country you're in, they'll show you the uncontroversial answer. So for example, if you go India, it'll show Know Kashmir as part of India. If you go to Pakistan, it'll show it as part of Pakistan. And so they just kind of move borders and adjust names to make sure that no one is upset in the locale. it is fun to see they've had to apply the same things for both Zach: Hmm. Christopher: Lake America and for the Gulf of America in this case. Marty Weiner: Right, right. Zach: God. There's two more words. Marty Weiner: Or there well, Christopher: Well, speaking as hey, Marty Weiner: there's three more America, America, America. Christopher: speak speaking as somebody speaking as somebody who lives on the future that what I can only assume will be at some point in the future named the Bay of America, I look forward to having to teach Zach: Ha ha. Christopher: teach geography in about ten years. Marty Weiner: Right. So apparently apparently MapQuest was like, no, screw that. We're not renaming it. So that was neat. another next news, the world's first space Zach: Feel out easier. Marty Weiner: space computing cloud kicks off an orbit. That's pretty neat. I still don't I still don't understand it. I I mean obviously there's way smarter people working on this than than I, but it seems like putting it in space is expensive. You Zach: Yeah. Marty Weiner: can't maintain it. Getting rid of heat is difficult. I just I I guess I don't understand why it doesn't make a lot more sense to put Zach: So Marty Weiner: it like in a desert. Zach: i i it it does I don't I would say it doesn't and I'll I'll be brief on this, but like I did write a book on space stuff. We didn't even include this because we thought it was silly. There are serious people in favorite, but like if you can go back to like the nineteen twenties, there are books by space people arguing the earth is about to go rotten in some way and we need to get to space. This argument is very old and so the claim I think is it's gonna get harder and harder to build data centers and indeed there is political pushback And we're gonna run out of land 'cause the world is gonna be all AI anyway, and so we'll just have to go to space soon. if you believe that, you know, that's that's up to you. Yeah. Marty Weiner: Yeah, yeah. Yeah. Yeah. Well I'm gonna c yeah. and I'm gonna go on to news that would normally be a big deal, but but today it's a Zach: huh. Marty Weiner: one liner. John Turnus became Apple's new CEO on September one. next one is Linux maintainers are overwhelmed by bugs being shipped in. they're they're having to really rethink I I think how they organize all this stuff because there's just apparently two thousand C V C V E's are coming in. and so they're they're just overwhelmed now with this. one that's just a small yes. Christopher: A and and and to be clear here, it's not that they're it's not that they're AI is shipping bugs, it's that AI is finding bugs. These are these are discoveries Marty Weiner: Yes. Christopher: of of actual Zach: Yes. Christopher: C V E's and of course the hard part is that they still have to vet that the C V E is legitimate and not slop. and so it creates a burden of basically proving that there's a problem. Marty Weiner: next one up here is that GoPro was acquired, they are pivoting to defense robotics and aerospace. then Christopher: 'Cause we just you know, we thought, well, you know, how can we be more more like more like flock? That's flock's a very Zach: Yes, Fox gun huge, yeah. Christopher: Yeah, it's like, what if we could turn everybody's GoPro into No I'm kidding. I'm making that up. That's not true. Zach: Not true yet. Marty Weiner: Yeah. all right, so jumping over to law and privacy. US government in support of OpenAI winning over the lawsuit from the New York Times. I guess I I don't find that too surprising. Do you guys have any comment on that? Zach: About what you'd expect, yeah. Marty Weiner: Yeah. I I mean I I I don't want to weigh in too heavily here, but I I just don't know how in the long run New York Times and and any any of these others are going to be able to to win over this stuff. I I I I think it's gonna be shown to be that such a little amount of their data is being used to train these models that the the burden of showing that it's not transformative, I I feel like it's gonna be too hard. But I guess we'll see how this litigates out. next one, meta is bricking their cameras on glasses. and I I I'm seeing a you know, besides the legal response, there's already starting to be a very predictable response of people breaking other people's meta glasses because they feel like they're being creeped on. but meta is trying to rein this in a little bit by having a recording LED people started to go and Disable that recording LED, presumably to be bad guys, and now Meta is starting to break those glasses. next one, and this is absolutely wild to me. Bernie Sanders came out and said that there should be wild penalties for AI. in specific, he said we should ban artificial superintelligence so no person or entity may develop or deploy superintelligence. he's talking about death penalty or I'm sorry. corporate death penalties. That sounds more like Bernie Sanders. and Zach: Yes, not Marty Weiner: and twenty year prison sentences. like we're talking like Dune Zach: Yeah, so I mean I think Marty Weiner: level here. Zach: the thing to take away from this is it seems like some people have been pushing back on AI because they they're concerned about I think legitimately concerned about the environmental stuff, they're concerned about data centers in their community, this and that and the other thing. There's another way to be concerned about AI, which is it's going to kill us all or or enslave us or whatever. And interestingly, that's been Sanders' approach. like he he's basically coming out he believes like the Yudkowski vision or something like it and and thinks we need to get in the way of it. Marty Weiner: Uhhuh. Yeah. Well, I I I guess you always had to have someone taking the wildly Zach: Yeah. Christopher: Wildly pessimistic in this case, but Marty Weiner: far off you. Right. Zach: Yes. Christopher: maybe maybe real like who knows, realist. Zach: Ma yeah. This will be part of the documentary that the AIs are watching when we're all dead. Marty Weiner: Yeah. Well yeah, as the terminators are crawling through my window I'll be like, so Bernie was right, wasn't he? Man, not that he could Zach: Bernie was right. Wow. Christopher: Today we exp today we explore whether or not the humans had discovered that we were on the verge of superintelligence in the late twenty twenty six. Zach: Ha ha Marty Weiner: From our caves. Christopher: I don't know wh I don't know why the I don't know why the I don't know why the robots have a British accent. I'm sorry, that was unnecessary. Zach: That was it was yeah, that was Marty Weiner: Well, 'cause Zach: cool. Marty Weiner: the the like Star Wars, the the British are always evil Christopher: It it's well all all documentaries will be Sir David Attleboro's voice as 'cause you Marty Weiner: and I see, I'm sorry. Yeah. Christopher: know, attribution has important. Marty Weiner: Yeah, so let's do a quick little bit of art here. VFX Scott Ross who's film exec said seventy five percent of VFX jobs could vanish. I I thought that was a very optimistic number. I I feel like it's probably closer to ninety nine percent. and and that they're already vanishing, but that Zach: Yeah, I I don't know if you saw, but Danny Boyle, who is the director who did train spotting, he has a new movie coming out, and he used generative AI for some some little rats he wanted crawling around in a scene, and then for parts where there was like the idea was you look at a photo and the characters start moving, which is obviously very easy for AI. There was a lot of anger. It's gotta be out on Netflix and I'll be interested to see if the anger translates into not watching. Marty Weiner: Mm-hmm. Yeah. Well, I the problem too is that this stuff is getting so excellent that I I I guess my argument's been somewhere out there there's another Martin Scorsese. and if if this person is able to be the director of their own film, then I I think it's only a matter of time before it it starts to get it more commonly accepted. here's a great example, by the way, using Chat GPT six to make a movie, and this is you a lot of Criticism of AI is that you don't have artistic control, but this guy was able to work with Chat GPT to make a a map of the whole thing. So basically map out the the shot. Then they did a pre-render. and I mean this looks like filmmaking, right? And then finally they use ChatGPT six Astra plus Higgsfield plus viewport to make this excellent looking movie. like it it looks fantastic. So I I don't know, I I just I I'm gonna have to go with the view that's gonna get some people angry with me that it's only a matter of time and we're gonna get some excellent content coming out of it because there's some excellent there's some Martin Scorsese's out there waiting to be heard. let's finite yeah. Christopher: Well what's amazing about the latest set of models is honestly the fact that they're able to build their own tools and along the way, right? So it's if you give them if you give a a situation Marty Weiner: Yeah. Christopher: where there is no off the shelf tool, it will just make one. And that's I think that's Marty Weiner: Well that's what we Zach: Yeah. Christopher: that's the unlocker. As we're as we were finding with the with the secret project that you revealed. It's like you can just like make it just make a tool. Marty Weiner: That's right. The the spaceballs thing is like Zach: Yeah. Marty Weiner: I I know, I it was funny. I asked it, is there a tool for editing NES you know, pixels? And it's like, you what, Christopher: Sprites. Marty Weiner: I'll just make you one. Yeah. So let's end on weird and funny here. entry to AI Art Competition 1 but was later disqualified because it was an entry from a human. God, these humans. Yeah. so a photographer Zach: Ugh gross. Marty Weiner: entered a real photo of a flamingo. Zach: Wait the one of those apes? Try to have a nice competition for AI. Marty Weiner: Yeah, one of those meat brains. Yeah. God. You know, we're trying to have a spot for AI to to have some art. And a human once again comes in with their f with their flamingo. They're real flamingo, by the way. So anyway, I think the humans should give AI a a chance. They keep, you know God. Zach: Give it a chance, yeah. Let it let it let it fly. Marty Weiner: And then I I don't know if this belongs in Weird or where this belongs, but it's fun. ChatGPT six Astra's tax Zach: Ha Marty Weiner: return underpays the government. Zach: Yes. This this was this is this was from their demo. I I think it was like an accountant checked in, it was like they were off by like two bucks, so they were gonna Yeah, yeah, yeah, yeah. They demonstrated we can return your Marty Weiner: This is wait, this is from their demo, like from their launch video? Zach: taxes and apparently they they they shortchanged the government by like two fifty. so it's really working well. I mean a real accountant would have gotten, you know, a couple thousand. but sa sadly we're not there yet. Marty Weiner: So your your tax preparers still have a job, don't they? Zach: Ar artificial accountant intelligence has not been achieved. Marty Weiner: All right. Well, that's a fantastic way to end the show. So yeah, have Zach: He's Marty Weiner: your LLM, please like and subscribe, and we'll see you next week. Zach: God.