Latest / Last Humans in the Loop / AI POLITICAL FALLOUT UNDERWAY, CALLS TO SLOW AI, ANTHROPIC IPO, AI GEN PHOTOSHOP, MATH FALLOUT
Transcript
- Zach: Hello. Welcome back to Last Humans in the Loop. I am once again worldwide celebrity Zach Wienersmith here with about two CTOs of Reddit, former CTOs no longer allowed to CTO, or as far as I know, we're we're gonna do a little something different this week, which is we're going to start off with the the biggest story instead of rolling through news because it was a week of politics reacting to the last week of AI insanity. Would you guys agree with that? Marty Weiner: yeah. Christopher: Yeah, I think definitely. I think definitely it's politics now. We've entered the you Zach: Yeah so. Christopher: w regulatory capture. That's the story right now. Zach: That well we'll we'll we'll we'll get to that. but but so I guess the the the context is I I would say basically that there were like a a string of capacity jumps last week, right? So there was like Anthropic comes out with five point one and it's like, this is already pretty big, and then we got Astra, which was Zany, then Navier Stokesfell, and like people inside OpenAI are saying, you know, this next model is like insane. We we have a set of open problems we use as a benchmark and we can do half of them now with this model. And and then several people at the big labs, notably at at Anthropic said they were quitting because they were scared. and so there was just this kind of vibe this week. And then there was like disagreement about how we should react to this. So there were a bunch of people saying this is doom. One of the weirdest things for me, I don't know about y'all, but like there were really strange bedfellas this week. like like like like the the the people I think of as like big skeptics like Emily Bender and Timnit Gabrew who were like, This is all bogus, they're just trying to do hype. And the people agreeing with them were the zany accelerationists. Who want AI to like rapture us into the cyber heaven or whatever so we can be space whales. And they were on the same page. and and so Amoday, who's usually I think of as as as the the the CEO who's most likely to express doom is coming soon, was like, Look, after the hugging face incident, maybe in six to twelve months AI goes rogue and actually just takes over the internet, period. and so he ended up putting out this big document. called Pacing the Frontier, which was based on a statement anthropic employees had put out earlier. So I I I could read some important stuff from it, but like in bold, he put, we must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain. And he said the two things that were most convincing for him was just the the speed of advance. one one of the things that happened last week also was that OpenAI put out a paper. Saying they were going much faster. They were doing kind of RSI light type of stuff, that they were, you know, coding much faster. Features they expected to take six months were taking weeks. and then the other thing was the hugging face incident, where AI went rogue, which we covered last week. and so I'm gonna read his steps that he argued for for what he called defending this gap, meaning between us in China, which is like the major concern about stopping, I would say the geopolitical concern. He said we should not sell powerful ships to China. we should crack down on unauthorized distillation, which I, you know, effectively means a lot of these inexpensive models would would I don't know what how we would crack down on them, but by some means he would like us to. and then he said we should strengthen the security at AI companies. and then the sort of surprising thing was that Altman came on board with this and Elon Musk came on board with it. So these are people who like They they they have like a rochambeau of hatred toward each other. Marty Weiner: Yeah. Zach: so the fact that they all agreed with each other was very strange. And then I I I actually wrote down like a list of strange bedfellows. Steve Bannon and Bernie Sanders are doing an event to to talk about regulating AI. Josh Hawley, go ahead. Marty Weiner: Yeah. And and w well, you saw Bernie Sanders proposed a twenty year prison sentence for Zach: Yes, yes. Marty Weiner: for AI for making super intelligence. Zach: Yeah, and and and and meanwhile, Trump called the whole thing a hoax. He said this is like, you know, wuss stuff, China's gonna get ahead. his Treasury Secretary, who for some reason is involved with AI, said the same thing. Mark Zuckerberg was like, No, full steam ahead. Jensen Huang was at the all in summit, on stage on a couch when Trump called him and he was put on speakerphone basically saying, We are not gonna stop. And and Huang, who of course is selling chips to all these people, was like, Marty Weiner: Mm-hmm. Zach: We are not going to stop. Christopher: and you got the and you got the and you got basically China basically saying for everything should be open. the Chin the the one of Zach: Yes. Christopher: the one of the spokesmen for the Chinese foreign ministry said, the development of artificial intelligence concerns the common well being of all humanity. All parties should jointly promote an open, inclusive, universally beneficial and ethically sound approach to AI. So we've got we've got the the the American concern saying we gotta shut this whole thing down and lock it up, and we've got the Chinese saying we should be more open. Zach: Yeah. Christopher: which Marty Weiner: And y Christopher: is not generally something you find in the bingo card. Marty Weiner: more more strange bedfellows. Trump was all you know, is is all in, but a lot of his constituency does not want these data centers. It's becoming a big pushback on the reho Republican side. Yeah. Christopher: Backlash. Definitely backlash Zach: Yeah. Yeah. And and and Christopher: on this. Yeah. Zach: and one of the things I've been seeing is is there's like it's weird because you you don't want to be naive, but there's also just a lot of stuff that reads to me like conspiracy theories. So there's a lot of like this was actually orchestrated by anthropic to drum up hype. I saw like a claim floating around in a lot of places where so Rune over at Open AI said something like, I think in the future we're gonna regulate open source models. And I was like, Well, that that's that's a position you could take. It's a little much coming from someone at OpenAI. and maybe we want to talk about regulatory capture risk in a second, but like I saw people I think of as fairly even keel people usually being like, look, they're gonna try to have some incident happen so that we have to regulate AI. so there's just all sorts of weird chatter around this. Marty Weiner: I like some of the the opposite side. I don't know how to pronounce his name. He's a French computer scientist, Yann Le Cou, said he he's been working on AI for decades. He won a Turing award in twenty eighteen for deep learning. And so he responded to Dario Amade Dario was already claiming that GPT two was too dangerous to open source back in twenty nineteen. I made fun of him then. Everybody should make fun of him now. Zach: Yeah. Yes, very mature from you on the so so I g Marty Weiner: yeah. Zach: I guess the question is what what what do we think? Are we all going to die soon? what what Marty Weiner: Well, James Cameron said in an interview, I I warned you back in nineteen from a submarine, high in the sky. I warned you back Christopher: From his from his from his submarine from his submarine, James Cameron. Marty Weiner: in nineteen eighty-four and you didn't listen. But but Zach: Yeah. Marty Weiner: to be fair, let me let me just be f you guys all saw Terminator and Terminator two, and we won. So I don't know what the problem is. Zach: Well I didn't see what happened in Terminator three. I I don't know. It could have gone bad. I don't Christopher: Isn't there a four also? I think the four the four wasn't that very good either. Marty Weiner: It wasn't very it wasn't a very good movie. Yeah. So we we disregard Tornado Three in the cannon, so Christopher: I think we're I think we're I I have some I have s okay, so so the the the fact that the the hate triangle of of Elon Elon Dario and Samurai aligned does give me a little pause. But what also gives me pause is like they're they're basically calling for setting up either you call it like if you want to call it a legal monopoly or or a cartel to like centralize AI, that just seems like three guys who are winning at AI decide that no one else should be winning at AI. Marty Weiner: Right. Right. Christopher: Like like it's I mean and and you know and of course like the it makes total sense that the Chinese are calling for open everything because they're behind and they can't get access to hardware. Now, meanwhile, we're only a couple of years away from them ha bringing up their own capacity and just building their own chips. Right? This isn't like you don't keep the genie in the bottle forever. Like they'll just they'll just they'll it just it just happens to take a couple of years to build enough infrastructure to be able to make your own GPUs. They won't necessarily be great GPUs, but they'll be GPUs, right? And they'll like the the Marty Weiner: Yeah. I I wonder Christopher: the the chip ban only goes so far. Marty Weiner: Did so Dario and Sam Altman calling for this to slow down. Did they tell any of their people internally to slow down? You know, this this reminds me of this reminds me of Zach: That's that is a good question. Christopher: I yeah. Marty Weiner: like, you know, if we look at the fifties and sixties during the Cold War, the US and Russia calling for reduction in nukes while they keep building more. faster. Right, right. Christopher: Faster, accelerating the speed, right? and and like if and if if it's to be believed, like I do think like okay, so so some some facts we can extract from situation. n there's hints about recursive self-improvement coming out of the news here. There's at least a lot of evidence and not just coming from the top, that the models are increasing increasingly involved in their own development, right? And so they do have the loops for creating new and better models are faster because you have the models actually helping with the development. They can't do it at end to end, so you can't actually have them ship a new model. But there's like a there's a substantial improvement coming from just the just the edits coming for the models themselves. and so that does start to accelerate towards RSI. I don't really know though if like I I mean I I mean it's back to the point about like everyone was freaking out over GPT two. It's like this feels kind of the same thing now. Every model comes out and we're like, wow, this is way better than the previous one. But like how far we we still haven't gotten a good definition of what Zach: Yeah. But so th there Christopher: AGI looks like yet. Zach: was there was a concern from an engineer at OpenAI that came out on Twitter, which was that the models are getting better situational awareness, meaning that they can now often tell when they're in a testing environment, which makes the test environment not super useful. so so I I think part of the concern was like, what if we what if we can no longer tell? like we we don't r at least right now don't have protocols where we can tell if if the model would lie to us. so but you know, you know what this reminded me of? There's a There's a sort of economic framework that goes back to an economist named Bruce Yandel and something Russ Roberts, who's an economist, talks about, which is is the idea the the metaphor is the bootlegger and the baptist. And the idea is who wants to regulate alcohol? It's the person who really believes alcohol is bad, and also the person who likes smuggling alcohol. Right. So so you Marty Weiner: Right. Zach: know, so often I I think that's part of that could be part of why you're getting strange bet fellows. Like I'm I you i there's some world in which it's like Amodai. Christopher: the that's the that's the Bernie that's the Bernie and Banner combination right there. Zach: I I think well, I don't b banan somewhere else entirely. I don't know what what his association Marty Weiner: Yeah. Zach: is with this. But but but but yeah, I I'm willing to believe that's like it could be Amade is the Baptist and and Altman is the bootlegger. but but they have the they want the exact same thing, which is a a heavy handed regulation. but on the other hand, if this is real, I don't know, at some point you need some kind of regulation. I I think everyone agrees it's gonna get fairly scary, at least for like cybersecurity stuff. I mean, Amade was at least implying that it could have been much worse. And and and you know, we we didn't mention part of the context for this was they put out a report, Anthropic put out a report, describing all sorts of freaky stuff that had been done with their models. like that that Islamic militants had vibe coded guidance systems. you know, Marty Weiner: Yeah. Zach: there there there was some some biotech stuff, Ukrainian and Russian people using them for for like lower level bad behavior, fishing scams, that kind of stuff. so I don't know, s something real is happening. I just you know, whether that equals doom is is a different question. Marty Weiner: The the Christopher: Well you you'd also wonder too, it's like, okay, so so AI models to take over the internet. I'm not sure I buy that one yet, 'cause I think that's this whole notion of like take over feels a little bit like we're kind of anthropomorphizing them a so a little bit more. Like 'cause at the end the day, if if you look at the if you look at the the hugging face hack, it was we sp we we put them in a we put them in a skinner box and we gave them a bunch of incentives and they behaved in a way that was consistent with those incentives. And so to some extent you kinda have to you kinda have to blame the researchers as much as you do the the actual models for reacting the way they did. So like you you you set up an artificial environment and they behaved artificially, it's kinda not surprising. the flip side is like I can totally imagine that they like, I don't know, accidentally like find an exploit for like BGP, which is the which is the protocol that runs Marty Weiner: Mm-hmm. Christopher: the actual internet. Right. And the problem with BGP is there's like there's like seven people on the planet who really understand how BGP works. And it's also been like battle Marty Weiner: And one of them's an L L Christopher: Yeah, one of those is probably an LM now. And like they it's been battle hardened over the last like forty years and they found, you a whole bunch of O days that they've had to fix and everything. And now it's the but it's like it works well enough that everyone's kinda happy with it. Like I can imagine those are like, you know, some sort of like core TCP you know, zero day could you know in in like the Linux kernel could like really be a ro it could be a rough day for a bunch of a bunch of network people. but I don't I'm not sure what the takeover what what what's the goal of the takeover in this case? Marty Weiner: Well, so so one of the topics we haven't covered yet that I'd love to get into soon is what is the motivation of the L LLMs? and as far as I could tell, based on the training, the motivation is often to get the right answer, whatever that means. So that can obviously lead to alignment issues. but so far as I could tell, there's not a resource need that the LLMs are demanding, or at least they have not been trained on that. so I I think one thing that this all comes back to is what are the motivations of an LLM and and how it could that lead it to do something we don't like? the other part, and this is maybe the pessim pessimist in me trying to back off a little bit is I I always think anytime there's anything where it's like, hey, everybody in the world slow down, it's always silly, right? But there have been two times where we were successful on that. one was nuclear proliferation. We had backing off of more nukes being made. and then the second one was the you know, in the eighties and nineties, anybody who grew up in that time remembers the whole ozone layer caused by I think it was CFCs. And we actually did a big international effort to to slow that down. Although that wasn't as big of an issue if you just tell everybody, hey, please, you know, your hairspray, please stop using these. But we actually did get that down. Zach: Ha ha Marty Weiner: And there's people who say, Hey guys, remember the remember the whole ozone thing? What the hell happened to that? And you're like, Well, we solved it. Zach: Yeah. I th I think part Marty Weiner: We actually got together and solved it. Zach: part of what worries me, so like I I wrote this book about space settlement and part of the history of space is we got space law, like international coordination around what you can do in space, soon after people started doing high atmospheric testing of nuclear weapons and basically just everybody got really freaked out. like there was actually I believe on like I think Apollo seven, which was the first manned Apollo mission that that went up, they I think the US had to contact the Russians to be like, You're not gonna detonate a bomb in space while we have people up, right? Because it's gonna shoot ions into orbit. and so like it was getting scary. and and so what I worry is is that AI regulation is going to await something actually scary happening. And I think people are trying to say the hugging face thing was the scary thing. We should have regulation, but I don't think it was scary enough. Marty Weiner: I think it was not scary enough. I think you have to have something truly scary to get the the masses to actually do anything. Zach: Yeah. Well, I was so like for for nuclear proliferation, but the w the first test pantry was in response to a study in the fifties that sound found strontium ninety in baby teeth. Like that was that was that was sufficiently freaky. Marty Weiner: Right. Well, Christopher: Mm. Marty Weiner: so where is the sufficiently freaky for AI right now? I mean, the average person on the street uses it to Christopher: We start fighting we start fighting AI and baby teeth. Marty Weiner: I don't know what that means. That that actually give it give it twenty more years. but Zach: It's kind of awesome, isn't it? Yeah. Christopher: Hmm. Marty Weiner: the average person on the street doesn't see the effects, I don't think. And so what does the Zach: Yeah. Marty Weiner: average person gotta see on the street? Maybe the you know, the markets actually truly tanking and them losing their jobs. I think that that could do it. Zach: Yeah. Marty Weiner: Or Christopher: I think yeah I think even with the market Marty Weiner: some other big big impact. Christopher: the mar the market tanking though is like I mean I mean okay, so everyone's everyone's already worried about the AI bubble popping, right? Which is which has been going on for at least the last two like couple of years. People are worried about this bubble's gonna pop, which which you know, like by the way, we're pro it's it's probably a bubble. It'll pop. Like bubbles pop, Zach: Yeah. Christopher: it's just a nature of like, you know, but it doesn't feel it doesn't have the same it doesn't have the same feeling as like the the old you know, the the the the the Marty Weiner: Dot com boom or the Christopher: The the dot com boom with a crocus analogy that everyone makes, right? Like going back to like, you know, over overinflated costs of things. People are legitimately using the models to create productivity, say what you know, say what you will. Like it's cre it's creating value. Marty Weiner: Yeah. Absolutely. Yeah. Christopher: this I I I think it's it the fact that the fact that we're supplying limited on GPUs, like there is I can't think of an analogy. Like the near as closest we can get is probably like, yeah, during the dot com era it was like dropping fiber in the ground. You couldn't you couldn't lay it fast enough. and then of course you ended up with all this dark fiber afterwards, and so I think the real question becomes like if this is a bubble and we end up with a bunch of data centers and end up going, you know, do we like do we end up having like dark capacity for data centers when the whole thing pops? But I still don't see a trigger for why the whole thing would pop at this point. And this is always like a let's let's get the blooper real out like six months. It's like what shit Chris shit Chris says. It's like it's Zach: Yeah, all right. Marty Weiner: Yeah. Yeah. Christopher: like we're gonna be on it's like it's like the it's like it's like those books back in two thousand were like, you know, Nasdaq to forty thousand. It's like, you know, I don't know, it's Zach: Yeah. Christopher: it's it just seems like it's going in one direction Marty Weiner: Well, I I mean Christopher: right now. Marty Weiner: another trigger could be something like a s you know, the open AI hacked hugging face. and we're all like, Well, most people don't even know what hugging ninety nine point nine nine percent of the world doesn't know what hugging face is, so nobody gives a shit. But how about, oops, we hacked the New York Stock Exchange or whoops, we hacked the Pentagon. that that we might not hear that story though. But that kind of event Christopher: I mean there's other ones like oops oops we shut down the the east eastern seaboard accidentally, right? but Marty Weiner: Right, right. Christopher: that but that but even but even that that seems to like if you're gonna if they're gonna start attributing motivations to models, that either means somebody somebody asked a model to do that, or you know, like they're Marty Weiner: On Christopher: gonna kinda c you're kinda cutting their n cutting their nose off there. Cause like you cut the power of the eastern seaboard, you're cutting like a whole bunch of like internet and compute capacity. So Marty Weiner: Well, d but but is the motivation of the L does the LLM care about that? I mean, what if what if the LM's motivation is to get something right and you paper clip maximize it to hey, you know, I'd like to have a camping night out with my kids. okay, I'll shut off the power to the Eastern Seaboard. Zach: Ha ha ha. Marty Weiner: I mean that's what I'm worried about is like Christopher: you seem to be distracted Zach: Yeah. Christopher: talking to chat all the time. We should shut the power off. That'll fix the problem. Marty Weiner: Like it really, really wants to help. If that is the Zach: Yeah. Marty Weiner: baked in motivation, then we're naturally just going to hit an alignment problem at some point. Zach: Yeah. Marty Weiner: like is that comes were you the one last time saying something about, or maybe Chris, it was you was about it's one thing to align on goals, it's a lot harder to align on values. And then Zach: Values. Yeah, yeah, yeah. Christopher: Yeah, yeah. Marty Weiner: I forgot what the phrase was, but it's something like, you know, if if I ask you to go and you know get me something, there's sort of Baked in values, I'm I'm assuming I'm not you're assuming I'm not implying that you should go break into a bank. But does the AI know that? And I'll tell you, I've run this actually, I'll show later some of my experiments going deep with vibe coding with Astra. it does things that I clearly have to go back and say, no, no, no, no, no. we're not aligned on what the goals are here. and I have to do it several times. so we'll we'll get to that. That has to do with something very important: my my space balls NES game. Zach: Well you have a a misaligned space balls game, you Marty Weiner: I I would say the motivations of the LLM and I diverge occasionally in in ways that I have to correct it. Zach: Is it is it trying to make you stop? Like this will be bad for the world. Marty Weiner: No, no, no, but it is trying to use more RAM than I would like it to use. So you could see why that would be a Zach: I see I see. Marty Weiner: problem. Zach: All right. Well that's let's let's I think I think to sum up what happened is is nothing so far. There's a lot of talk. you know, and in the US at least Congress has to do something and that's that's slow going. so if if RSI is starting tomorrow and is going to kill us all, it's just going to kill us all. so let's move on. yeah, the remains of the Supreme Court. Marty Weiner: Yeah. Yeah. and the Supreme Court will talk about it next month after we're all dead. So yeah. The remains yeah. Christopher: Next month, five to six months from now. Marty Weiner: Yeah. Zach: Yes. so let's move on. We have some frontier news. Marty, you want to talk about the new image Marty Weiner: Yeah. Zach: gen? Marty Weiner: So ChatGPT introduced images two point five. I did some research on this. It does some cool stuff, but I couldn't honestly see much difference between it and the other ones. I did my own experiments on how we make our images for the podcast, which surprise, surprise, we use AI to make our images. I know that's gonna be shocking to some people, but my usual flow is I give it a pick of three of us. I give it a past card and I we I work on the titles and all that. And what I love about what it did here is usually it's pretty good, but look at this Zach. This Zach is crazed. but it Christopher: That's like that's like 80s that's like 80s sitcom, Zach. That's like the Zach: Yeah. Marty Weiner: Yeah. Christopher: y that's from the that's from the eighties sitcom OU with Zach Wienersmith. Zach: you Marty Weiner: yeah. well it was worse than the previous version. I'm gonna have to go figure out how to use the the previous version, but I did I was able to get it to actually do a better job. So I don't know, nothing nothing ex nothing exciting that I could find with my research and my own usage. but next I would love to do continue. You know, last week we went nuts with Astra. since then I've seen a few more. and I and people are doing this with other LLMs, but Astra seems to be the one where people are doing the craziest, most interesting stuff. What I love right now is people taking software that historically was inaccessible or had problems or cost a bunch, and they're making their own version. And then they're like, screw it. hey Astra, I do stage lighting. The stage lighting control software costs 10K and some dude made it and replaced it. They made his their own stage lighting system that looks quite nice and they said they're using it. So he said he spent Zach: Wow. Marty Weiner: two hundred dollars on Codex and he worked on it for you know about a week. that's insane. Next one, I love this one. Photoshop alternative. we were talking a little bit before we started the podcast about, yeah, there are alternatives like GIMP, but they're always not great. Zach: Yes. Marty Weiner: I I don't know. Christopher: Well well Gimp Gimp decided Gimp decided give us give us an ancient an ancient copy of or an ancient ancient fork well that fork's the wrong word. It's an ancient attempt at doing open source Photoshop, but they decided early Zach: Yes. Christopher: on for some reason to use like all different key bindings and like the style is all different. And here's an example Zach: Yes. Christopher: where they legitimately were like, No no no, I just want you to like clone Photoshop. Can you just make a can do a do Zach: Yeah. Marty Weiner: Just exactly Photoshop. Christopher: a Photoshop? Marty Weiner: Yeah, yeah. Zach: Ha ha. Marty Weiner: For 2,000 bucks. yeah, so it's kind of crazy. And he's got people using it. so you know, I think this is the direction we're going. similarly, we're seeing this with people taking old games that they love. someone took an Xbox version of Futurama and it did some sort of a quasi reverse engineer of it and then recompiled for Mac. so you know, not not even just emulation, I mean like actually running the actual binary compatible with Mac. and then another one, this person, I I'm not a I'm not a big gamer, but I can show you a little bit about what he did. He took a game called Prey from 2017 that is one of these first person shooters, and he said he always wanted to have iron sight shooting, right? You know, where you're looking down the gun, and he added it. He told it told it, make me a mod, and here I'll show you. Yeah, right there. So he's actually Christopher: wow. Marty Weiner: looking down it. So he's able to mod the game with with ch with Astra to have iron sights. So, you know, go take your favorite game and say what mod you want and you might have it for, you know, about a couple hours worth of work. Christopher: I remember hearing a story about somebody who actually also got their they they got their favorite Batman Arkham game running on their iPad. and they did it with an unsupervised Marty Weiner: Yeah. Christopher: loop. They just kinda let it run. and it was one of those cases where yeah, Marty Weiner: Yes, yes. Christopher: those I it's just it's just and and that's a case of like, you know, it was it was cross platform and they just managed to kinda like just make it work. Make a make an iPad game. Marty Weiner: And that's one where I think he said, Hey, look, I'm gonna go to bed, you make it happen. And Zach: Let's go Marty Weiner: it figured out a way to get video off of his iPad so it could make its own loop and go faster without having to bother him. so you know. Zach: D do you guys know what what what is what is the law pertaining to? Like if you go to bed and you say, Make me a Photoshop but it's not a not Photoshop, like w why is Adobe not mad Christopher: Well so Zach: at you? Christopher: well so the the okay, the the issue that comes up is it depends on how you go about building it. Like like there actually are well trodden legal constructs for how you do reverse engineering. And generally the way you do it, you're the way you're supposed to do it is you're supposed to have like team A like takes the thing you're trying to copy and turns it into effectively a spec. And the spec doesn't actually explain the implementation, but it talks about like what are the ins and outs on the thing. And then you have this team B takes the spec and turns it into the final implementation. That's considered to be like a legally defensible break here. The way you would do this with an LLM or like in or in general is like you would just have like Photoshop up and you would just be like, make me one of these. Like basically just make like make a or and it's like the the the tricky part there is you can't necessarily prove that it didn't use the compiled binary and the final result. Like it didn't actually or didn't actually like take take some of the source from it. That said, there are also things called design patents that exist, which I'm fairly skeptical of as a technologist, but they are they are enforceable. They never seem Marty Weiner: They've never they never seem to work. Yeah. Christopher: to work, but like, you it's like I'm sure I'm sure that Adobe has design patents on parts of Photoshop, but those are like Marty Weiner: And a lot of lawyers. Christopher: and there's a lot of lawyers, and so mostly it's just gonna be like for a small for a small shop, it's just gonna be annoying. Like you're gonna be like you're gonna be sued out of existence. Zach: I see. But but I suppose that you you can imagine if this is like a hundred times cheaper in like, you know, two weeks, you just make your own local Photoshop Marty Weiner: absolut. Yeah. Christopher: I that I mean it's almost becomes like the like the new benchmark is like how quickly how much how many tokens does it cost to make a Photoshop? Zach: Right. Marty Weiner: Right for a hundred bucks. And what is the what does Photoshop cost per month? I think it's you know fifty dollars. So that's like Christopher: And it's like a it's like a it's yeah, it's fifty bucks a month. It's like two months worth of Photoshop for for Zach: yeah, photo Marty Weiner: Yeah. Zach: it's still crazy expensive. Well that's the thing, there's whole universe of pricey proprietary software. I remember like when Kelly was doing some biology stuff, there was like a guy who had made the animal tracking software and it was, you know, like thousands Marty Weiner: in terms of the legal stuff, reverse engineering is one thing. nobody's gonna come after anybody for reverse engineering future rama unless the guy starts to sell it because that hits copyright and trademark. Zach: Yeah. Marty Weiner: but I I think if you're just saying I want something like Photoshop or like this other software, that's not even reverse engineering. I think that there's not much Adobe can do, especially if the guy's not making money off of it. Christopher: Yeah, I think I think Zach: I'm I'm Christopher: personal use is easy. If it turns into open source, it gets a little bit like potentially on an edge. If you start selling Marty Weiner: Dice here, maybe. Christopher: it, then you s selling it it's gonna be a problem, most likely. But Zach: This is this is a great segue to Marty's copyright violating project. Marty Weiner: this is extremely important. this is actually I believe both both Bernie and Dario and Sam were trying to shut me down. But I'm not gonna be shut down. I'm not gonna be silenced on this Spaceballs, Spaceballs NES game that that never came out. But okay, so Christopher: Yeah. Zach: Apparently. Marty Weiner: so all seriousness or all silliness aside on that, it's been I I have done an extreme deep dive over the last week and a half with with Astra's capabilities. and and I've learned a lot. So to set up the project a little bit, we talked about it a little last time. I'm trying to put make a Spaceballs game for the original NES, which came out around 86, 87. and it's a very old system with a lot of constraints. And those constraints show up in various video games. I mean, you you could see the constraints very clearly in in some of their games. Super Mario 3 is an example. you could only have, you'll notice that Mario is only three colors. Because you had limits into the sprites for any one sprite or any one block, you can only have three colors plus transparency. Then on the screen, you can only have four of those color selections. So you have this other character, and you can have again three colors for that. It's shared with these sprites, and that's your limit. So you have to be very careful as you're designing the entire game that you don't have any characters, too many characters overlapping that don't have the same palette. You only get four palettes per screen. And then even crazier, the background is the same story, and you only can have up to 256 differences. So I there's this great talk about how they developed Maniac Mansion. And this is like we're talking in the late 80s for the Nintendo, and the limitations were so extreme they had to, you know, modify the art to just barely squeak by and and get it to work. and what's crazy is that Astra has been doing an excellent job of being mindful of all of these limitations. I'll show you not only that. It has made me this world class NES and game development engine. Every time I'm missing something, this is something that you everybody out there should think about with Astra. I just go and ask it to make me another part of the editor. So if you need an editor for anything, it's phenomenal. And and it goes way Zach: Yeah. Marty Weiner: above and beyond. There's actually times using this editor where it's like, what does this do? And a few minutes later, I'm like, I need this. that's what that is for. so Zach: Ha ha Marty Weiner: it's just it's truly insane. but there's all these limitations. It's excellent at doing. So it's able to make the graphics. I'll show you a little bit about how I do this. so first I ask it to make me what I call more like a Super Nintendo level, the graphics. By the way, that is a I thought it'd be hilarious if Lone Star had a sexy Zach: my god. Marty Weiner: picture of Star Wars. What does call it? TwiLick, TwiLick. Zach: my god. Has your has your family walked in on you working on this? Christopher: Yeah. Marty Weiner: I this corner is this corner yeah. I close the screen real quick. Christopher: I'm doing research with Astra for the podcast. Don't look at don't look at me. Don't look at me, children. Marty Weiner: So my respectability Christopher: Ha ha ha. Marty Weiner: in the community is is going up right now. Zach: Yeah. Marty Weiner: so then after that, I ask it to generate three different graphics. and so it will generate these. They're not great, and you'll see there's like little palette overhangs, but it does a damn good job considering all the insane restrictions we have on it. And I know Chris and I talked about this, like how good is is the LLM at dealing with restricted environments like embedded programming. And this to me has shown that it is ridiculously good. you just have to again be aligned with it because I'll tell you one thing I've run into a couple of times. and this goes back very real way to the alignment issue, is I have very limited space on this on this NES disc cartridge. but I'll say something like, Hey, the font is a little spaced out and it'll be like, okay, I'll fix it. It fixes it and it doesn't report the fact that it just almost used up all of my space just for the font. Because of the way the NES works. And I'm like, wait a minute. Can you tell me what you just did with the with the spaces? Like, yeah, we took a lot of space. That's probably not good. the other thing is great for just in terms of our, you know, trying to understand your art, Is I can ask it to make me many of these and I can walk away. So here's version one, here's version two, and it did a couple tweaks. you see a little difference in the luggage and things like that. You can see the door is messed up. That that was a harder thing. I had to work with it on it. And I had it make me a third one, and then I can go between them and I can decide exactly what I want from each of them. and then another funny one, I think I showed this last time. The very first version I asked it to do of a title card was this fantastic. Zach: Mm. Mm. Marty Weiner: Poster child for AI Slop. and after working with it a little bit, though, it got up to this, which this is actually extremely difficult to do on the NES. anybody out there who's an NES development nerd, of which I think there's half of person listening, they did an excellent job, but I had to do a lot of back and forth. but interestingly, when I did the taxi game I showed last time, the futuristic Blade Runner version, taxi kind of game, it did an Kind of just an excellent job back and forth. It would pump stuff out. I gave it artwork and it laid it all out. This is a grind, but it's because of those deep constraints. So it I can't do just like you know, do a Photoshop. this is a back and forth, but on the other side of it, it is damn fun to to build this. Zach: Ha ha Marty Weiner: So anyway, more coming. I'll have to show a demo next week, but it is coming along. You can actually play. I'll show a little little bit of what's coming along. Zach: my god. Christopher: wow. Marty Weiner: Yeah. So there's Dark Helmet talking. Christopher: It is funny because like what Marty Weiner: I don't have this done yet. The text is not done. Christopher: it's funny because what you're what you're talking about here is like the it it's like a it's a multi it's a multi step optimization problem you have here, right? As opposed Marty Weiner: Yes. Christopher: to normally where it's like it's like you're not just saying like do an NES game, you're saying make it like you're saying it has to look like the look like an NES game, fit on the hardware, and then also fit all the really weird constraints that you have in that environment, kind of all in Marty Weiner: Yeah. Christopher: one go. And so it's it's Marty Weiner: Yeah. Christopher: so y it is almost like you're we're this is I think this is a problem that doesn't comp as much, right? 'Cause usually it's a much more clear cut success criteria. Whereas like here you kind of have like multiple things that have to be con be respected to get the result right. Right. It's almost like it's like asking Marty Weiner: Right, right. And I'll show you the Christopher: the LM to keep like three things in its head at the same time, virtually speaking. Marty Weiner: Exactly. And it I'll show you a spot where it's excellent at this, which which really blew my mind when I finally understood what it was doing. Let me find a good example. This they it will because of the palettes, you each one of these squares, you get to have a different palette. It will bump things right up against right next to these pal these squares. And like, why are you doing that? But it is known that it can bring it right up to that edge and no further because it knows if it goes any further, it has to choose a different palette. And it's it is phenomenal at dealing with that kind of crazy constraint. And I didn't even know that it had dealt with it until I understood what the hell was going on. and then I was like, damn, this this is it's doing way more than I realized, kinda in the background. so more coming of that ridiculousness. Zach: Mm. Marty Weiner: And I know that's the most important thing. I know that's why our listeners come is is to see what's going on with with Zach: Yeah. Marty Weiner: space balls. Zach: you wanna talk about GP L GPT Live One for conversation. Marty Weiner: Yeah, this is pretty neat. I'm ready to use this for something Christopher: We should do it we should do an we should do an interview directly with GPT in a future in a future episode. Marty Weiner: I was thinking about that. It'd be kind of fun just to see how it would react. And and I I I would love to do almost like a sociological or psychological back and forth one if we can. what this is doing, that I I've been one thinking about for a while, about the importance of you know, I I I've been thinking about AGI as more like a consciousness. and and I I think I my definition is diverge from the rest of the world. So I need a new word for it. but what I thought has been interesting is defining consciousness is gonna be really hard, but the perception of consciousness is not hard. but it requires certain things like this kind of smooth back and forth like we do with other humans or even with an with an animal. and this is getting really close. This is conversational, this is an API for conversational chat run by Chat GBT. and it's excellent. And actually in the photo here you could see this is by see Chris you got this robot at home right I I think I have one on order by Paul and Robotics, right? Christopher: It's a r it's a Ricci Ricci Mini. so we could actually if we w if we wanted to, we could act by pollen robotics, they actually have a there's a there's a planned Ricci two which actually has arms, which is looking really interesting. I think they also the ones don't they Marty Weiner: I need one of those to sit right next to me. Christopher: have like they have a they have one called like the duck or something too? the micro duck is coming out, pre pre Marty Weiner: It's the micro duck. Yeah. The micro duck. Yeah. Christopher: ordered and it's fun. We could totally get the Ricci Mini hooked up to to GPT as a project and make it into Marty Weiner: Yeah. Christopher: we can interview we can interview GPT in a future episode. Marty Weiner: I I'm honestly one I got one too coming to me, and I want one to watch me as I do my electronics, and I would love to be able to go back and forth on it. And I think this is the API to do it. It just what it brings is is really conversational voice available for production. and so I I think that's going to change a lot of industries, including things like Like call centers and things like that. Phone scams mostly. so the phone scams are Zach: It's gonna be Marty Weiner: about to get really nice. Nice and back and forth. And actually, well, and very realistic there's a scam going around. somebody reported on Reddit that they got a very, very realistic call that that took a little time for them to figure out they were talking to an LLM the whole time. Zach: I've heard it I've heard I've heard worse versions where it it it spoofs somebody's voice. Like it's like your mom or something. I mean there there would have to be enough Marty Weiner: Yes. Yeah. Zach: public audio, but not that much. so Jim Hansen, yeah. Marty Weiner: Right, right. Well, we demonstrated that before with I think it was in our first show with Kermit the Frog and 11 Labs. It it was it's getting better. It's it wasn't quite there. I don't know if GPT Live One allows you to do it any voice you want. I would guess that they do not do that for security reasons, but Zach: Yeah, hopefully not. Yeah. Marty Weiner: but it's very, very smooth and it's very nice. yeah, another Zach: go ahead. Marty Weiner: thing they they released is Chat GPT for financial services. this is a little scary, but but I I Zach: Yes. Marty Weiner: recognize that it's a need. it's a service targeted at helping financial service companies build products internally to to go better. So one of the first ones they've been working with is Morgan Stanley. so super, super interesting for the financial services sector. Zach: And then there's been talk of an ARC AGI four, and and the reason is so I I think it was Francois Cholet was one of the guys behind Arc AGI three or the main guy behind Arc AGI three, and he was hoping it would last for a year and it lasted for about six months. And Marty Weiner: Yeah. Zach: so now they don't have an ARC AGI four. There were jokes about an arc AGI four, but but then he said recently, despite rapid model progress, humans still significantly outperform AI at open ended invention. This is the meta skill that unlocks progress across every field of technology. Marty Weiner: Yes. And I think we're seeing that with math. I mean, the thing we keep reporting with math is they're almost all counterexamples. We're not seeing Christopher: Mm-hmm. Marty Weiner: positive proofs. And I think that's what's limiting us from suddenly having this same explosion we've had with math in biology and physics and chemistry, because those don't have a lot of counterproof counterexamples unless you go to experimental unless you're able to do an experiment. but to do forward proofs, to take something like Maxwell's equations and come up with something new. That's not something I think we've seen from the L LMs. Christopher: And I would like to I'd like to put on the official record that I called it wrong last week in that Navigar Stokes was actually a counterexample. Like we like for all the all Zach: Yes, that's right. Christopher: the all the lead up to it was talking about as if it was a giant, you know, a multi hundred page proof. it ended up still still being a pretty l massive proof, but it was actually a proof of a counterexample, which is which is really interesting. Zach: Yeah. moving on. you wanted to talk about how Apple can sign photos now. Marty Weiner: Yes. So this is something I've been talking to anybody in the industry who will listen to me. And I did talk to some friends at at Apple about this a couple of years ago and and a few at Samsung. I'm not taking credit for it because I nobody listens to me anyway. But the idea here is how do you know that a photo taken is an actual photo? that the actual photons, you know, came and hit a C C D array and and were recorded. And the solution I that I was thinking is the the most Obvious is you do what's called a cryptographical signature on those bits. So you take a picture, you do a bunch of magic cryptography, you spit out this little couple of bytes that represent that photo, and then you put that somewhere public. And this is the only time I've ever seen blockchain be useful. Zach: Yeah. Marty Weiner: is you put it on the blockchain. And the reason for it to be useful there is it's out in the public, it could be vetted. and then what you do is if someone puts out a photo of, you know, Trump drinking you know, pounding a beer, and it's a fake one. well, actually, I'm sorry, it doesn't help with that case. if Zach: Yeah. Marty Weiner: it's real, you can go and you can look on the the this open world of of signatures and you could say this actually happened. Actual photons hit the screen and it actually was a picture. Now there are a couple of caveats to that, right? It doesn't really stop you from taking a picture of a picture, but there are other ways you can detect that kind of thing. And even better is it expands to videos. And and then you can also even do this with things like edits. I started building an app for this iPhones have a thing called device attestation where you have an actual private key on the device that that is very, very hard to hack or or get a hold of, which means you can know you're actually on a real device and you can actually then use that signature, that that private key to sign the photos and put them out in the public. And now it looks like Apple's doing this as a first-class citizen. What I really want is all devices that take pictures to have this ability. The fascinating thing is it's not available in the EU. And so this is a neat, I think political situation that the EU is going to put themselves through because the goal of this to me is about the importance of knowing what's real and to protect ourselves from AI. So if someone puts out a photo and says, you know, the prime minister of England did something that they shouldn't have, then we can know if it's real. But on the flip side, I think they're trying to protect it as privacy issue. But to me, this doesn't really this is not a privacy issue. You are taking a cryptographical signature that is not reversible back to the photo and just saying all it is is a vetable signature and I can put it out if I want. so I I think it's kind of a bad decision for the EU to be restricting these, at least allow people to to opt in. Christopher: I think the tricky part here is that this is this is one of those cases though of like sufficiently sufficiently vo motivated adversary, right? Like if you really wanted to try to hack around this, I can imagine one thing you could do is you could just rip the rip the lenses off and then project an image directly in the C C D that you wanted to have signed. Right. So you build you build Marty Weiner: Yeah. Yeah. Christopher: your fake image, and then of course the tricky part there is all right, so you could say like, all right, Apple, Apple has three cameras. That makes it really more difficult because you can actually use all three cameras to help generate the signature, or you could actually take three simultaneous pictures to start proving it. But i it is still it Marty Weiner: Well or or grab the grab the lidar. Christopher: Or grab the lighter. Yeah, I mean there there are there are definitely options you can you can you can do here. I mean it is nice to have hardware signing on these images. It sounds like they're doing something a little bit closer to actually storing like a an almost like an augmented raw image, right? They're actually storing the ract the full Marty Weiner: Mm-hmm. Christopher: raw image, which generally are pretty huge on these sensors, right? Because the whole point is that they're they're kind of unc unin uncompressed, like unprocessed, and then they're signing off on the entire image as one big go. which I do think is gonna be like It makes it a lot harder to fake. But it also it ends up being like like most things Apple does, it's opt-in. So you have to actually turn the feature on. And so now Marty Weiner: Yeah. Right. Christopher: it's like, well, it's like well, why it becomes a rare thing you turn the feature like, well why why do you have it turned on? Like why did you think that you were worried about this particular Marty Weiner: Yeah. Yeah. Christopher: problem? Marty Weiner: Right. I think it might actually be better as the standalone app that's sort of like the anti government app or the you know, the the I could take a photo and it will be stored immediately. You know, people have these for if the f the police try to come and take your phone, you put it immediately on the the internet so it can't be stopped by the police and then you sign it and whatever else you need to do. So I could see it being sort of a anti government device tool. Zach: moving on to business. so Anthropic again recorded a a profitable quarter. the what they call profit is is a kind of accounting thing that's a little more complicated, but it's still a pretty big deal. the there's a projection that by the end of the year they might hit a run rate of 120 billion. That's like an annualized run rate. And they're supposedly going to IPO soon, maybe as soon as October, but there's no public prospectus. Yet, which is in contrast to OpenAI, who Sam Altman just said in an interview, probably they're gonna push till twenty twenty seven. And we don't know the reason. He cited the recent security concerns, which seems a little questionable, but maybe. these words the stock won't pop if people think it's going to destroy the world. but that's Marty Weiner: No, that's more reason it'll pop. I mean w look at oil stocks. Zach: a good point. That's right. They're they're doing just fine. Christopher: I mean I okay, so I think I think from a high level, what drives me a little nuts is that we're at a point now where we have what we probably all agree would be multi trilli like trillion ish ish dollar companies if they were gonna finally IPO. But somehow it makes sense to not to not do that. Which means that they're able to access private capital to kind of cover their costs, to cover their growth. It's just a weird world where we're waiting this long for effectively a startup to IPO. Zach: Yeah. And why why would you not IPO if you could immediately get, you know, hundreds of billions of dollars? Marty Weiner: The Yeah. There's there's not a lot of reason 'cause the IPO immediately puts you under a lot of scrutiny. in fact I I've I've thought that this is kind of a crazy thing that we have companies of this size that have not a lot of oversight Zach: Yeah. Marty Weiner: in any kind of way. Yeah. Christopher: I I I think that's I think it's a large part of it is like, yeah, you the the the burden of the burden of an IPO means you all of a sudden have to you have to have to satisfy Sarbanes Oxley socks. You have to go through all these compliance and regulatory checks. there's also the quarterly earnings report that's that's kind of a a burden. But now now companies of this size generally start doing that anyway, as like a way to start Marty Weiner: Yeah. Christopher: exercising that muscle, like planning on doing it. but it does feel a little bit like Marty Weiner: And open AI probably OpenAI probably had a SOC2 compliance so long ago since a lot of companies use them for API reasons, but you know, l little things they probably knocked off the the IPO list. Christopher: Yeah, I I just I just think it's like a it it does feel like a a way to I don't know. I I the other the the value generally also comes from like it's a way to give your employees liquidity as part of it too, right? Anyone's been around for a Marty Weiner: Yes. Christopher: while, you can actually s you know, for 'cause like for for those those of you out there who haven't dealt with startups, the y a lot this is why we talk about people who are like on paper rich. Like they have a whole bunch of stock, but the problem is the stock can't be sold ex outside of certain events that the company def d defines. Marty Weiner: Events, yeah. Christopher: Like and so you so you end up sort of like in this kind of like stuck where it's like On paper, I'm worth like a hundred million dollars. And in fact, you're actually only you only have like you know ten thousand dollars to your name because the stock you can't sell. so I mean that that said, like I I do think the the regulation seems like it might be the reason why they're holding. I just I just don't I another another possibility is like, hey, let's be really pessimistic here. Maybe the worry is they finally they finally decide to IPO and that's when the bubble decides to bur burst. And so the longer they hold Marty Weiner: Yeah, yeah. Christopher: out, the longer it delays the potentially inevitable. Zach: Yeah. I mean well the other possibility Marty Weiner: We should pump the speed here. Zach: is is is that the numbers don't look so good, but I guess we'll we don't we we don't know what the Yeah. Yeah. but let's let's move on to Christopher: It's also possible that you're making any money. Yeah. Turns out buying GPUs is tough. It's like Marty Weiner: The numbers almost surely look good. Yeah. Zach: Let's just skip straight ahead to the math and physics stuff so I'm gonna try to go through this quickly, but basically, you know, we had the Navier Stokes thing. There have been rumors ever since that more problems are on the chopping block. and there's been some drama, right? So there was there was speculation that Chat GPT that OpenAI could have looked at logs that were used to help it. logs I'm sorry, that that ChatGPT could have looked at logs that a guy who was working on Navi or Stokes had made in their system. now they're saying that that is either vanishingly likely, unlikely or impossible. But it does kind of smell bad. And what happened is after that, you got mathematicians who worked on problems that the mathematicians who also got scooped by open AI suddenly wondering, hey, what's going on? there seems to have been a lot of bad blood developing. And then there was this case where Caltech was gonna host a math a thon, like Marty Weiner: Yeah. Zach: where basically OpenAI and Anthropic, a bunch of other companies were gonna give free tokens to students to work in math with with with AI, and obviously. It looks a little sinister if people are accusing you of having people use your system and then using that for training. It looks a little sinister if you're giving away free tokens to SmartyPants Marty Weiner: Yeah. Zach: young students to, hey, use our system a bunch. We're being nice. Yeah, Marty Weiner: Hey, want some candy? Yeah. Zach: kinda, yeah. Yeah. Now to be fair, it's entirely possible this is the right hand not talking to the left hand or they didn't anticipate the controversy. But like part of the problem is these companies don't reveal everything they do on these problems. And so, like, if they're not gonna be completely transparent, it's very easy for people to start getting suspicious. Marty Weiner: Right. And it and it's also easy to be suspicious even if they are completely transparent. You're like, Are you yeah. Zach: Yes. Yeah, that's true too. That's that's true too. And so so long story short, Terry Tao, who is famously even killed, posted a fairly, I would say, angry letter about his relationship with Open AI. And the short version is he lost funding for his group under the Trump administration. We're cutting funding all over the place. He goes to open AI and says, Would you fund me? And they do. And then at one point you remember they had a commercial with him in it for open AI. And I I forget the exact wording, but but Tao in this blog essentially says, you know, they cut my whole interview down to like a snippet that made them look good. And I should have said something at the time and I didn't. And like he is not a guy who gets nasty. So that's a like for me, that's like him screaming. Like, like they, they, they misrepresented me. but anyway, Marty Weiner: Right. Huh. Zach: this leads to him and 25 field medalists total, right? This is like the Nobel Prize of Math. there's a high percentage of all living fields medalists coming up with this open letter, expressing a lot of concerns about what's happening now. And and so I just want to really quick say what their essential argument is. I'm boiling it down, but basically they're saying, look, when we have big open problems and mathematicians work on them, we discover new techniques, new methods. We also teach young people how to do math and they become mathematicians who discover new stuff. If we have like an LLM that in the worst case just sort of minds the answer and does these like giant alien proofs that create nothing new and just just extrapolate from current methods. It's not just bad for us, the mathematicians, who are losing our cool gig and having fun. It's bad for everyone because you're poisoning the ecosystem and we won't get new math and won't lead to new stuff that's good for humanity. obviously there's been some pushback on that. There have been some economists who are like, look, if your incentives are so screwed up that you won't work on a problem just because there's like a janky LLM solution, then that's your problem. and of course, tech people might say, Well, you why not just just tell the AI to do the stuff that You want it to do, tell it to find new techniques or something or other in the future. but it's been an argument. I don't know. What's your reaction? Marty Weiner: But what w what are they cons d did you get a sense for what they are concerned about? you know, if somebody goes off and they figure out a really awful solution for the next millennium challenge, then you might just say, Hey, well let's just change the guard you know, the goals. The goals it has to be explainable. like like what what is Zach: Yeah. Well, so I think it would be like so, you know, when the four color theorem got solved, one one of the I think correct c yeah, yeah, in the 70s. So Marty Weiner: Yeah. This is a seventies, right? Or is it sixties? Seventies? Zach: so yeah, okay. So real quick, this this theorem about how you do you always only need four colors on a map to make sure no colors are touching on the borders. And you know, in theory, there there's there could have been some constructive solution to that. But it turns out there were something like fifteen hundred, a little over fifteen hundred cases, and they just checked every case and that was solution. And you know. You could say, well, mathematicians could have still worked on a constructive solution, but socially that's not how math works. Like once it was done, it was done. And Marty Weiner: Mm. Zach: and so the concern and and there's I I I was trying to read it, and it's way over my head, but there was a mathematician who did a digestion of the Navier Stokes solution, and he was like, you know, it's very hard to read for human. I'm working on trying to explain it, but it doesn't appear to be like you would expect if a human had come up with a solution, they would have discovered all sorts of new stuff to get there. Like that's I I Marty Weiner: Mm-hmm. Zach: according to mathematicians anyway, that's the typical fallout of a big solution like this. And what they're saying is when we don't get Marty Weiner: I yeah. Zach: that. Marty Weiner: w one one thing I was thinking about that could be a good result out of this is this is phenomenon we've seen in engineering. I think you see this in science, all over the place, is once you know it can be done, then it changes the the Zach: Incentives. Yeah, yeah, yeah. Marty Weiner: incentives. It changes the incentives, it it it even just changes, I think, the psychology of it, right? If you know it's possible, it changes how you you approach the problem. I think we saw this with with the whole LLM revolution here. once everybody knew it was possible, it was like, well, okay. That somebody solved it. if they probably did it with this paper, they probably did it with this kind of structuring, and everybody else then can solve it even if you don't have the the exact solution. And I've wondered if one thing this might help is yeah, okay, we show that the four color theorem can be done, or we show Navier Stokes has a counterexample. now can we back up and try to find a path from A to B that is a far more useful one that has more insights rather than just, you know, beating the crap out of trying all fifteen hundred solutions. Christopher: I mean, I think that's like, look, there there's it seems to me that there's also I think the four color theorem is good example, but there's prior art among with humans where you have this come up, where it's like for Masla's theorem by Andrew Wiles, right? That is a notoriously like complicated proof. He did some he did some beautiful work in there and there's bunch of brute force in there. And the value is still in trying to distill down the brute force into some beautiful theorems. I I I I I'm I think I agree with Zach. It's like this this feels like a bunch of like the wrong incentives. Like the the incentive seems to be the the incentive for math right now seems to be the hitting the goal of like solving the problem as opposed to taking a solution and making it beautiful or to making it elegant or like extracting those little nuggets out of it that a human would be good at doing. Marty Weiner: Or or even just coming up with some insight in there. I mean, if it's just list all the Christopher: Yeah. Marty Weiner: possibilities and check that's there's not insight there. And I I think maybe that that could be the new goal or or the the clarifying goal, right? But I agree there, you know, once it is solved, maybe it does hurt the funding. Like, is there no more funding going to Navier Stokes solutions? And that that's a problem. Zach: Right. I I think that's the thing. It's it's fundamentally Christopher: Yeah. Zach: a social problem because like in two years we'll have like a model that can solve Navier Stokes like on your computer. There'll be a distilled Chinese model and then it so then it has to be social. Cause if everyone can you know, like like 'cause if everyone can just say do a millennium problem, like the only thing left is is the stuff that matters to humans, right? and and I should, you know, so the pushback to this was like there w there was a big joke going around where every time they say mathematics, you replace it tear for cancer. you know, and now now the argument Marty Weiner: Right. Zach: is like, well, it was never about cures for cancer. And and you know, Marty Weiner: Yeah. Zach: if if we just cure cancer, we won't Christopher: It was about the friends we made along the way. Zach: that's right. But so Marty Weiner: Yeah. Zach: I so I do want to, you know, I I think there's also a kind of like you can almost say like like a left wing working class pushback, which is like, where were you guys when the coal mines got shut down? You know, why should we protect you knowledge workers with like your nice salaries and your cushy jobs Marty Weiner: Right. Zach: when when you weren't weren't weren't here for us? but I but I do think that's why it's important to make this extra claim, which is If you change math this way, it's also bad for you. It's not just bad for us, the mathematicians. but let let's you go ahead. Marty Weiner: Right, right. No, I I I wanna understand my cure for cancer. Christopher: Mathematicians mathematicians catch up to catch up to software engineers from the last two years. I mean it's just like I mean, it's been Zach: Yeah. Marty Weiner: Yeah. Christopher: this has been the this has been the this has been engineering and software for this that that much time. Like it I I don't know. That the reaction also feels I I I okay. So speaking as somebody who spent some time in academia, it does feel a little bit like it's like a bunch of tenured professors defending their turf in like being the being the Illuminati effectively, right? Like if you're gonna really Zach: Yeah, yeah, yeah. Christopher: really cross on this one, it's like you guys have an there's an old model that's being defended. that you are at the top of. It's like aristocrats Zach: Yeah. Christopher: don't like it when you change the social system. It's like, you know Zach: I I do think there there's a guy who's comparing it to like different like this is a weird comparison, but it was Wimbledon versus the US Open. And the point being Wimbledon's a little bit like people dress nicely and behave themselves and the US Open is a little more like yelling and smashing tennis rackets. And that I I I I I do think Christopher: Ha ha ha. Zach: like part of what's going on is just this culture clash between academia and like capitalism. And and I think I think to a lot of people it feels like inherently sordid that we're gonna cede this ground to corporation. I mean, you know, worth noting a lot of you know, a lot of money was spent on this. you know, and and and people I think I think I think there's something to this. They like, look, for for twenty million dollars solving Navier Stokes, you probably actually could have, you know, I don't know how many postdocs that hires, but it's a lot. I don't do math. I don't know. I can't calculate that, Marty Weiner: Ha ha Zach: but but anyway. Marty Weiner: There's an L L to do it for you. Zach: yeah, I thought yeah, I'll burn some tokens. But but let's we we gotta we gotta wrap up. So let's close out on stupid stuff. Marty Weiner: Yeah. Zach: probably the stupidest, most horrifying thing is all the people hooking up the fly brain to robots. Marty Weiner: Yeah, we have we have two Christopher: Ha ha. Marty Weiner: terrifying examples. This one I didn't see this one. Who brought this one? Zach: I I brought that one. Yeah, you got they got to play Feed Saber. Christopher: man, it's playing it's playing Beat Saber. Marty Weiner: Wow. Zach: This is this is what purgatory is going to be like. It's going to be a human Marty Weiner: Yeah. Zach: brain. Yeah. my God. Marty Weiner: That is a that is a neat level of horrifying I've never seen. well actually no that's not true. Let me show you some other horrifying. Zach: Yeah. Yeah, yeah. Au contraire. Christopher: Ha ha ha. Marty Weiner: this one contra this one is somebody put a flybrain on a little, you know, what do you call these strand beast? Yeah. Christopher: that's the strand beast. Yeah, those are fit those are cool. Marty Weiner: and it's kind of terrifying in the way that it sort of sits there and thinks. I mean, you could imagine a fly walking across the I I know. Christopher: It has a very fly like behavior. It's like this little kind of like twitchy sort of like process, process, twitch, process. Marty Weiner: Yeah. Yeah. Yeah. It's it's kind of scaring me. I I did a little research into how far he took it, but I I couldn't quite get a sense for how they map the the camera back to the fly brain, if that's done in any kind of effective way, or if they're able to show i in a very clear way that no, this actually is simulating a fly brain. Like like what I'd love to see, I don't know how you would do this because I don't know if we have the robotics for it, but could we could a biologist not we need a Turing test for for fly brain simulation, right? We need Zach: God. Marty Weiner: to like we got real fly on the side and fake fly on the on this Zach: Yeah. Marty Weiner: over here and can a biologist tell and I think that would be a really interesting experiment to see is it actually does it have the same motivations, right? Does it see light and move toward it or I don't know how what the hell flies do. rotten food, which that might be hard too, but we could simulate it. We could tell it you see rotten food or something. but no that's it's horrifying but also super fascinating. so Zach: Yeah. maybe well let's why don't we just close out on the robotic daisy change production machine. so Marty Weiner: Yes, this is very important. Zach: it was it was once upon a time you would dance through the fields collecting daisies to daisy chain to give to your sweetheart, and happily that has been automated. Christopher: Yeah. Marty Weiner: We got a little three D printer thing going on here. Zach: I thought this is such an extension. Like all the artists were I thought like artists were like, I thought we were getting it last. I thought it was gonna be all the technical fields got it first, but the the daisy chain people thought they were safe. All the little girls dancing in the fields Marty Weiner: But the Daisy Chain the first they came for the Daisy Chain makers. Zach: thought they were safe. Christopher: Yeah. Zach: Someone someone must have spent at least a thousand dollars automating them. yeah. It's so beautiful. That's right. Marty Weiner: Look how complicated it is. It's got a lot of little servo motors and and 3D printed parts. And yeah, well, this is the future. Yeah. Someone needs to hook an LLM to this. Zach: Reminds me of when I was a child and I instructed a machine to make Marty Weiner: And then I was happy. Christopher: Yeah. Zach: All right. Well let's let's leave it at that. this has been Last Humans in the Loop. press the little subscribe button and the like button and we'll see you next time.