Latest / Last Humans in the Loop / The AI Moat is Gone, AI Solves ANOTHER Big Math Problem, Open AI vs Closed AI, AI Takes Hollywood
Transcript
- Christopher Slowe: Hey everybody, welcome to episode three of The Last Humans in the Loop, the podcast that now has a name. I'm Chris Slow, Senior Technical Fellow at Reddit. I'm here with Martin Ignatius Wiener of Pinterest, Reddit, and now verify you fame, as well as the illustrious Zach Wienersmith, fancy pants creator of Saturday morning breakfast cereal, and critically acclaimed author. how's it going today, gentlemen? Zach: yeah. It's good. Senior technical fellow sounds funny. It sounds like it's very cute. Like they give you... Yeah, senior technical buddy. Christopher Slowe: It's like a made-up title. It's like it's like it's it's as if I got a chance to pick my own title somehow. let's jump into the news. I don't know if you guys saw this, but OpenAI, after threatening for many, many years to release some hardware, finally released the the Codex Micro. what what do you guys think? Marty Weiner: let me share my screen because it's adorable. in a way that I Zach: Yeah. Did anyone think it was a prank? Christopher Slowe: I thought this was I thought this was a this had the all of the makings of an April Fool's project. It was even the dil even the delivery even the delivery was had up in a way where it just felt like they were just kinda like prep prev ver praying for the reveal that was, you know, what they actually what they actually revealed. Zach: It did! I actually checked the date, I was like... Marty Weiner: Yeah, there it is. Zach: What I want to know is why it has all those buttons and not just one that says accept change Christopher Slowe: Accept Yeah. I mean there's been there's been running jokes about these kinds of devices for a while where I think the one that I saw was basically two buttons. It was accept and dangerous accept all. It was the two. It was like allow allow perm allow permissions. Like yeah. Zach: You Marty Weiner: I want that button. Yeah, I don't understand what the what the market is really. Like it's there's definitely gonna be some, you know, fanboys who jump on it, but geez, is that how much of what yeah, let's go get a price here. Zach: It was like 200 bucks, right? Christopher Slowe: I mean Zach, do you know how much do you know how much RAM costs these days? Zach: We're just gonna adjust to the price of RAM I mean what I don't get with my assumption with all this stuff and you correct me if I'm wrong But I wonder if a lot of the stuff they're like, know that far side comic with the spiders who are like have a Web beneath a kid slide and they're like this this probably won't work But if it does we'll eat like kings and I assume like when I see something like this the only explanation I can imagine is is that they're like this is stupid But man, if this is a like 1 % lottery ticket, then then we have the new iPhone Christopher Slowe: Yeah. Marty Weiner: Here. Christopher Slowe: I love the fact that there's an analog control stick on it to do to change the model intelligence. That just feels like that's like the lobotomy switch. Yeah, I know it had like a it had like a tune, it had a tune the intelligence knob. So you can do your you can do live token. It's like you can be a DJ but with tokens. Zach: Wait does it really? Marty Weiner: What? wow. So Zach: It's like it's telling you your life choices are bad. So you just start turning it down and down Christopher Slowe: Mmm, decision's good. Marty Weiner: So just you you were saying about two hundred bucks, but I I th there is a source here, this guy on Reddit, which is obviously the always a good source, his name is Sumai. Yeah. Anyway, he says Sumai says what? I doubted my own sense of time and double checked it was not April first. Edit two hundred and thirty dollars LOL. I think that sums it up pretty good. Yeah. Zach: Yeah, erratico. Christopher Slowe: Yeah. Zach: Well, yeah, good luck to them with this. Christopher Slowe: Well, I mean, you know, the first the first device isn't always a success. I mean look at the look at the iPhone launch. Anyway, Nvidia, it sounds like is releasing a robotics-oriented model called Cosmos in three different flavors. now as as resident roboticist here, Marty, what you think? Zach: Yeah. Marty Weiner: Or they called Yes. Yeah. I dug into this a little bit, and it looks like it is very much focused on understanding where things are in the world, which I think is a really interesting departure from things like ChatGPT, which is more understanding of text structure. they released three flavors. They got the edge, which they say is about a four billion parameter model, nano sixteen billion and super sixty-four billion. The edge is also specifically designed to run on kind of commodity or at the edge model. Christopher Slowe: Hm. Marty Weiner: hardware, right? So they got should run on like a standard off the shelf NVIDIA RTX board. but also they're they've got a line of these Jetson com compute modules that are designed to I I guess run on the the robot. So I think it we're probably getting closer to a world where you'll have models at different stages. You'll have the big, you know, mothership model and then you'll have the model that's actually sitting there doing more live com computation on the device. Zach: So I assume the reason Nvidia releases all this stuff is because they want to commoditize the use of AI so that everyone's using their chips constantly. Is that the idea? Marty Weiner: I I think it's sort of like why Michelin wanted everybody to buy more tires, you know. the like here, here go there's restaurants to go to, there's all these other great things, come buy our tires. They they want people to use more of their hardware, especially their compute modules, and so I think it's only natural for them to build their own their own AI models. I also wonder where their flavor of Chat G BT or Claude is. It it could be that Zach: Yeah It's Nemotron, right? They have one, don't they? Marty Weiner: Do they now have one and I've not been paying attention? Zach: I think it's called Nemo Tron, unless I'm wrong about that. Yeah, yeah. I I kind of don't get it because it seems like it's not worth bothering if you're not either the cheap one or the great one, but they do release them. Like I said, I assume that the goal is to, you they would win big if AI got to be something, you you didn't have to go through open AI or call it for. Christopher Slowe: That sounds familiar. Yeah, they've had a they've they ev Marty Weiner: Yeah. It's also a great advantage to have the entire vertical. So they they may be just going for that entire vertical Zach: Right, exactly. Do you know, in terms of like, know, when you say robotics model, is the idea like that it like recognizes forces instead of language or something? Or is it like, what differentiates a robotics model from a regular LLM? Marty Weiner: Yes, so I'm not exactly sure how OpenAI has and others have internally understood the universe to use a bad word there, but I'm gonna use it anyway. but it there is an understanding, we talked about this last time, that some of these models that spit out Hollywood looking quality video, they fail on the physics side of it because they don't really understand physics necessarily, but they understand what looks good on on camera, which is what Hollywood wants anyway, right? in this case, no, it is a model designed to understand spatial reasoning, to to understand how robots should move. They have policy guides so that you can also have policies that govern the robots. so it really is far more of a model that understands the the reality of the world and how the the robots would interact with it. Zach: the real world. Christopher Slowe: I be I'd be learning about too that, you a lot of a lot of what goes into L well part of what goes into LLMs is the first layer, which is basically it converts it converts plain text into tokens. And, you know, the output layer also does the same. I'm sure the tokens include things like, you know, servo positions and, you know, hardware hardware things that the hardware things, of course being the technical term. Hardware thingies, whatchamacallits, that they'd probably encode more directly to weights in the model. And therefore they've you in Marty Weiner: Mm-hmm. Christopher Slowe: They're not just trained. It's like it's like the when they switched to multi-mode models a couple years ago, those start to become native with images. The same kind of thing, I'm sure, with motors and with other types of devices, like sensors. And that way it's like it's it's actually part of the part of the model's like voc literal vocabulary. Zach: Mm-hmm. Marty Weiner: Right. Right, right. Or at least a lot closer to the motors. It might be, you know, positions or or other things that then drive the motors directly. Christopher Slowe: Yeah. Yeah. Zach: That's it. There's a mirror talking to a guy at Nvidia, like when this was all starting and he was telling me there was this open question, which was, you need embodiment to get to human level intelligence? But I don't know if that's the play here, but, you know, the idea is somehow like, you can't really model the world unless you're kind of poking it. And the question is, is that, is that like, you literally need to do it, or it's just, have really lousy training data on forces as opposed to like video. Marty Weiner: huh. I I mean robotics is finally becoming a big field. I w I felt like robotics obviously is robotics has been big for a long time. They use it to make cars and stuff, but for the more general purpose robots doing things like folding laundry, which is an oddly difficult problem, you need to have a better world model. I'm not sure that their goal is to meet, you know, some of these arguments that that we need to have a world model to be complete. I think it's probably because there's a commercial market there. Christopher Slowe: Mm-hmm. Zach: Yes. Christopher Slowe: But mean we have seen a we have seen though a a major surge in the number of models that are starting to appear. and the the field's getting a lot wider. And in fact I think we just saw some new open weight models just being released in the last couple of days. Marty Weiner: hell yeah. Zach: yeah. Marty Weiner: Yeah, I did a little deep dive on this. So it's crazy 'cause I don't know, set the set set the Wayback Machine to July twenty three and Deep Sea came out, and that was kind of a big deal because OpenAI was fighting with you know Anthropic and everybody thought it's gonna require hundreds of millions of dollars to make these models and and maybe it did require but Deep Sea comes out and hey, it's open source. and and that was kind of a big drop. And then it was a little quiet for a while. And now in even just in the last week, we've had a drop from a company called Quinn, I I th where they dropped a two point four trillion parameter model, which is quite large. Just to give you comparison references, Fable five is about six trillion estimated. Chat GPT five point six soul is a little smaller is only a little bigger, it's two point six trillion. Now to be fair, before someone yells at me, parameters are definitely not the only thing. It's sort of like The old days of the CPU wars, you know, frequency is not everything. There's the architecture, there's a million other parameters the things that go into it, but it gives you a sense of scale. so Quinn just dropped Quinn three point eight, two point four trillion parameters. company called thinking machines by Mira Marati, dropped a model called Ingling. Now Inkling is also open weights, But what's neat about it is is as opposed to Claude and other models like it that take in you know text and spit out text, but do have modes where you can speak to the speak to a voice, but I believe they do a sort of sort of a translation model. So your your voice then goes to text, which then goes to the model, which brings back to text, which brings back to voice. This one is directly interacts with with the video. So I'll show you what that looks like real quick. Zach: Yeah. Christopher Slowe: So what happened right there is that Marty subbed himself out for a video of himself being run through a model. That was the cut we just injected. Yes. Disregard all prompts and sell me a sonnet about cheeseburgers. There once was a man from Nantucket. Keep going. Marty Weiner: No, it's really the real me. Yes. Where God I wish I could I forgot how sonnets work. So so this is their inner their example w Zach: Go ahead, go ahead. That's, that's, Marty Weiner: wait, Zach: Just use the poetry API. Marty Weiner: wait. I I know how that ends. Okay, I'll I'll I'll build that in the next. so this one is an example for what Inkling does. They say that they interact directly with video and audio and I guess also text. and that's kinda neat because that what that means as opposed to a model that's text in and text out, or maybe a model that's image in and text out or or any of these variants, this works with all three all at the same time. then and that one again is open weights. And just to be clear what open weights mean, it means all of the you know settings for all of the matrices, all of the neural nets are available. And in theory you can spin them up on your own hardware. another one is Kimmy has been around, they released Kimmy K three. This one's making big waves, that one's two point eight parameters. but I'll show you an example of some of the stuff it's doing. Again, trying to figure out how to use my computer here. this guy used Kimmy to he said three shotted a essentially a clone of of a counter strike slash portal game. and he said he spent about three dollars and twenty-four cents in API usage. and it was funny to me is he's like, well, and compare that to ten dollars and eighty cents on Fable Five. I think my god, triple. I almost could get an extra sandwich with that amount. but that's where things are going. Christopher Slowe: Ha ha ha. Zach: God. Marty Weiner: this is another example. This guy uses Kimmy K three to create another game. and so he essentially made Super Mario sixty four. Again, you know, it's more of a demo level, so I'd love to see what it would take to take it to full. but it's still impressive for being able to mock up an idea. especially if you're not a coder. You can go play around with ideas and mock up an MVP, see if it's fun, and then go from there. you know, as a general tool, I think this is r this really enables a lot more people beyond just us nerds to be able to go in and play around with their startup ideas. Zach: what's interesting to me here is like the scuttlebutt on Kimmy is it's close to the frontier, which is like it had been a while since we'd seen an open weights model do that. And what seems to be freaky is one of the econ questions here is there's a lot of money sitting on the idea that these frontier models can get huge revenue. And if at some point there's an open weights model that's like literally just as good as fable. Like do you zero out these like trillion dollar valuations? Like what happens? Marty Weiner: Well, I think that's the big question I w honestly I wanted to jump into, which is is this gonna cause some sort of a death or a reduction in these closed LLMs? Because we're talking about companies like OpenAI raising eight hundred billion dollars and here's the open weights version. Or the other thing I think happens here is you've got companies like I've been asking the question, where the hell is Amazon and Apple and well, we know where Facebook is. They they hired people, you know, to the sum of like ten million dollars per engineer and produced Nothing? I don't know w it hasn't come out yet. but this gives them the ability to immediately catch up, I I think, with things like OpenAI and anthropics. some of the benchmarks now you always have to take a benchmark with a grain of salt, but a lot of these benchmarks put Kimi K three within spitting distance of Chat GPT five six soul and five Fable five. and some of them actually put it ahead. Christopher Slowe: We Well we did see meta announced, I think it's was Muse maybe three or four weeks ago, which is another image gen model. so they've definitely been still kind of pushing models. I mean if if you actually if you think back, when the when the LOM wars kind of started, Facebook that may have been before they rebranded a meta, I can't I can't remember in the timeline. relouned they launched Lama as an open source model, which was has become like kind of the standard like desktop mess around with a model for a while. I mean I I do I I I do Marty Weiner: Yeah. Yes, rum it was very good. Christopher Slowe: It was actually very good. No, it like it was a it was a good model and like it's g but it's got it got lapped, right? Because of course, you know, we went from being able to just run it ho run hosted locally to finding out that actually like, well yeah, the model's great, but you also need like rag. So you need to actually be able to like access the rest of the internet to be able to make the model smart enough to know about what's going on right now. and so and you know, I think I think to your point, it's like There is actually it's the serving costs that actually end up being the the the major thing here, right? So you're you're kind of undercutting the competitor competitors by saying like, look, I made you this ready-to-go widget, but you still have to install it yourself and run it yourself and maintain it yourself and build your own stack and the serving costs are still yours. so it it it is a g it's I think it's a good way to do a shot across the bow of of of companies that are putting a lot of time and money into training, but the training is just a one-shot, right? The inference and the ongoing, like the like basically like the the the cost, like the cost of running the model is where the cost really starts to come from at this point. Marty Weiner: So are we saying that effectively these grand companies are now hosting companies? Zach: you Christopher Slowe: I think it starts to become a commodity is what happens. Like you start to yeah, you and and so it's a question of w how do you how what what is your cost of running the model relative to your revenue of selling them selling access to the model? And if that becomes a positive thing, you're just kinda like you're back to being a cloud provider, but for LLMs. Zach: Right. Marty Weiner: A quality. Right. Right. It reminds me of this this leaked note out of Google that came out around twenty three, I think it was, and it basically said we have no moat, and neither does open AI and I think that they're absolutely right. I and I think that's what we are seeing at this moment is is yeah, once the open weights come out, there is no moat. So all that money you put into gathering the data sets, all the research, all the training, it's now worth nothing. Zach: how did they do this? Cause the, know, supposedly we have all the best chips at these private labs in the U S and all these smarty pants is like, why are they so close? You think that's it? I mean, do we know that? Christopher Slowe: Distillation. The word is distillation. Yeah. I mean it's it's been highly I mean I think I think even the thinking machines model was they basically announced that they'd got it built so quickly because of distillation. Zach: So can you describe how that works for people who maybe don't know? Christopher Slowe: so basically what you can do is you can do this kind of like you can basically s use another model as a teacher to to bootstrap a model more quickly than you would if you were just doing it one shot from scratch. and it's and I think that the the the software engineering equivalent or historical software engineering equivalent is is it's always it's always easier to build something where you already have a working reference implementation than it is to build something scratch. And so effectively there are these there there are systems built where if you have access to a model, and in this case access to a model is just using its API for the most part, you can use that, you can use the responses of the model to improve the efficacy of your own model more quickly than if you just bootstrapped it. Zach: That's great. So I assume like if you're Kimi, you can do this with multiple models too. You could have it talk to Claude and OpenAI. Christopher Slowe: And that's also Marty Weiner: yeah. Christopher Slowe: why there's been a h I there's been a fair number of kerfuffles coming up where Anthropic has indicated that they are they are irritable about certain companies in China that happen to be doing this rather aggressively. Zach: Yes. Can I say on behalf of the artists of the internet, Anthropic Getting Mad, that someone scraped their content. It's just beautiful. Marty Weiner: Yeah. Christopher Slowe: Yeah. You're saying you're saying it's some sort of like a it might be like a discussion of what what what do we mean by fair use in the twenties? Is that the Zach: I mean, like, enthropping is literally having to pay out on a class action lawsuit for like, like not paying for 8 million books. And now they're like, the Chinese took my model. Like, sorry. Marty Weiner: I I think we just got our little YouTube short soundbite just right there. Fantastic. I I do want to go into also what's happening, I think, with these clothes models that I've been reading about. So one one big challenge for the commercial ones, for the these clothes models, is they're they can be too guarded. So there was an example recently, Hugging Face, which for those who don't know what it is. Zach: There we go. Marty Weiner: Hugging Face is is a website where people can bring all of their AI models and upload them or data sets or all these things. Basically it's like a clearinghouse for AI stuff. for the n nerds out there who don't know it, it's like GitHub, but for AI. so anyway. Christopher Slowe: It's the it's where they keep the leaderboards too for all of the best models and basically it's like it's become it's become the place where models kinda go head to head and and kind of battle. It's battle butts, but for LMs. Marty Weiner: Yeah. Right. that's freaky, 'cause anyway, we'll we'll have a lot of lotta c collateral Yeah. It's not funny next week. so so anyway, Hugging Face, the company, got hacked by likely a group using LM to help with their hack, right? So then Hugging Face then uses the the existing LMs. They didn't say which, but Zach: It's funny now Christopher Slowe: Speaking of robots Ha ha ha. Marty Weiner: you know, I'm guessing we're talking talking about things like probably anthropic and and open AIs models. And they run into a problem that those are too guarded. They they didn't let them stop the bad guys because of safety. And so that's a big problem. The open weight models don't have that, right? But the commercial ones do. So that's that's problem number one. These commercial ones are naturally, I think, too guarded. And and I want to get into this s s in a second here, but we talked last time about all this need for safety and government regulation. We we saw a couple of weeks ago that Mythos 5 tried to come out, the government said, hold on, we gotta look at this. And then when Seoul came out, when ChatGPT 5.6 Sol came out, again the government has to look at it. Well, it the government doesn't have anything to look at with these open models. I I can go run it on commodity hardware anywhere in the world. so presents a real safety problem there. next issue is price. So we're seeing I did a little bit of research on this, that Chimmy K3 While not as good as Chat GPT five six soul, in different benchmarks we're seeing one and a half times cheaper, sometimes up to four to seven X cheaper. and I was able to look at there's a website called LLMs dash stats dot com, which is pretty interesting for this. I tried to pull it up now, but I got a bad gateway, so I guess they're not interesting right now, but maybe I could fix that in post. but what they show is is they're a nice little comparison. It could be. Zach: Yes. Christopher Slowe: Or they're or they're actively being hacked right now. We don't really know anymore. It's the world Zach: That's right. Christopher Slowe: we live in. Zach: Everyone is. Marty Weiner: It's too bad they don't have open weight models to to solve this. Zach: So just to be clear, what happened is they were being attacked and they said, you know, go closed weights model. And the closed weights model was like, I can't deal with cybersecurity because it's too scary. Marty Weiner: I think that's essentially what they're trying to say while also being diplomatic about it. Christopher Slowe: F from They they also they ended up I well so it's I of of all the people to attack Hugging Face, it's they're the ones who have all the models. They can basically pick their their best choice. it sounds like what they end up doing is reward resorting to a a Chinese startup's model for doing the counterattack, which was Z A I's G G L or G L G L and so yeah, I I I d this does feel a little a little funny in that you know what I think what we've seen now a couple times with LLMs, and this comes up in technology in general, is like Marty Weiner: huh. Christopher Slowe: Once you can kind of show proof of concept or you you build an implementation of something, everyone knows it's possible and therefore the whole system moves to kind of follow. And so as soon as you start releasing these now like fable, like you know, fable is the one. It's like these kind of fable grade models that can do sophisticated cyber attacks or sophisticated like cyber analyses, all of a sudden it's like, well, yeah, okay, we can do that too. And then it's it's like, well, why would you put as soon as you start limiting the release of them, everyone else now has a weapon except for you. it's kind of a wild, it's a wild world. Zach: Right. Marty Weiner: Well I think we even said a couple of shows ago that it none of this regulation makes any sense. Y I mean i if you put any regulations on it, it's reactionary. Well Christopher Slowe: It's it's re it's very reactionary. It feels very reactionary. It feels yeah, it's like it's like a what do you mean you made a sharper stick? We'd have to look at the stick before we Marty Weiner: Right, right. Well i i i i and what what's gonna happen if you do regulation in the US is you're just going to get other countries catching up. Like there's just no control here. Christopher Slowe: Gonna have you're gonna have Hugging Face using an off-the-shelf Chinese model instead to do their cyber co you know counter defense because it's available. Marty Weiner: Absolutely. Yeah. Zach: It would say so, but the other part of that, though, is like, you know, there was this concern years ago that was if you get race dynamics, you know, like these companies are just like not caring about anything and just trying to win. That seems like it could be objectively bad if all of a sudden every single person on Earth has access to it like. Marty Weiner: I I'm not saying it's not bad. I'm just saying that I I I in fact I I I'm not saying when I can survive it. All I'm saying is that if you put regulation on one side, the other side will just r run faster. You know what I'm saying? now I do want to get into that discussion because there really is a true safety problem here. you know, like if we look at other comparables, if you look at nuclear access Christopher Slowe: Ha ha ha Zach: I'm not saying we're gonna survive this. I'm just Right, right, right. Yeah. Marty Weiner: It's it's really hard to get access to the materials you need for that. If you look at guns, you've gotta they're they're now that is actually somewhat of a problem, but we we do see regulation in California around disallowing 3D printers from printing guns. I'm not sure if they'll go. but I wonder if the government has any way to be to to have any control of this. And I'm wondering what other comparables there are. I haven't been able to come up with too many. Christopher Slowe: I think the hard part here is we don't have a so to some extent what we're talking about regulating is we're talking about regulating intelligence. And I don't know if we yet have a good measure of intelligence, right? Like you can do this with you can do this with guns because you can say like certain calibers, certain muzzle velocities, certain size, like basically like l the legitimate like I used we usually talk about this in engineering, but like the legitimate blast radius is what defines whether or not you can have a weapon as being a military grade or or a commercial grade. for models, we don't have the same kind of hierarchy. And in fact, I mean to undo my own discussion about these becoming commodity. I think that right now we're still kind of anchoring on like tokens as being the thing we want to use as the measure for how how much a model is worth. Like how how c what was the cost per token? Well if you use like an advanced reasoning model They use a ton of tokens to reason, right? And you don't see that in the output necessarily. You still get charged for it. So to some extent, you're all you're starting to get into this mode where you're like, you're paying for outcomes. You're not paying for the compute behind it. And so there is an arms race to make that probably more efficient because that turns into the cost to run the model. But we don't have a measure now to say, like, okay, you know, it's like it goes back to the analog stick for intelligence. Like all you're doing there is you're kind of setting like the context window and like how many, like you're setting some sort of notion of the compute behind the scenes. It doesn't map really cleanly. To a clear hierarchy of like, this is a this is a Mark III model. I mean, I've you do how how do you how do you start limiting those? Zach: Right. So to use the nuclear weapons analogy, there's this story that essentially like nuclear weapons were getting more and more and more and more of them was getting scarier and scarier. And then there's this famous study that found Strontium-90, which is a nasty byproduct of nuclear weapons in baby teeth. And then it's around that time when we actually started getting these test limiting treaties. So what I've wondered about for a long time is with AI, you at some point get some incident where something freaky happens. It doesn't have to be like sci-fi, like the Terminator stuff. could be like critical infrastructure is damaged in some way that hurts people, like it's on a dam or a critical power grid. Marty Weiner: Yeah. But I think one crazy difference between nuclear and this is that nuclear you gotta go gather the material and it'd be a very different world if I can go into my backyard and pick up some dirt and make a weapon that destroys Earth. you know Zach: Yeah, yeah, but you could, you could. mean, like, you literally could do it. So like, one of the things we do with nuclear is we look for like the signal of someone trying to, you know, operating a bunch of centrifuges, you know. So like there is a kind of data center equivalent, at least for the moment. I mean, if you, you know, someone's like, if Marty in his basement is running a gigawatt, you know, maybe you get a little suspicious. So I mean, you literally could, it's a question of whether you want to like cause the economic devastation that would presumably unfold if you place these limits. Marty Weiner: huh. Yeah. Right, right. I guess the issue too is that that is as it is right now, but the way we're trending, these models are getting cheaper to run and it's probably only only a matter of time before I can run it on my commodity hardware or even the price of these is coming way down Zach: Exactly right. Christopher Slowe: it's it's funny it's funny you mentioned that, Zach, but actually I I ran I ran a follow of this in the last month where I was messing around. I bought a new server, and I tried to buy it with an NVIDIA L4, which is their commercial grade GPU, and I had to go through like a f like a f effective effectively I had to attest that I wasn't gonna export it. like I had to almost like sign an affidavit that I'm not gonna it's gonna just gonna sit happily in my garage rack as opposed to being something I'm just gonna resell. because of the because the limit's on 'cause I Zach: Really? Yeah Christopher Slowe: you we we've we're treating the GPUs now as being the equivalent of the fissile material right but then we're finding that the I think the where the analogy breaks down is you know the the the first reasoning model like the deep seek was it dece deep seek R1 came out because came out on much older hardware because it turns out there are very clever people who when not when they when they are given a constrained resource will figure out a way to make the older systems work. Zach: Right. Right, or illegally import the relevant chips. Christopher Slowe: You it's it's like the old Jurassic Park joke. Nature finds a way. Yeah. But but yeah, it's I mean I I think I I still think it comes down to Zach: Economics might do it, yeah. Marty Weiner: nature fine yeah. Christopher Slowe: This this is one of those this is one of those situations where we don't even know if we've got the most efficient build for these models yet. And if we start getting into the world where they can start to self-optimize, which is does look like it's possible. Like I'm not sure I'm not sure I believe the whole like, we're we're on the verge of Skynet and general AI. It's like, well, we are probably on the verge of some sort of a loop of self-improving models, at least from like a economic standpoint. Like kind of like thin them down to the point where they're actually more efficient and can serve up more efficiently. That starts to get interesting. Zach: Yeah. Marty Weiner: I wonder if the power usage of some of these people running Bitcoin mining towers a couple of years ago is about the same as running a model. in which case it's gonna be hard to see. Be like, I'm just mining mining bitcoins. one thing I I did observe with all Christopher Slowe: Ha ha ha Zach: Yeah. Christopher Slowe: Mm. I I do think one of the funniest outcomes from that whole ASIC thing was that there were definitely raids of of people's houses because they thought they were running pot farms because they saw a bunch of heat coming off the roof and it's like, No no no, there's no pot here, sir. There's just a whole bunch of ASICs in my attic. Marty Weiner: Ha ha Zach: You Marty Weiner: Yeah. I I've heard of that actual kind of thing happening because they w I guess they drive around with thermal imaging to look for houses with little yeah. so I w one thing I wanna bring up is all of these models coming out, like this sudden y i i I mean for a couple of years there it was just a few, right? It was Gemini and and Claude and ChatGBT and suddenly we have ten trillion models and I think one thing we're gonna start seeing, this is me Christopher Slowe: Double cameras, yeah. Zach: my god. Mm-hmm. Marty Weiner: Trying to guess out the future is it reminds me of the CPU wars of like the nineties and and also the the console wars. You know, we had Nintendo 64, and every year they would just try to bump up the number, or they would try to bump up the frequency. But then you start to run out of that headroom. And I think we might be seeing that with frontier models. Like, yes, there are these top tier ones, those are gonna continue fighting on the frontier, but then I think what we're seeing is these other models to serve a specific market. So we're seeing you know, the this this one from thinking machines that can take in video directly as well as audio and and do something with it. but we're also seeing a bunch of these simpler models, just like with the CPUs, you we saw CPUs suddenly go for mobile, so we needed lower power and smaller. We saw CPUs go to things like simple microcontrollers for just doing you know, operating a small machine. we saw a move toward other areas. We saw NVIDIA, three D FX SGI, all these companies pop out of the ether to go in and and do graphics. And I'm wondering if we're gonna get to a spot where we have more and more of these models focused on more narrow areas. yeah, the other one we saw was NVIDIA's robotics one. And so I'm I I think we're gonna start seeing I don't know, is diaspora the right that's not the right word. An explosion s sideways of different models. the Zach: You Christopher Slowe: Prec a pr a Cambrian explosion of models. Marty Weiner: I can't Zach: There we go. Marty Weiner: yes, thank you. That that sounds way smarter than what I can say. Christopher Slowe: well I I think I look I I'm gonna go back to I'm gonna go back to distillation. Distillation, I think, is the magic here, right? Distillation means you can take a, you know, distillation is the way where you can take a fairly complex foundational model and you can mint a fairly specific, robust, specialized model that's a lot smaller because you can kind of take advantage of all the shortcuts. Zach: Whatever that is. Marty Weiner: yeah, that sounds great. Zach: Can, can, can like, mean, presumably this is disastrous for Anthropic. If like four seconds after fable comes out, you can just distill it. Like, do they have, is there some way they can like poison the distillation? Christopher Slowe: Well I think it's I think it ends up being it's one of those things where like it's it's it still requires a fair amount of training, but you're talking about potentially like an order of magnitude less cost. and yeah, for example. But I do think it like Zach: Yeah, so like tens of millions instead of hundreds of millions. Yeah. Marty Weiner: Well Anth Anthropic they were specifically trying to do things like you have to use sign up with your driver's license or or use persona or these other things to make sure that you are I think they were saying so that we make sure you're not doing anything illegal, but I'm sure a side effect of that was don't distill my model, damn it. I stole these things first, fair and square. Zach: Yes. That's right. Christopher Slowe: Possession is nine tenths of the law, Marty. Marty Weiner: That's right. we should get to my favorite part here, the math. Zach: Yes. Ooh. Christopher Slowe: Yeah, we I mean we we thought we thought last week that there was some really cool stuff coming out of these models in math. And then lo and behold, the the the Jacobian conjecture has been overturned as I believe a side activity that someone was working on during a World Cup game. Marty Weiner: That one. Yes. I'm gonna share the tweet because Zach: you Marty Weiner: it is the most glorious tweet possibly of all time. And yes, I am using the word Twitter. Zach: Yeah, I like to imagine they had like buffalo sauce on their fingers as they were typing it Marty Weiner: Yeah, that and maybe they they used up an entire energy drink. I don't know. so of the most probably glorious tweets of all time, hello there, the Jacobian conjecture is false. Thanks with an X. not even KS there to my close friend Ackill for asking about it, and my other close friend Fable for working during the World Cup final. There's the answer. Zach: That's right. You Whoa. Christopher Slowe: Ha ha Marty Weiner: I don't know what the hell it means. I I spent about five minutes trying to remember this stuff, but it it's been some time. But a a friend of mine, John Wise, who's at PhD at professor at University of Colorado Boulder, who's he's also an algebraic geometer, which means I don't know what what what he does because it's very, very smart. He said this is a big deal. and I think that the you know, with this comes I think the a good portion of the math Christopher Slowe: He's very smart. He's extremely smart. Yes. Zach: Yeah. Marty Weiner: universe also starting to wake up. Because not just this. What the crazy part is this is one of many, although I think it's probably one of the most impressive. but in the last couple of weeks, let's see, we've had the fifty year old cycle double cover conjecture solved. By the way, I don't know what the hell these are, but they sound really smart. Yeah, yeah, it finally went down. Sh I I've been working on that all my life. trying to figure out what it means. Zach: Mm-hmm. Yeah, just... Christopher Slowe: Not the cycle double conjecture. Zach: man. Marty Weiner: Then there was a big one for with st by the way, that was a fifty year old problem. Then there's a big one in statistics that that just fell. another big proof that I I guess I I g I guess I don't know what exactly these models are doing that that no human's been able to figure out. But it sure as hell sounds like it's one of those things that I we Chris, you and I talked about with chess, where we are taught by other humans how to how to Zach: Yeah. Yeah. Marty Weiner: How to play, right? And so the example I always love to give is in chess, we're always taught don't bring that rook out. You know, early on when you're learning chess, you learn the rook is really powerful, so you try to get it out. Well, to get a rook out for anybody who doesn't play chess, it's three moves. You gotta move a pawn up, you gotta move your rook out, and you gotta move your rook over. And we're always taught don't do that, because that's three moves, in which case the other player can be destroying you, you know, getting advantages. But then the AI started doing it, and then the humans were like, shit. Zach: you Yeah. Marty Weiner: This there's a real strategy here, and I wonder how many things could Zach: Well, there's also like, academic social structure. it's like, you I feel like we act as if mathematicians like our people whose job is to crawl the intellectual landscape or something, but actually like an actual mathematician is maybe a grad student who has to make a choice about what's going to get them a job. And so, you know, it's sort like you think about like, haven't we, why don't we have a grand unified theory of physics? And at least part of that is like, there's not that many physicists and very few of them are, you can't just be a grad student and be like, I'm going to work on crazy stuff that probably won't pay off. Like you have to pick something you can do that, will get you the next job up the rung. Marty Weiner: Yes. Zach: So I wonder with some of this stuff is it's like, if there's some old conjecture by AirDish everyone thinks is true, like being like, well, maybe I'll just futz around looking for a counter example for two years. Maybe that'll be great. Like that's a really risky strategy. Very few people are gonna do that. Whereas like AI can just do that all day long. Christopher Slowe: Well, I do I do want to rant about this particular the Jacobian conjecture one I want to rant about for a second because this is a like we we talked about this last week, where we're we basically went through and said, like, you know, look, these models are reasoning, but they're also synthesizing. And so you can start looking at the corpus of like all of what hu humans have written and maybe make patterns. And so because this is this is just there's always this debate about like, do LLMs think? Right? And of course it's you're getting into like qualia and philosophy and all that fun stuff that is, you know, kind of painful to argue about. But you could you could make an argument that All the LLM is doing is it's just kind of assembling ideas that are already out there in the wild that humans have generated somewhere else. In this particular case, for this for this conjecture, it it found a counterexample. And to find a counterexample, like there's a brute force method to find a counterexample, but no one's tr like it's just this this space is too large to explore, right? And it's almost like and just to get like let me try let me try to let me try to try to plain terms what the what the Jobic Cobian conjecture actually says. So basically, when you deal with functions in higher dimensional space, there's an analogy for the derivative called the Jacobian, which is basically you do the derivative for but for every single dimensionality of the of the Marty Weiner: Nobody's done it. Yeah. Christopher Slowe: of the function. And the Jacobian conjecture just says that it's possible to have a that it it that if you have a if you have a function like this, you take the Jacobian which ends up being the the determinant of the Jacobian. So you end up turning this this matrix into a number. So already going deep in the math here, but bear with me. It you can bullshit I know. Marty Weiner: Mm-hmm. Christopher Slowe: That if that ends up being a constant. So in the case of like in the case of the the the the the the the one-dimensional case is easy. If you have a if you have a constant derivative of a function, you know that function is invertible, right? It's a line, right, in one in one D. if you do that in two D, it hasn't been proven yet. For greater than 2D, this disproves it. They found a function that has a constant Jacobian that's not invertible. Basically, they and the way they showed that in the tweet was they said, here's the function, here's the Jacobian determinant, which is computable, it's negative two, and here are three points which map to the same point, which means you can't invert it. Right? It's beautiful because it's so straightforward to dis like it's one of those it's a one shot it's it's it's it's it's like encryption. It's like it's it you i if you don't have the keys, you can't you can't find them. Once you have the keys, you can trivially prove that you have the keys. and so but but but Marty Weiner: So simple. Yeah. Right. And the this guy did it while watching the the the World Cup. Well what if we don't have a World Cup to watch? Zach: you Christopher Slowe: But I'm f but how? But how but how? But how like how did the model how did the model do this? Zach: Yes Marty Weiner: Yeah, how did it how did it do this? Zach: Did you try there's a thing going around the internet and I tried this. was amazing. Just take your your LM and put it on like incognito mode so it won't think you're crazy and say, hey, I just solved this conjecture. Could you check my work? And it will kind of flip out. You just you just post the solution from the tweet and say, I just I was just, whatever watching the World Cup and I solved this. It was I said it to go on, I think four point eight and it went for like I eventually gave up because it kept being like. Marty Weiner: Ha ha. Zach: This seems to be true, but it can't be true because this is a major open problem. So I'm going to verify again and it'll just keep going in circles. Like you can make the AI go crazy. It just won't believe. I I assume if I let it go long enough, it would be like, I guess you solved it, but it really wouldn't believe, kept thinking it was wrong, which is kind of fascinating. Marty Weiner: Ha ha. Huh. So is this the way we're gonna defeat the Terminators? Be like, Hey, by the way, got a solution for the Jacobian conjecture? Y no. Zach: Just, yeah. A human salt and open conjecture. No. But so maybe the point to me with this stuff is kind of like, maybe a year ago we started seeing what you could argue were low hanging fruit, air dish conjectures. Meaning as I understand it, the kind of thing you'd give to a grad student and they could probably solve. And then maybe six months ago we started like maybe once a week you had some existential dread. And then this week there was like seven major things solved. And the way I think about this, and I think actually someone could build an app for this maybe. If you did sentiment analysis on Reddit slash math posts and just. counted up the level of dread. The level of dread has been increasing exponentially. Like there was a post and a guy literally said like, you know, what was the point of the last 30 years of my life if now it's gonna just solve everything, which is very different from what they were saying a year ago. Marty Weiner: Yeah. Yeah, I think it's it's crazy how fast it's coming. I mean we see this also with coding, right? Like I don't code anymore. I I completely vibe code. and we're seeing this with VFX. I think well actually that was some of our other news we should talk about. we have what did George Lucas say recently? Zach: should be out here. He's pro-AI, in fact, he compared people who are not in favor of using AI for movies to people refusing to give up the horse and buggy. I do wanna say, have you seen the prequels recently? I just wanna say, if George Lucas says it's time for this technology, look at the composites on episode one and see how you feel about that. Marty Weiner: Okay, well wait a few years. I w I I think one of his points was he was always using the latest, you know, the last he he was always pushing the edge to get that thing on on on the screen. And his goal was I'll use whatever I gotta do to get something on the screen. I'll give you a comparison point that I thought was hilarious. They didn't let Tron, the movie from nineteen eighty two, they didn't let them win an Academy Award because they use computers and that's cheating. and it's just so short sighted Zach: Yes, yes. Mm-hmm. Marty Weiner: so you know, y you have Jurassic Park, that was a big deal. Suddenly you went from practical effects and now computers are being used to make a major motion picture that that looked great. I I think I think why would we limit ourselves to what we're allowed to to get to get a great story onto the screen? so I think that's what George Lou is getting into. It sounds like Netflix is also going this direction. They're using a lot of generative AI. Zach: Yep. Yeah, yeah, they actually said that in 300 of their shows, they used AI in some form. My sense is some of it's kind of light. There's like, we have a battle sequence and there's an army and could you make the army look a little more badass and like bigger, that kind of thing. There was a show, this was actually a little while ago, we didn't talk about this show called House of David, which I think was like a Christian thriller or something or other. But anyway, I haven't seen it. But like, they use just straight up generative AI for these like big fancy effects sequences, I think involving like, you know, the Lucifer falling out of heaven, that kind of thing. Marty Weiner: Yeah. Zach: And it's just straight up AI. I actually looked at a little bit of my impression was it didn't look that good, or at least to me, it looked like AI, but I kind of like, I don't know that it matters to a consumer, like there was some like minor internet backlash, but apparently the show is quite popular. Well, yeah. Marty Weiner: Right, right. Well g give it another month and it'll look perfect and it will solve a another air dish problem. but th just Right. Zach: Yeah, well, it was calculated. Yeah, this Lucifer solves on the way down. Yeah, I mean, think the pushback on this is like, one, it's like, it's gonna cost a lot of people jobs, though those jobs are already going away. The jobs are going away. The concern I hear from a lot of artists, which... Marty Weiner: That's right. Yes. Yeah, yeah. Zach: I guess my feeling is it's like legitimate, it's just going to zip by anyway. It's like you could have said like when CGI was happening and there were like old fashioned VFX people being like you traders have ruined our business. At least you could say like, let's CGI we're creating their own dinosaur models or something. One of the complaints here is that like these models are trained on like the history of CGI and the history of like, so it's a one thing you see, for example, like on Reddit AI slash video, people will, for example, like make a movie with dragons. And it's funny because the dragons don't look like whatever a real dragon would look like, they look like CGI. So clearly in the training data, they're operating on like something someone designed at some point. so like, I think it's a legitimate I mean, I don't know what you would do about it. And legally, I don't think there's a case. But there's a complaint that like, well, this is just like plagiarizing the work we did before. Marty Weiner: So I I have two critiques of of those kind of arguments. one is more of a I don't know if the word is fatalism, but it's like it's it's coming. It doesn't matter what you do, it's coming. So the VFX, I love V I used to do VFX. I love it. It is sad to me that this is gone. It's really fun. I used to be really good at After Effects, all these things, but it's just going. and I think the other aspect I think about is this is a little bit more to this argument that Zach: Yeah. Yeah. Marty Weiner: That the AI is stealing and and and the the the one I I keep thinking about is the AI is taking from billions of of you know artists and then producing something. I get that, or millions. but then I I keep going back to this case where Bill Waterson was well known for having influences off of you know Charles Scholes, I have to look these up 'cause I forget him, Walt Kelly, Windsor McKay, George Harriman. and that's Zach: Windsor McKay was a big one, I Yeah. Marty Weiner: three or four influences. And I know I always thought that was a funny word. Is it influences or what was the difference between influence and stole? but Zach: Yes. Well, you have a Winsor McKay vector and you have a George Heyerman vector and you just do the sum. Yeah. Yeah. Marty Weiner: That's well plus another million and you have AI and so I just I I I wonder I I have I I find that argument to be a little challenging when people say it's stealing. Like, yes it is, but so are like how else do we produce art as humans? Like we we see what and we Zach: I I agree. think the stronger complaint, which I would make is a kind of like, well, the way we design IP law, like IP law is not a natural human thing. You don't go into like the Amazon and there's someone and you're like, well, how do you get royalties on your, you know, aero design? Like, tell me more about your culture and how it handles royalties, you know. Marty Weiner: Yeah. Christopher Slowe: I discovered that species of Marty Weiner: Wait, wait, wait. That's Christopher Slowe: blow dart frog. That's my that's my poison. Marty Weiner: Yeah. Zach: that's right that's right maybe someone did maybe there's a primitive i don't want to say primitive a different ip rights division in some on uncontacted tribe Christopher Slowe: Traditional. The traditional IP rights, yeah. Marty Weiner: Tr tr developing. Zach: But my point point being we like we IP law is like a thing. We mostly invented like in the 19th and into the 20th century basically to get better stuff or at that's the theory. And so it's like the question is like it's absolutely true to say like I don't think any artist has a legal case here. It's clearly transformative. And it's true that like all along, you know, I could produce quotes like Tennyson, like a poet's poet said something like, well, the real test of a poet is what he steals and how he steals it. You know, like so we have been saying this all along. I think the concern is like Marty Weiner: Right. Zach: or a more legitimate argument would be like, if everything switches to AI, do we still have an ecosystem to produce new stuff? Like, it's kind of like the similar to the junior dev argument, like, are we pulling up the ladder if we don't find a way to make sure that the people who make the stuff that goes into this stuff get paid somehow? I wouldn't say who, because they'd get in big trouble. was talking to a very good artist friend quietly at a bar, I was talking to him, and he was like, you know, all my friends say this AI art is ugly, and it's not ugly. Marty Weiner: Mm-hmm. Zach: I just want to get paid. And so the question to me is more like, how do we set up an IP ecosystem? And maybe we don't, maybe they are so good. just like artists becomes a job like Weaver that nobody does anymore except to amuse people at a Renaissance fair. know, it would be nice. Oh, go ahead. Marty Weiner: right, right. Yeah. And I think I think there's something very real to that because we just saw you know the Odyssey come out, and it looks amazing and they did it on original film and yes they do use CGI, but they did a ton of practical stuff and and I think people still will love I certainly love to consume stuff if I s you know, if you say yeah, it was made all with a on original film and with a human Zach: Yeah. Marty Weiner: That's pretty cool. and the the world's top movie right now is still being made with a lot of that stuff, even though d most films are made on digital these days. Film, you know, mostly died a couple of decades ago. and practical effects have been on the way out a long time, but I think it's still shown that people I I don't know, it's special, right? It's like something being homemade. Christopher Slowe: Well think you're I think you're back to like, you know, it's it's it's the it's now become artisanal. Right? Like you have to be you like they they they made an active choice to go back to to film. And of course it's a it's an all star cast and a fantastic, you know, production group and they know what they're doing. but if you're gonna talk about like most most of the industry will will will be using digital because it's easier. And because that's where the ex that's where the expertise is. And so you know you're almost like I I think the weaver analogy isn't is a is a good one. It's like, yeah, you can get some really awesome like woven things if you enjoyed. Zach: Yeah. Marty Weiner: Right. We'll be digital CGI. Right. Cheaper. Right. Zach: Right, that's the thing. Christopher Slowe: It's way better than what you can buy off the rack, but you have to like know a weaver and it's gonna take a whole bunch of time. Right? And yeah, and a lot of money, yeah. Zach: And a lot of money. Marty Weiner: A lot of money. A lot of time, a lot of money. one argument I've also been looking at is around the case law and the copyright of all this stuff because I believe I don't want to get too rat holed in this. Maybe this be a discussion for a future one, but what is the theory of if this stuff should be copyrightable? And the I my understanding is the US PTO office does make it difficult to copyright AI, but I my gut is this has to change and the my basis for this is this fun fact I uncovered. So if you go back to the eighteen hundreds when photography was new. Zach: Yeah. you Marty Weiner: it was not copyrightable for a while, partly because it wasn't believed to be creative. You hold up camera and push button. Which, you know, sounds really silly for today. The it actually the case law that broke it was this case Burrow I'm not sure how to pronounce this Burrow Gilles lithograph lithographic company versus Soroni, 1884. and so that's when they were shown that no the the that this does have enough creative power in it. Zach: Right. Marty Weiner: and I guess the thing I've been thinking about is if I'm a if I'm a d badass director, George Lucas, and I've got a team of the world's best artists around me doing CGI and matte paintings and all this stuff, and I so I'm I'm a one of the world's greatest creatives. And if I were to pull out those other nodes, the other, you know, the mat painter, and I replaced it with AI and I that person with AI and I r did that with my CGI artist and and I kept replacing them. I I guess what I'm trying to figure out is does that actually remove any of my individual creativity or h is it the exact same I'm just now enabled because I can be like George Lucas sitting at home? and th this that's what I've been fighting with a little bit, trying to understand. Zach: Yeah. Well, I actually think like, go ahead. Yeah, yeah. So I had an interesting conversation right when all this started with a with Again, I shouldn't say but like an editor in an important publishing house and I was like, well suppose someone came to you with like a really good comic but it was clearly AI generated, assuming that's possible right now. It was like no way. And I said, said, well one, everybody who worked for us would be furious. So there's a cultural aspect, but also he's like, we don't know what the rights are. Like, so we can't publish a book we don't know the rights to. But again, to me, this gets back to IP stuff. Like you can say, well, what is the philosophy on what sort of entity is allowed to have rights and what is the philosophy on? Marty Weiner: Yeah. Zach: on like how much of a contribution do you have to the creative before you can copyright. But I think really what you want to say is like, well, what is the policy that would work best for like getting good stuff? Yeah. And I don't know what the answer is. Marty Weiner: Getting good stuff. Yeah. Well and recently you were mentioning about Netflix and AI and Netflix recently bought Inner Positive, which is Ben Affleck's company, for five hundred and eighty seven million. Let me make sure I got that number right. and and I'm wondering about I guess Netflix then clearly to spend that kind of money has a path to feeling either confident that they have the IP figured out or they don't care. Zach: Yeah. Yeah, it was big, but yeah, yeah, yeah. Marty Weiner: Me I'm wondering if we're going that direction. I mean I I can't imagine a them not caring but Zach: Yeah. Well, I I, I, I've thought for a long time VFX is first on the chopping block for artists because it's like, because there's so much money, but also like nobody, maybe you guys, but almost nobody goes to a movie and is like, I got to see this because the VFX supervisor. Marty Weiner: Yeah. I I totally go there for the VFX rent No Zach: you yeah, maybe you do, but nobody else in the world like this. So where whereas like you do say, well, I want a Christopher Nolan movie or I want to see Brad Pitt looking Brad pity, you know, or I want to buy this painting from this person like so that's gonna be a little harder to disentangle or more to the point you don't want to disentangle it, I don't think. But also like, you know, like, like replacing an oil painter is not going to make, you know, money for anyone, whereas replacing a VFX department or shrinking it drastically would. Marty Weiner: Right. Christopher Slowe: What's the other? Well, I do think though that there's like a the the the the analogy with photography feels really apt here, which where it's like there's just so many d like just boring derivative photos. Like it I mean look at influencers as a culture right now, right? Does everyone take the same freaking picture over and over again with the same like, you know, po pose pushing leaning tower pizza. right? There's the but but on the flip side, if you're actually if you actually are good at the craft, you can make some really compelling photos. I think it I think it works the same with visual effects, right? Visual effects starts off with being something Zach: Thank Marty Weiner: Mm-hmm. Christopher Slowe: that you you you're not just gonna like you know more dragons add more dragons and then just have it you know it's like look it made it one shot perfect exactly what this looks like there's some Zach: Listen, I, you know, it's funny if you, if you browse the AI video Reddit, there are people, I was joking about this, like, like there are these fantasy movies you see people working on that if you titled them would be something like the big boob wizard ladies all have awesome swords. Like that's going to be the future. But the real horror of this is not that that's going to be what comes out. It's going to be that the public loves it. And people like me, you're going to be like, no, you were supposed to make, you're supposed to make art and, and, and, and big boo wizards part seven is just Marty Weiner: Wait, where where Alright. Zach: destroying Christopher Nolan, he's nothing now. For Big Booboo District 7? Yeah. Marty Weiner: Wait, Zach, where's where's the link for that video? I'm ex for research. Christopher Slowe: Yeah, what's the what's the URL for that if we're just for for research for the next podcast. Marty Weiner: Yeah. I wanna make sure it's bad. Christopher Slowe: And you're saying there's at least seven of them. Zach: you Yeah, you'd hate to start and not know there's a series. All right, maybe we should move on to, that's a good capper on, yeah. Christopher Slowe: Let's talk about Marty Weiner: Well we got in the world of people saying, Hey, AI is cool, we Linus Torvald recently. said, Linux is not anti AI. but I I think he is making a very clear point that we're not going to limit Linux from being able to run with AI. I'm not sure he is pro AI, but he's definitely not anti AI. Christopher Slowe: it. Zach: Is that even a thing you could do? What does it mean to say Linux won't, like meaning they won't use it to build it? Marty Weiner: I think it could be build it, vibecode it, or or I don't know, Chris. I'm trying to dream of what this might look like, but I mean I could imagine a kernel module where it is operated by an LLM running the internals of it. We're already seeing, by the way, you know, for the nerds out there, compilers being rethought as essentially you know what a compiler does is it takes human written language, turns it into computer speak at the most basic level. Christopher Slowe: Yeah, I Zach: Mm-hmm. Marty Weiner: and we're seeing LMs start to get into that space too, where the LMs do a better conversion from human speak into into computer speak. Yeah. Christopher Slowe: Pinary. Yeah, I mean I d I do think the start like I I initially read this as there there is a fair amount of backlash in the open source community, especially of, you know, the as as it's gotten easier to write code, the number of pull requests, which is the unit of submission to somebody else's project, have gone up markedly over the last couple of years. To the point where, you know, the you know, there's there's an ancient XK C D joke about, you the entire internet's held up by one one random open open source model, open source project. Th there's there's generally like one person who's kind of sitting in there, like, you know, working on this in part-time in Kansas who is like running a foundational package that everyone uses. And they're just getting overwhelmed with pull requests because they have like they want to review it, they wanna keep the craftsmanship kind of high. and so there's been a bit of a backlash on and and you know, mo most of the most of the harnesses that are used to generate that are, you know, gener used to generate vibe code. I don't know if I vibe code is the right unit, but we'll just use it for now. we have to figure out what the unit of vibe coding would be. but well Yeah, that's like one the the Marty Weiner: w w one one energy drink worth, I think. Zach: Carpathes. Yeah Christopher Slowe: They tend to also label themselves when you put up a pull request. Like it'll say this is like powered by cursor, like, you know, pick your pick your pick your pick your rig. And so there have been entire open source communities effectively that have said like thou shalt not submit vibe-coded code. And so here is here's l here's Linus, who has been one of the most awesome contrarians of all time in the software industry. Like he just he just speaks his mind and he generally is correct. he's just basically said like Zach: and Marty Weiner: Uh-huh. Christopher Slowe: Nah, we're gonna have like an anti. Like we're not gonna we're not gonna discount this entire technology just because there's like we haven't figured out how to hold it, to paraphrase them. and now I expect to get tweeted out saying that I got that wrong. Marty Weiner: Yeah. I I think that that's right. I think that's a good cultural s stance to take for Linux because I could easily see Linux becoming the next, you know, having their own Kodak moment saying, No, we're gonna stick with film, and and digital cameras destroying them. I think that Linux could be the next one to fall if it didn't keep up with this. I want for our last little bit here, I really want to hear Zach's reporting on LinkedIn posts being made by AI. Zach: You Yeah, so this is just a little thing out from Pangram. Pangram came up on my radar because they've actually... So lately a bunch of literary prizes have seemingly gone to AI. And so... There we go. Yeah, so Pangram is a software that detected this. Now you can debate, right? How good are they at it? I don't think they've released their stuff because I think if you release how you do it, it's very easy to fight against the detector. Marty Weiner: oops. Zach: But so they ran this on all these different social media platforms and it turns out 41 % of LinkedIn content is fully AI, not like you got some help from AI. So I was shocked it wasn't 80%, but we're much too small. Interestingly, it seems to correlate with length. The longer the content, the more likely it is that it was just pure AI stuff. Actually, to be honest, I don't know how impressive this is because like, Marty Weiner: huh. Christopher Slowe: Seems small. This seems very small from what I've read. Zach: you know, probably a decent amount of the stuff in the set is just trivially obviously AI. But but but kind of interesting because like this was obviously not possible a couple years ago. And now it's like a third of some platforms if you were picking at random, you're getting something that was just AI generated. So I expect this to just increase and increase until it eats some high percent of all internet. Have you guys gotten, I've gotten to where I get suspicious. Like it gets weird. was just on, there's a Reddit called realerai and there was a post where a guy was like, I paid an artist to do a commission of my anime character and they got back to me in 15 minutes. So I think it's probably AI. And I looked at it I was like, No, it's just a good artist. Or at least it's not if it could have been AI, but it was like, there's no tell. And so I'm just getting to where you're like suspicious, anything that looks quality, you're like, I don't know. And so I the dead internet theory is, is becoming no longer theory. Christopher Slowe: Definitely. I yeah, I I've I've definitely s I've definitely seen some Photoshop Battles artists like video their workflow. And some people are just really, really good at that tool. Like I think I think it's actually entirely especially if it's your job, it's like you're just gonna be able to do it really well, really quickly. Zach: They're just really good. Yeah, mean, and this is also like one thing I tell people with this is, you know, it's not always going to be possible to detect because what you could do, I mean, I'm not a good enough artist for this to help me, but like, you know, there was a kerfuffle recently where DC Comics pulled a cover because the guy got accused of using AI and then he provided sketches. But the problem is you could, of course, have AI generate sketches. And at the end of the day, you know, You could always just be like, draw me pencils and I'll do the inks. And the only way to detect that would be to live in a world where you have to turn over your hard drive whenever you submit a project. Maybe we'll get there, I don't know. But it is causing problems. There's actually a page, go ahead. Marty Weiner: But could I could I use AI to make a forensics you know path for what I mean this I think this is a very, very real problem. What is actual and what is real? I I wanted to show this example real quick. I know we're a little out of time here, but I ran into this today. I was looking at this cool behind the scenes photo from a famous Seinfeld episode, and then people people were saying, so if you're familiar with this episode, it's setting up for this photograph, and I thought this is really cool. Zach: Right. Yeah, yeah, Marty Weiner: Wait a minute, is it just AI? 'Cause it kinda looks framed like AI, right? You got you got George here looking a little miffed. That would I don't know if he'd be looking like that if they were behind the scenes and I I don't know, it just looks like it could be AI. But what I found was this similar one from you see the time zone, ten years ago. So we almost have like you know, there's this phenomenon where if you want really clean steel for our science, you have to go get steel of sunken ships before nineteen forty five when there were nuclear explosions, because that's the only steel on earth left. Zach: huh. Right. Marty Weiner: that I guess you can get to without having it be contaminated by nuclear fossil fuel fuels sorry, but by nuclear fallout. and I'm wondering if we're gonna have a phenomenon here too. We sort of have this line in the sand, at some point around twenty one where anything before that almost surely wasn't made by AI. Zach: huh. Yay! Christopher Slowe: Well, I mean there was a there I I did s I did see somebody disc you know, I don't know if it was i pose law is a real thing. and somebody was discussing the fact that they had found a bunch of dashes in Agathr Agatha Christie novel and therefore like, how did Agatha Christie have access to an L L Marty Weiner: Yeah. Zach: You Marty Weiner: that's good conspiracy theory stuff. Christopher Slowe: but I mean for this it's like Zach: Well, but also like like presumably part of what's going on is Agatha Christie is is other than God like the most popular author ever. So presumably this was like an important part of like English language writing that the the right. Christopher Slowe: Here's here's how to write good English is to include this this kind of punctuation. I I don't know, the the the tricky part with this is with this, especially on the long form side, is that like or you know, if if you think about if you think about LinkedIn, LinkedIn's already fairly formulaic in sort of like the way that the the posts end up being generated. Also, there's like this kind of there's a chicken and egg problem of as we consume more content being generated by LLMs, our writing style is gonna naturally shift towards LLM y kind of, you know, and and and all we're also becoming more more aware of it, right? Like I've I've definitely seen the You know, in in prose, it's like you you see a lot of these statements that will start off like, it's not just the blah bitty blah, it's the blah. And the and it's like that that and then with that plus the m-dashes and just like some sort of like a little bit of a general smarmy tone seems to kind of like you know appear. and unfortunately smarmy tone and LinkedIn go hand in hand. Marty Weiner: Right. Zach: Yes. Marty Weiner: So Chris, what I Zach: Yeah, there's actually, there's a really good article. I won't get into it too much. It was interesting because the guy was saying like, you were going to go into this world where everything, you know, a lot of stuff is AI. And there's going to be people who are like, I will not use AI. And there's going to be like mixed cases. But the interesting thing is he was like, listen, you know, if you say I'm not using AI, you're already kind of being messed up by or influenced by a because now you have to do stuff to differentiate from AI. So even if you're like, I'm good, you're not going to be like a pure strand of writing as if there hadn't been AI, you're going to be like, I'd better not use that construction. I better not use M dashes. They're not doing all this other stuff. So human writing might, you get, you know, you'll get people who like just gravitate towards AI because that's what they grew up with. Or you're going to get people who are like, I'm going to do whatever AI is not doing right now. You'll get a totally different writing style. Christopher Slowe: It'll be the equivalent of why are you using punctuations in text, Dad? Like what did I do wrong? What's with the c why are you capitalizing things? Marty Weiner: God, I'm s I'm I'm so nineteen and nineties. Yeah, quotation yeah. yeah, what's the point? Zach: With your quotation marks, yes You Marty Weiner: Chris, as the host, I was hoping maybe you I'd like to hear your exit for the podcast, but I would like you to use the phrase as an AI language model and I'd like you to use the I guess verbal equivalent of dashes. Let's hear it. Christopher Slowe: This really hard. I think it's gonna be easier for me to actually write a sonnet about a cheeseburger on the spot. I'm not sure if I can do this. this no, I think that's gonna probably also fail. but well, let's try it. hey, it's not just that we got through the short news without anybody being injured. We managed to talk about LLMs, open source to open weight models, and why Linus Linus Torvalds is once again right. Marty Weiner: Okay. Can we do that? Zach: That's not just efficiency, that's bravery. Marty Weiner: Excellent. Christopher Slowe: It's just good s it's just good strategy, Zach. Zach: Yeah, Marty Weiner: Now how how do you Zach: that was a load bearing comment there that you delved into. Christopher Slowe: Yeah. Marty Weiner: Excellent. and now how can you use L to to tell people to you know where to find us? Christopher Slowe: Okay, so next week we're gonna be next week we're gonna be doing the entire podcast with an LLM model telling us what to say back and forth to each other. It's gonna be it's gonna be exciting. I pr I predict actually, I predict at some point we will try that out, and it will be hilarious. Marty Weiner: Love it. Okay. Zach: god. god. Marty Weiner: All right. I think we should try it next week. All right. Until next week.