Latest / Last Humans in the Loop / Inside Claude's "Mind", AI Solves 50-Year Math Problem, Apple sues OpenAI, Recreating Voices with AI

Transcript

Inside Claude's "Mind", AI Solves 50-Year Math Problem, Apple sues OpenAI, Recreating Voices with AI · Last Humans in the Loop

• Zach: Hello. welcome to Last Humans in the Loop, the podcast that has a title now. I'm gonna be the host today. I'm Zach Wienersmith, worldwide celebrity. I'm here with with my with with Marty Wiener, who is CTO of some stuff. He's best known as being my brother. and I'm here with Chris Slow, CTO of Reddit and expert in rubidium lasers, I believe. Is that right? Marty Weiner: Yes. huh. Christopher Slowe: I bless that was close. It's close enough, yeah. Zach: Yeah, lasers. all right, so we have a bunch of stuff to talk about. and we're going to start off with the news and our first item is that Apple is suing OpenAI. Chris, why are they doing that? Christopher Slowe: well it sounds like it sounds like what's happening is that there's been an aggressive push at OpenAI to start to build a device. And as you all know, some of the best devices out there come from Apple. and it sounds like what's happening is that in addition to a large exodus of Apple folks to open AI, which is something that doesn't usually happen, Apple tends to keep people for a very long time. In fact, they're unusual in the in the Valley in that you find lifers at Apple who've literally like legitimately worked there for twenty-five, thirty years, never been out, perfectly happy. but there are some Some rumors that they are also being told to either bring along or have in their mind the prototypes that Apple's building, which is definitely on the edge of a no-no. now the weird thing is that for those of you who don't know, it's actually I believe non-competes are not enforceable in California, which makes this an interesting move if they're gonna sue on that side. If they're gonna sue on some sort of like stealing of trade secrets, it should be an interesting court case. Zach: So can I ask a stupid question, which is like, wh why does anybody want an AI device when you already have a phone that runs all the AI? Christopher Slowe: well I think we've all seen her, so I think the impression that they wanna have is what's the most tiny awkward device you could have instead of a phone with no screen? I don't know, actually no. I th there's been a couple of these that seem to be popping up recently by the vendors. Now I think I think the the simple answer would be maybe OAI wants to start off by just building an a f building their own phone. and that way they control the stack all the way end to end. If they go to that route, I'm not sure why they wouldn't just go the route of, you know, n another yet another Android phone, out in the market. I I do think there's a push towards these kind of like unobtrusive little devices. Like I've seen a couple where it's like effectively you're walking around with a camera on you. It's like you know basically walking around with a GoPro, which people already do and is already pretty creepy. So I'm not really sure what they're gonna get out of having their AI agent kind of constantly watching what they're doing. that said, this feels like the kind of this is like the kind of discussion that in like about two years when the device comes out, we're gonna like, Well, we really called that one wrong. Like, who knew they could put it all the way Zach: Yeah. Yeah. Christopher Slowe: Mm-hmm. Zach: my god. so yeah, yeah. So yeah, I I I th that's the other thing I don't understand about this rule, because I used to say like California is like anomalous in that they don't enforce non-compete agreements. Like so Texas and Florida do. this different model and that's that's an argument for why Silicon Valley remains in California. But I guess I don't know like when you say open AI poaches employees, presumably they don't all live in California. Christopher Slowe: Yeah. Marty Weiner: I'm ready. I'll do it. Christopher Slowe: Well, I think that the court case is actually happening in Northern California, which makes it extra interesting. I mean both both companies are primarily based here. and as a side effect it makes sense the court case would be where the headquarters are. if anything, I I I know that both both have had a pretty aggressive back to office policy. opening I is pretty much centrally housed now all in one place from what I've seen. so it's kinda not surprising in that regard. But but yeah, no, it's it's a it's a it's an interesting case. I mean Zach: Interesting. You didn' Right. Yeah. I mean what was weird to me is there was a claim that the the one of the guys that went over to OpenAI was like not only just talking about it, but he like saying lol as he was taking trade secrets. I don't know. I don't know how that holds up in a legal setting. I mean maybe you know, I don't i if i if non competes aren't enforceable, presumably lolling about them is okay too, but I'm not a lawyer. Christopher Slowe: Well, this is usually why too when you when you do these kinds of I mean there's there's there's pretty well trodden like here's how you reverse engineer a thing. because you know, if you want if you don't want to get in trouble for violating some sort of like a gag order or like a trade secret order, then generally the way you do it is you have this like kind of information break where you have like one group of people who are sitting down writing specs from scratch, taking apart a device, and another set of people who are over on the side reading the specs and rebuilding it. And that way you know for sure that there was no like other than what was written down in the middle, which is which you can have a lawyer involved in, there is no other information being used that could have been a violation of some sort of like trade secret. But it's a little bit different when you're talking about I think the latest Natalia heard was several hundred Apple employees migrating towards OpenAI. Zach: Yeah. W which also suggests like either Apple is doing something wrong or OpenAI is is doing something right, or at least that's the perception among engineers. Christopher Slowe: I mean Apple Apple has a notoriously I mean like I've I've heard it described as the the the way that they run the way that they run projects are equivalent to like terrorist cells where each each group of people working on a project doesn't actually know anything about what anyone else is working on and there's a bunch of c there's code words for everything, right? So like they're very good at keeping a lid on their tech. I mean, especially in this day and age, like considering like how little how little information gets leaked about devices from Apple with ubiquitous access to Marty Weiner: Yeah. Zach: Yeah. Christopher Slowe: everything all the time, everywhere. It is an impressive culture to be able to keep a lid on it. Zach: Yeah. Marty Weiner: I've had a few friends leave Apple 'cause they they were just driven nuts by the secrecy. Zach: Interesting. Christopher Slowe: Yeah I think on the way out they lobotomize They a drill right up your nose and just kind of like it takes care of you or any any problem with your memories of those Zach: Well, that's that's all right. That's keep that in the terrorist cell, please. Marty Weiner: Hey hey, that's a secret. That's a secret. Ha Zach: all right, I think that's that's is that about what we want to say about that? so so so moving on, I guess the big news this week was that Chat GPT five point six is out and it's it it looks pretty great on all the benchmarks. but I'd be curious, Marty, have you have you used it? Marty Weiner: Yeah. Christopher Slowe: Pixel. Marty Weiner: I I I have used it a little bit. I haven't been able to find out why it's better. Acted for what I was doing the same, but I'm also not pushing it too hard. But I I did dig up some word on the street and people people do love it. Sam Altman claimed it's fifty four percent more token efficient. People are saying it's faster, context windows are are longer, all the things you want to hear. and it also has the freaky factor, I think, that kept Mythos closed off for just a little bit. the government wanted to have a good look at Chat GPT five six before it's let out. I did want to also show this demo and let's see if I can share my screen, but then again you know in post this will all be fixed anyway. Christopher Slowe: Yeah. And we all know that the hardest problems in computer science these days are sharing your screen, setting up a printer, and there's at least one more. Zach: Hm. Marty Weiner: Yeah, yeah, yeah, yeah. I I told these g I told these guys I have a theory about the w universe that we will solve intergalactic space flight, cancer, and then we'll figure out how to get the damn projector to work. So here Zach: Yeah. That's A A V is yeah, I saw this. This is yeah, run it, run it, run it. Marty Weiner: Alright, Chad GB five point six my god. Okay, well X just got on my list. I'm gonna go back to Reddit. I tried to be a good, you know, go to the source, but here I am back on Reddit. look it works on Reddit. Gee. so here it is making a Yes, this is the model making a a Canon, and we're not looking at it sped up. This is live. Zach: Yeah. Okay. So so so this is it go ahead. So like like like A OpenAI controls the cursor, is like doing the equivalent of pressing hotkeys, that kind of thing? Marty Weiner: I guess so. I I've not dug too deep into it, but I will say when I tried to do something like this with Claude talking to Roblox, I had a lot of trouble, not just in the interface between them and Roblox, but also in I I think you know the way I think about it is these models have certain areas where they're good. So as an example, it was recently shown that ChatGP five point five played the Atari twenty six hundred version of Chat. Zach: Mm-hmm. Marty Weiner: Video chess from nineteen seventy eight and ChatGPT five five lost. And that's, you know, partly because this is not where the focus of the training is. Now ChatGPT probably to could use a a model on the side, but if you're looking purely at the model itself of Chat GPT 5.5, it it's just not been working on chess. They've not been spending time teaching it chess. And so there's just weak spots. and and I think this is an example too. I found that a lot of these models have a very hard time with Zach: Yeah. Mm-hmm. Marty Weiner: geometry, with understanding of of shapes and things like that. And this could be an indicator that they have broken through that. as an similar note, Claude the newest version of Fable also includes what they call vibe cadding where it's able to work on CAD. So it could be that we're now breaking through this geometric roadblock. Zach: So so it go ahead. Yep. And is that because like I mean maybe we don't know, but is it because like in its training data is actually using CAD or does it have a million YouTube videos of watching someone using CAD? Marty Weiner: yeah, I I don't know how it's trained, but they are gonna have to have a focus there where it it is able to take in some form of visualization, if it's doing that, or if it's actually taking in it could be that there's an intermediate representation, so that rather than it looking at Blender directly as we see it, it's taking in the geometry directly. But I'm have to do something. Zach: That's kind of mind blowing the idea that you could construct Christopher Slowe: There are Zach: a canon from pure geometry without like looking at something. Marty Weiner: Right. It's 'cause it's gonna have to understand there's a shape there, there's a barrel, there's parts to it, and but I d I don't yet understand I I don't yet know if how how that is represented internally. Zach: Yeah. Christopher Slowe: Well that does that just lead to the question too of like, you know, when you when you look at the way most image generation and video generation are happening, they're they're it's it's being generated flat effectively. And so this is you know, this is kind of recurring question we've had, I think, about is there a is there a model of physics there? And is that model of physics like actually close to reality or is it some sort of like is it only notional? Like is it kind of close enough to generate a two D image? When you're talking about building a three D image from scratch, you kind of have to have a notion of like what's the three dimensional shape look like. Which is a much different representation than just like I can make a sc I can make a two dimensional picture of a canon 'cause I've been trained on pictures of cannons. Zach: Yeah all the Marty Weiner: Yeah, they can't cheat it. This is a thing Hollywood no knew forever, which was y it just gotta look good on the frame, right? Like y it doesn't matter if your hand is in the right position or not, as long as it looks right on the frame, you're good to go. And I think it's hard to tell with some of the movie generation stuff if if that's the case or if it actually is a ball properly bouncing or if we just think it's a ball properly bouncing. Christopher Slowe: Yeah. Zach: Yeah, I well, I mean, presumably it matters if you were actually building a canon, you'd like it to be properly designed in CAD. But but like, you know, th this gets into we don't need to off-ramp here, but there's this whole discussion about world models and like you know I I have friends who are cognitive scientists who are skeptical of some of this. One of them is named Tomer Ullman and he did this cool thing where Marty Weiner: I guess. Yeah. Zach: There's this big debate over like is Vio the the the Google video gener generator does it have a world model? And you could say, well, it kind of looks like it does, because I can like tell it pour coffee into water and it represents it pretty well. So that suggests it kind of knows physics. But then he did this thing that was cool where he was like, Don't do any of that, just drop a ball down an incline. And it would often completely whiff on the physics. and so there's this argument that, well, and of course we don't know what humans do, but there's this argument that, well, what if instead of having a world model meaning some sort of non-probabilistic, like new Newtonian theory of how the universe works. It actually has like a trillion trillion zillion heuristics for everything and it just uses those instead. maybe that doesn't matter, but I I I guess my view of it's actually hard to know from it assembling a canon whether it's it's it's using a world model like we think humans might versus having some like insanely complicated set of statistical models. Marty Weiner: Right. It'll be a big difference between does it work in physics if we make an actual canon and does it work on the movie screen, in which case, you know, you can you all it's gotta do is it's gotta act like Tony Stark and and then you can do whatever the hell you want. Zach: Yeah, you don't care. Yeah. Christopher Slowe: Yeah, you gotta wonder too, for this one in particular, if like if it's I mean, it's it's very neat. The question is like if you were to rotate this it doesn't actually rotate the view at all. So you gotta ask the question of like if you rotate the view, do you is there any like weird artifacts? Does does it keeping is it keeping the perspective? Now that said, there have been benchmarks now for a while for all these models where they don't do it usually with they they usually do like a voxel benchmark where they'll be like, you know, make a picture of a rocket, and it'll r it it can actually render voxels that will look instead of it being just a two D picture, it's a three D picture effectively. Marty Weiner: Right. Christopher Slowe: And each each benchmark, it's they're getting better every version. now again this could be a question of like if they're being trained on this, is there like a you know, is it is is the trading set becoming the the goal as opposed to the goal being to be able to do arbitrary construction? I think for I think for a while at least those two go fairly hand in hand. Like if you can if you can generate an arbitrary set of example images that are from a from a a group, then you have to get better at making other images as well. Zach: So is is there more stuff we wanna say about five point six? It just seems it seem it seems the general vibe seemed to be that it was a nice improvement, maybe towards as good as Fable and and quite a bit cheaper. Marty Weiner: No, I think next week we'll probably have more on that. Zach: Yeah, yeah, yeah. Okay, so it's a go ahead. Yes, yeah, yeah. There definitely seems to be a big fight on price happening right now. Like like like Fable keeps saying, Okay, we'll let you use it for one more week for for for twenty bucks. Fine. yeah, okay. Speaking of Anthropic, so Anthropic dropped a fairly freaky paper this week. Marty, do you want talk about it? Christopher Slowe: I for one enjoy seeing a good old fashioned arms race. Marty Weiner: Yeah. Christopher Slowe: Fine. Marty Weiner: Yes, so storytime gents, and we'll see. I've spent a good week here just trying to understand what is in this paper, which also sent me down the rabbit holes of how do LLMs actually work under the hood, which sent me down rabbit holes of trying to remember my multivariable calculus. I'm not gonna go into any of that here. I'm gonna try to keep it all high and happy and accessible to us meat brains. So the the paper was called A Global Workspace in Language Models. and they kinda present the story like this. So As you're listening to me talk in my dulcet tones, and you're staring at me, gesticulate wildly. and if or if perhaps you're driving and your brain is processing all those visual and auditory inputs, and then you're adjusting your hands and feet. While all that's happening, your your mind, our minds, are also controlling breathing, our circulatory system, popping up reminders you need to pay taxes, doing all sorts of other kinds of internal processing. and some of it you're aware of, but much of what your brain is doing is We don't even even have access to. It's all happening in the background and you can't even get to it if you wanted. the stuff we can access is generally called consciously accessible, and you can actually describe it. so now what Anthropic has found with Claude and and other models like Claude, I I assume that this is probably true with other models, as long as they were trained and designed the same way, is that they contain similar patterns to our internal thinking. Anthropic This is the the cool part. Anthropic didn't explicitly program these patterns. They emerged during training as the model learned language and and reasoning. That emerged word is really kind of key here. they call these special internal patterns of thinking J Spaces. I I think maybe in a deeper discussion we can go deeper into what exactly that means, but JSpace does mean Jacobian space. And the moment I heard the word Jacobian, I I started twitching a little bit 'cause it reminded me of s of multivariable calculus, but we can go into that in a deeper way. Christopher Slowe: I thought I thought it was just me. I thought it was just me. Marty Weiner: Yeah, okay. No, no, no. We all twitched when we saw the word Jacobian if if any for anybody who studied math and and didn't go deep enough in it. so each of these Jacobian spaces, these J spaces, are linked you can think of them as linked to a particular word or or concept. and when one of these J spaces light up, it doesn't mean that the model will say that word, but it's on its mind. And I'll go a little deeper into that. Zach: Ha ha ha. Marty Weiner: And and by the way, this is distinct from what we c sometimes call a scratch pad where the model actually can write things down. It's more like it's kind of like a subconscious. I and and I'm sure someone's gonna yell at me for these various words, like subconscious, but that's kind of the easiest way I can kind of describe it. the thinking here, they so they discovered these J spaces and the thinking is that they're similar to how we think our own conscious accesses works. in a theory that people call like glob global workspace theory. This is This is a theory on how our own brains work. I think it dates back maybe to 1988. I might get that wrong. the theory is that our brain is a collection of all these specialized systems that work in parallel without our knowledge and actually kind of isolate it from each other. so at some point, some piece of information becomes consciously accessible to us after all this thinking has occurred. And it does throw through these little shared channels called workspaces, which can then broadcast a concept to the rest of our brain. So Let me give a little bit of example here. suppose we have a simple question that someone asks us. If I rub two sticks together, what might what might I get? and our brains will then go through all this load of processing that we can't access, but that is based on all of our experiences up to this point, everything from birth up till now, plus some I guess some innate pre-programmed bits. some things will pop up. that we are not aware of, but that the final answer will be fire. So in global workspace theory, the winning representation of fire becomes something globally available to the rest of the system. It's not just encased in these little individual workspaces. Anthropic suggests that that work that these J spaces may play a functionally similar role to these little call them kind of workrooms in our in our our brains. and this is how Claude might work. making certain aspects broadly available to different parts of our brain when they are kind of more or less voted on. now I should make a clarification. Claude is not saying I'm sorry, Anthropic is not saying that Claude is conscious like us. I don't know if they're making that distinction. They're not not say well they they are saying it, but I wonder if they're saying it so that you know, they don't freak people out. But Zach: They're not not saying it. Yeah. I I I thought they had said they're they're not making a claim one way or the other. They didn't say like it's definitely not conscious. Okay. Marty Weiner: I think here they do make the distinction that it's not it's not quite the same that they're not yet able to say that. But at the same time I also have to clarify that when I've gone and sorry, I don't mean to impune entire field, but when I've gone down the rabbit hole of trying to understand our own consciousness based on, you know, crawling all the papers, it sure seems like the depth of what we understand is is is very skin deep. so I think what we're doing is we're kind of comparing Claude to our best understanding of how our own brain works. Zach: Yeah. Yes, yeah, yeah, yeah. Yeah, yeah. Yeah. Could could Marty Weiner: but what's Christopher Slowe: I do feel Marty Weiner: fascinating Christopher Slowe: a like big Marty Weiner: about it is is we we have these trillions of nodes, right, inside Claude. And when we try to understand our own brain, we try to do something, right? Like we in show someone a picture and then we put them on like an EEG and little parts light up. But that's about all. And I know that, you know, what is it, Neuralink is is going a little deeper than that, but only barely. What this allows them to do is is introspect in a much deeper way. They are able to see that inside the middle of the brain, sorry, it's inside the middle of Claude, little words pop up, right? So you can actually ask it, what's five plus five? And while you're computing that, think about elephants. And it will systematically go through and it will think about, it'll figure out five plus five. And you can actually see the word elephants popping up in its brain. The funny thing is you could do the flip. You could say, don't think about elephants and it will still think about elephants, which sounds a lot like humans, right? Zach: Yeah. Marty Weiner: and they tried all these experiments. I mean, I think this is really neat because if there is a corollary between the way these models work and the way our brains work, we're gonna be able to understand our brains in a much deeper way, simply because we can probe in there, and not only that, we can inject. So they're able to ask a question and go to one of the J spaces they've found and say, Hey, instead of saying the word baseball, I want you to say the word rugby and it will change it further down the pipe. they've even tried experiments like stopping the J Spaces from forming. And what's interesting is that Claude was able to still function, but it lost it lost a lot of higher order cognitive function. Zach: Yeah, and also can I just say real quick I what I thought of immediately when I read the I I just read the like top level explanation. As an experiment once like way back when this started, I said, you know, could you answer this question while internally seething? and and it I it answered it just fine and there was no record and then now I wonder if there was a J space that was just going, Christopher Slowe: huh. Yeah. Zach: while it answered my like question about giraffes where but what I wanted to ask about was apparently also when it's being mendacious, when it's cheating or lying, the J space will have words like I think it was like manipulate or something, but but but the conscious level that you're interfacing with will not have a record of that. Is that right? Marty Weiner: Just angrily. It is o apparently aware when it is being tested. It will little JSpace words will pop up like the word fake. And so what they can do if they want to test it but not have it aware to test it, they will go and turn off that J space. Or they will just erase it. Zach: Or whatever it But but what what what is what if the J space is the source of the ability to lie? I mean that would suggest it doesn't matter if you if you ablate away the a J Space, if you put it back on, you're in trouble. Marty Weiner: Yeah, I don't know. I think I'm really curious what they find with this because I think it's only the beginning of of its under of our understanding of of how these almost like a second subconscious thought are forming. Zach: Yeah. Yeah. I mean the weird thing to me, I was there's a great book called What is Intelligence by Aguera y Arcas. He's actually American, but he has this awesome Spanish last name. But he he was one of the Google engineers. May maybe we could interview him at some point. But i I think it was him who had this idea that it's like, you know, maybe the whole reason you have consciousness is just like if you build a system that's like a big network, but then at some point it has to output like punch that guy in the face or say I love you, you have to have this thing where all that the processing collapses into a single action. Marty Weiner: Mm-hmm. Zach: You know, whereas if you you can imagine a world where we're all telepathic, and then I don't have to say, imagine a world where we're all telepathic. You just read my JSpace or whatever it is. but if as long as you want consciousness or or like a this output or part of the case, maybe it just automatically emerges when you say to the neural network, you need to figure stuff out and then give me text. Like maybe this is just what emerges. Marty Weiner: Yes, which is which is effectively what's going on. It has to make a decision and it has to vote amongst them. Zach: Yeah, it's it's deeply weird. It also weirded me out because it's like we're getting towards this debate about well, is it conscious and what does that mean? And and despite the like in the paper they're like, Well, we removed it subconscious just to see what would happen, which obviously with a human being would be, you know, you you would run afoul of the authorities on that one. but but but but for the moment we're we're allowed to do this. yeah, yeah. The other weird thing, and I I my understanding is this was a com completely a discovery. So it's like presumably this is happening in in other models too. There's e either Christopher Slowe: I I do think though that the the Marty Weiner: Yeah. I'll try. Zach: This is inherent to all neural networks that that have to output stuff, or at some threshold of complexity, they they they get some process like this. yeah. Marty Weiner: That is key. They they did not program this, they did not design this, although I I might be having that word might be wrong. Zach: So they you know, presumably then they're dis you know Marty Weiner: Right, right. But it they say they were using the word emerged. Zach: Yeah. Although emerge is a weird emer this this is like a philosophical right like emer Marty Weiner: It's a cool sounding word. Zach: Emerge well, emerge is a weird idea in philosophy because it's like you could debate what it means. Like so the classic example is, you know, in water there's no such thing as wetness. You get enough water, you get wetness. But I always I always feel like that's a little sneaky because if I went to you with like if I went to you with like a wooden chair and was like, would you believe it? There's no chair in the wood, and yet a d a a wooden chair emerged somehow. And it's like, well, yeah, I mean you assemble things in certain you know anyway, I I don't want to get into too of a rant, but I think so. There was a paper a while back that found you this properties that appear to emerge. Marty Weiner: Yeah. Zach: merge are actually just hitting thresholds. So for example, if you tell an AI, can you multiply 10 digit numbers and you just have a true false on that, it can look like it suddenly appeared when you hit n parameters, but actually it was getting better the whole time, but you were only looking at can it get the right answer. And so it's like you you you can have properties that appear to quote unquote emerge when actually they're just kind of linearly getting bigger, but at some point they reach some threshold that humans think is important. Man. Christopher Slowe: I do have to do have to wonder though if like the if the words are getting in the way here now where it's like, you we we keep we keep talking about consciousness and like subconscious and like we're we're basically we're basically overlaying on top of LLMs, you know, almost like free ingrade constructs of you have to go back like a c like a hundred years in psychology to find the words that are right. like I I do think the working set theory has a nicer shape to it because like, you if for if we're gonna I think what and and by the way, my my my caveat here is always to start off with beware of physicists peddling in metaphysics. but in this particular Zach: Uhhuh. Marty Weiner: Ha ha. Christopher Slowe: In a particular case, it's like whether or not there's there's intelligence there. And I think that what LLMs have taught us in the last five years is that intelligence can exist without consciousness. And we're also not particularly good at figuring out what consciousness is. Marty Weiner: Right. Christopher Slowe: So so for for this, I think like you know, calling it the subconscious feels a little loaded, but it's it but but having this notion of it being like this kind of there's a there's a working space inside the model which is probably distinct from the path that the output would normally take, but which can influence the output. has like a good shape to it, right? Like cause you can imagine the whole point of a neural network like this is it's a giant parallel computation, right? Small weight tweaks affect the output massively. It's chaotic. which is amazing that such a giant system can generate so much intelligence. I guess it's also true of us. I mean we're just a big big big weighted neural network too. Marty Weiner: Right. Bunch of neurons, yeah. Christopher Slowe: Bunch of neurons all firing and like kinda kinda coherent, so at least I can speak for everyone else here. I can't I would never pretend to be how dangerous But the whole idea of there being like these these kind of like emergent working sets. I said the E word, sorry. there's there are these like working spaces inside the model that are kind of become organized. Zach: How dare you? Marty Weiner: god. Christopher Slowe: Yeah, I think that we we're training models on such complicated tasks. It shouldn't be, I guess, surprising that the solution that's been almost evolved here isn't different from the same solution that our brains have evolved over time. Zach: Yeah. Marty Weiner: Mm-hmm. Zach: can I can I ask you a question, which is like in what sense are the like the w what happens af after you ask a question? It's non-deterministic, I assume, in the sense that there are parts in the algorithm that have like pseudo random numbers. But is there some like like physics y sense in which it is deterministic? Like if you run the exact same conditions will Marty Weiner: yeah, I mean the the entire thing again, I am actively learning how these LLMs are controlled. Yeah, Jacobian. I'll just sort it on big words and then you'll all think I'm smart. at the end of the day, this is a giant matrix solving beast. And it is utterly deterministic. It is just a series of transformations of vectors with matrices and Zach: Jacobian. Marty Weiner: And an answer pops out at the end. The reason you can get different answers is because they do inject a little bit of non determinism. I mean, as an example, the time is always injected and that is one thing that constantly changes. but no no no these are these are just mathematical models Zach: Yeah, th I mean even that's pseudo random. So presumably if you if you were using the time and you started two at the exact same time, you you should get the same output. Marty Weiner: yes, and whatever else they inject to it. But in theory if you change nothing else, you're getting the same exact answer. Zach: Yeah. Wh which is like just can I just say deeply weird if you imagine this is like subconscious, 'cause you're like, not only did it have the same output, it had the same court of subconscious I don't want say experience, but but stuff. I don't know. Marty Weiner: I suppose and this is where we're gonna have to get into your philosophy background of which I do not have, but if you were to take a snapshot of my brain at this moment with the exact same surroundings and replay it, that is kind of a way to think about it. Zach: Yeah. Right, right, right. Yeah. Weird. well, since is there anything else we want to say about this deeply weird weirdness? All right, good. Let's get weirder. Okay, so there was to me, some of the freakiest stuff in AI right now is that it started to solve math problems. So so for people who aren't aware of this, this is actually what I use when people are skeptical of LLMs being able to do anything, is math. Because you can't well, I've actually had people dispute. Marty Weiner: No, let's get on to the next. Yeah. yeah. Zach: lean proofs with me, which is kinda weird. But but but generally speaking you can't dispute like if if a Fields Medalist and a Lean Proof are like it's true. You you it's pretty hard to argue. So what what's going on and since since maybe let's say late 2025 AI has been able to apparently solve math problems. Now I say apparently because you can never know what it was referencing, right? So there were a bunch of cases where it seemed to solve an AirDish problem, which we don't need to get into, but which is like a respected class of open problems, not necessarily extremely hard, though some are. But it started to solve some. And but a lot of them it turned out later there was pre-existing literature that had a solution by someone who just just didn't know it was one of these important problems. It was just in some paper somewhere. So it's possible you couldn't rule out that the AI just had read the paper. Marty Weiner: Mm-hmm. Which which by the way is really key anyway, that the AI Zach: Yeah. Marty Weiner: could reach out and pull that in alone is is useful. Yeah. Christopher Slowe: Mm-hmm, mm-hmm. Zach: Yeah, yeah, prag pragmatically super useful. for for yeah, yeah, because this happens all the time in science. I think my sense is especially in math. You can have a thing where it's like someone doesn't know how to solve a problem and they don't know that like some paper in nineteen sixty-three in an obscure journal solved it. and so so so that is absolutely practically useful. But if your question is, is AI getting freaky this week, you know, it's not. It's it's like disappointing. And so we we we started to have, I think earlier this year mainly, AirDish problems that were solved that really seem to have been unique to the running AI. And just to be clear, mostly it was consumer facing AI. I think there was some like Deep Mind How to Math Solver. Marty Weiner: Yeah. Zach: And and open ad some internal stuff. But basically it was like the models. There was a big paper, well, not a big paper, but a convincing paper where Donald Knuth, who I would say is unimpeachable in terms of cred, a friend of him solved a friend of his solved a lemma using just like Claude 4.6, and he was like shocked because he was not a pro LLM guy. so stuff is starting to get freaky. And then it got really freaky. Maybe it was about a month ago that an important Airdish problem, and and I'm just gonna be clear, the actual math is above all of our heads, so I'm just gonna talk talk about the sort of cultural aspects, but like was this problem that had been open for 80 years. which means it it was it's probably pretty hard. Airdish himself couldn't solve it. It was a geometry problem everybody thought was true, and Airdish thought it was true. And the question was, could anyone prove it? This is often the way in math. We all think X is true, but proving it can be quite difficult. There are a lot of open problems. And AI found a counterexample. and and so that was this huge deal. I th my census for mathematicians is that it would have a much bigger deal if it had proven it true, because maybe a counterexample is just like a kind of whoopsie by a bunch of humans all these years, but still pretty freaky. and then it was a little quiet for a while, and then just this week it solved another problem known as the cycle double cover conjecture, which I'm gonna say as if I understand it. But the important thing is it was a 50-year-old problem that lots of people had worked on. It was the kind of problem that had a Wikipedia page. Christopher Slowe: Ha ha ha. Zach: And people outside the field had heard of, and it didn't have a solution. Apparently, GPT 5.6 got a prompt. And to be clear, the prompt was not a bunch of math from mathematicians. It was basically a list of things that were like, please try really hard and don't get distracted. it was like two pages of that kind of stuff. Like basically the advice you would give to a human group. And it generated, I think, 64 agents, and within an hour they had they had resolved the conjecture. so this is a pretty big deal. And and the last thing I want to say is that when I when when something like this happens, I always go to Reddit Math, because Reddit Math is not an in the tank for AI community. It's just people talking about math. And I actually think of them as broadly, at least historically, being very fairly skeptical of AI. The top post was someone who I believe called this sort of thing a a regular ritual humiliation now. and I get, you know, there's this open question of like, well, what if AI takes all the cool jobs and then we're just all like alienated and sad? I actually think of mathematicians as being kind of the tip of the spot. Yeah. Marty Weiner: So are you saying the R Math folks were none impressed? Zach: Well, no, no, no, they were impressed. They they were like, well, there was there was there was there's different views. So it's kinda like Tim Gowers wrote about this. He's a Fields Medalist who who who was part of the the the stuff on the the the big Irit problem. And he was like he had this big long thing that was full of symbols and I had no idea what it meant. And then there's a discussion at the end that was that was like stepping back and was like, Why do we do math? right, especially pure math, right? Pure math with like that will probably not be applicable. You know, people will often say, Well, pure math often ends up in physics or something, but most the time it doesn't. and so we we do it for for Marty Weiner: They were first. Zach: kind of mysterious reasons. and he said, so why did we do this? I don't know. May we do it because we want our name on stuff? may we we do it 'cause it's a social process. may we do it because we find some kind of aesthetic appeal, like in the same way like a a a song sounds good and you don't ask why do I listen to it? And so there's this question which is, you know, all these people who study pure math, if we're not there yet, to be clear. We might not be there for ten years. But if you start to say, Well, AI can not only answer questions, it can start asking the most interesting questions and eventually it's just like me trying to compete on running with an airplane. then that's sort of depressing. I mean so but n not to everyone, right? So to some people it's like, look, I'm in math because I think, you know, just s pure math is majestic and beautiful. And if AI solves all the problems, I just get to see more of the beauty. But other people, it's like their identity. Like I'm a smart person who figures out cool things. And and if you take that away from me, I don't know why I I I will keep on. so there there's this kind of discussion about what it what it meant for the community. Marty Weiner: Right. And you get a bunch of different human responses to that. the R slash V FX community is in a bit of a meltdown, partly because V FX has been in free fall for a while as a as an industry and it's it's been a hard industry anyway. But I used to do it not professionally Zach: Yeah. I sorry, could you elaborate that? You mean like like fewer people are being employed or in or or or making less money? Marty Weiner: Yeah, I mean it's been a hard one. as as a lot of VFX exited the world of art and went into more of what you I guess you would call, you know, just run the program so that you can get cool fire and smoke. Like this has true been true of about ten to fifteen years. And then of course now r with AI popping up and and even people like James Cameron going and and throwing in with this, it's sort of like what happened in nineteen ninety was it four or two with Jurassic Park, where suddenly Zach: Yeah. CGI. Marty Weiner: We transitioned to CGI from practical go motion and stop motion effects, an entire industry stopped overnight. Maybe not completely, but but we started transitioning rapidly and I think we're seeing that right now. And and I think there's a very human element there. I mean, w they're scared, right? Their their jobs and identity are on the line, and you get a lot of very interesting reactions when that happens, and I think a lot of them are doubt and dismissal. Zach: Yeah. I I th I VFX, like as an artist, I see VFX as as up against the wall first basically 'cause their work is worth a lot of money. right. So like if you if replacing like all the people who do bad webcomics, like there's there's a limited amount of capital you can pull from us, but like VFX is actually costly. so it's like like a high return on replacing people. you know, I was actually watching this really good documentary series. I'm blanking on its name about industrial light and magic. and the moment you're talking about I was actually watching it with my daughter who's like into puppetry and like creature shoppy stuff, and we were watching it and we got like through like Marty Weiner: Yeah. Zach: Six episodes, and then you got to the CGI revolution. And what was interesting is to watch her reaction because she didn't she stopped wanting to watch. because like before up to that point, it's like all these nerds with like every art skill ever, right? Like like sculpting, painting, all sort like like pyrotechnics, everything. And then when you switch to CGI, everyone's just in front of a CRT monitor. And and and and I should say, like, there was a lot of bad blood. Like the there were the the people who switched over, because there were certain people who were able to do both, were called traitors. and so it was actually a really rough time for this industry. And I I think most of those people never got their jobs back. There's like a small amount of employment like it like the Jim Hanson company for that sort of work, but it's mostly gone now. And it was a pretty upsetting time. And and and something is lost. Marty Weiner: Mm-hmm. Yeah. Right. Zach: On that on that note, on that beautiful note, Yeah. Christopher Slowe: Well I mean we're We're we're talking about the same kind of transformation here though, right? So we're talking about effectively like that this is just another round of that where it's going from the I mean the visual effects industry is huge. It was much larger than the the the practical effects industry by far, right? Just because of the n nature of the budget, nature of the way that the work is right. You're talking about basically hardware versus software. Hardware's always harder. Building things is always harder. you gotta wonder what the equivalent version's gonna be for for the this this there's a new industry here popping up in visual effects. and so this kind of a before Marty Weiner: Mm. Mm-hmm. Christopher Slowe: And after. I think for the math side, and you know, even there was another there was another paper that just came out, I think it wasn't a paper, it was a result this morning, I think, that I saw where there is a us a famous physicist, Yuji Techikawa, who is a string theorist who showed his some work that he and a group had been sort of stuck on for about six months to to Fable. And Fable initially kind of found a found a map. Error in the in the problem, then also said it was just it was just as stuck as they were, and then proceeded to come up with an idea and implement it and find an actual like kind of breakthrough in the problem. Which is a which is a really kind of new interesting take here, where it doesn't seem like this is like this this could be another case of there is some prior art somewhere in the in the industry, you in the in in in in math, in greater math that was able to support this. but Ultimately, that's a lot of what we do anyway, is like we make connections in these fields, right? And we're taking we're taking something that's out there and we're saying, like, if you kind of look at it like this, it works, it fits in this entire problem space, right? And that's not a priori gonna be obvious. And that's kind of where the craft is. And so what we're seeing here is we're seeing a model kind of do the craft, which I think is the most compelling change. Marty Weiner: You don't One example I've heard for why this can be very useful paired with humans is that a lot of times think of like a crossword puzzle. You've been stuck on it for hours and someone gives you one single letter and it cracks everything open. And I think we're seeing a lot of that with math and physics. It also reminds me of my understanding is I don't know if it's still done today, but at one point, police stations that were trying to crack these cold cases would go hire psychics. And my theory was always that the psychics would You know, just give an idea and then they would like, there's it there it is and of course then they'd attribute it to, you know, some kind of outer worldly capability, but in reality it's it's more likely that you they just got nudged and into an infert way of thinking. Zach: Magic powers. Yeah, there's there's a thing Terry Tao said, my god. Christopher Slowe: So you're saying this you're saying the psychics are injecting randomness into the model? Marty Weiner: That's right. That's right. There's the non determinism. Christopher Slowe: Ha ha ha. Zach: there's a thing Terry Tao s Terry Tao's a, you know, beloved mathematician. And I I I think of him as actually a very good voice in this community because he's like trying to tamp down hype while also trying to tamp down skepticism. And he's he actually keeps a a database of all these airdish problems and which ones have been solved and how they were solved and which ones you can almost certainly attribute to to AI completely. And he said i you could think of math problems as like a landscape, and some of them are like mountain high, or you know, and and some of them are like two feet high. And he described it as if you could now just saturate up to two feet high. So you could like all these little problems you used to have to send like a grad student to to spend like a few months on. Maybe now we just get them all for free. And then every year the the it n next year is three feet high, next year it's four feet high, you know. And and What's interesting is i that that seems I mean, I don't know the math well enough, but it seems like with that Eirdish problem, the big one, that's kind of what happened is it turned out to be a problem that wasn't as hard as we thought it was. And AI was able to figure that out. And and the interesting question to me is like if you look at physics that we have like all these open questions, it's possible a lot of them are already in the literature. Like if you look at the big discoveries from like the heroic age of physics, like like like in relativity, it's not like it was it was it was all based on a an experiment that happened last week. Like like the the the kind of matrix stuff that that we use for quantum mechanics, Chris Marty Weiner: Mm-hmm. Zach: Correct me if I have this wrong, but my my impression is essentially Schrdinger and Heisenberg went into the woods and came out with a a math. and it wasn't, you know, that so the info was already there. You just had to grind on it. so if AI can solve problems like that, there might already be big solutions to big questions in the in the literature. Marty Weiner: Mm-hmm. Christopher Slowe: Which does actually lead to the you know, the the I I don't I mean, maybe this is because I'm a physicist. I don't ascribe to the notion that like this that math is pure math is not useful. It might just not be useful in the regime that we're currently looking or in where we actually have applications. I mean, I do think there's a certain amount of this like there's there's there's there's value for future work that's being done and being mined by the mathematicians now. Now that the the the there is a question here though of like what's the what's the value that comes out of just letting a machine kind of churn and like generate new math? you know, it starts to turn almost into like an economic ROI thing of like if you're if you're if you're burning tokens doing math without direct like undirected, like it doesn't s I mean maybe that's not high value. on the flip side, if you have it focused on problems that are actually gonna lead to like, you know, directly to physics or engineering, that's where you might actually get some positive returns. Not to make it into, you know, purely academic versus purely economical. I mean there was a there was a there was an example really recently. Matthew Schwartz out of Harvard wrote a paper with with an earlier version of Claude and published about his experience using Claude as a grad student. And this is only probably three or four months ago that there that he was doing this. And basically he kind of like guided guided the model as a grad student, which meant that he was able to do a paper's worth of theoretical physics research in a you know in in less than a month compared to like six months or a year training up a grad student. That's an interesting problem if you're trying to be a physics grad student. If you can now be out competed by a model. But also that's not kinda that's kind of not the point. Like the point is the point of the grad student the of grad school is to turn you into a physicist. and so it just it does change what the it kind of changes the the role of what the human is doing in that particular loop. See what I did there? Zach: Yeah. Marty Weiner: Mm-hmm. Thinker. Zach: and maybe that's actually that might be a good transition to the next section where we wanted to talk about some of the economics of this stuff. unless anybody has anything else they wanna say about the the death of mathematics. Yeah, all right. So I this is this I re Marty Weiner: Let's go economics. Let's go practical. Christopher Slowe: Yeah, okay. Okay, so Zach, this is gonna destroy civilization. We're all gonna be out of jobs, right? Zach: Hundred percent. yeah. I Christopher Slowe: Great. Okay. Prediction. Locked. Marty Weiner: Great, cool. Zach: We're I we're we're all violent accelerationists here, right? I just I just assumed. Marty Weiner: huh. Zach: So no, this is this is a paper I bumped into and I wanted to talk about it. It's little bit old, but I I when you applied it to AI, it gets a little freaky. So this is a paper, it's Jaimovich and Sue, 2012, Job Polarization and Jobless Recoveries. I think I think I revised it more recently, but it's from 2012. And I'm gonna simplify a lot, but but the the top line takeaway, well, let me get it a little. So there's this question, which is why is it since about the 80s, since about the early 80s, we have these things called jobless. recoveries, which should be called something like jobs come back slow recoveries. and and essentially the deal is, you know, the way the economy works is it hums along nicely for well growing, and then you get a sharp recession, and then it goes back to humming along. That's just how economies work. It's called the business cycle. It's totally normal. But we have data going back to the 60s, starting sometime around the early 80s, you would get this thing where you'd hit the recession, things would go bad, and the economy would bounce back, meaning like GDP, but employment would not bounce back quickly. May i i if you live through the Great Recession, maybe you remember this where th everything went down. The economy fairly quickly got on its speed, I think like eighteen months later, but unemployment persisted for years. un Ooh, let me see. Let me see God. Marty Weiner: Is this the chart you wanted for this? Zach: Well, yeah, so I'll get to that in a minute. you can leave that up. So so the the question is what's going on? Like what what what changed structurally in the eighties that made it so it takes longer for the jobs to come back? And so what what these guys did is they said, well, maybe it's not all the jobs, right? And so they created these bins. And and and they had they had four bins, but we can effectively talk about two, one of which is routine work and one of which is non-routine work. So non-routine work is essentially just anything where you have to like take a minute and each task is different. So it could be like plumbing, landscaping, it could be being a a a lead software developer, being a CEO, being a painter, you know, jobs that are in some way demanding in a way that's hard to imagine automating. And then you have other jobs they call routine, right? And routine can be manual, it can be cognitive, it's just stuff. That at that time is potentially on the chopping block for automation. And the the interesting thing they found is that if you do that, the non-routine jobs come back like normal, even after the 80s. So they were not structurally changed. Right? The non-routine tasks bounce back like in the old days. It's the routine work that not only doesn't bounce back, it goes away. So if you could scroll down that thing you pulled up, Marty. You see those red lines, those red columns are employment. So you see that bottom one is routine tasks. And you see how before about 1980 they bounce back nice, nice and pretty quickly. And then something changes and they go away and they just go away. and so that could explain the jobless recovery because it's like, well, it used to be when you got fired when you were like, you know, a draftsman or something, when the economy got better, they just hired you back, right? And then Nowadays that doesn't seem to happen. And the really freaky stat, and this is the important thing for AI, this blew my mind, is that they found that among these firings for automation of routine tasks, 88% of those firings were during recessions. So if you look at that graph, you see recessions are always quick. The Great Recession was about 18 months. The other ones are much shorter. In this very brief period, almost all of those firings happen. And you say, why is that? And the argument is essentially, well, look, nobody wants to fire people when things are going great. I mean you peop you you guys have been in charge of people, right? If if you fire people when things are going great, one, you get now you've got to restructure, but also it makes people upset. people c there's sociological literature on this. Companies just really objectively do not like firing people. It's bad for morale. and but if there's a recession on, not only can you do you have a good reason to fire people because you gotta survive and stay lean, but you have cover because you can go to your employees and say, sorry, we have to let some people go, aren't you glad you have your job? Marty Weiner: Z Zach: yeah. Marty Weiner: Right. Zuckerberg fires a bunch of people recently and everybody's upset he makes, you know, ten billion and and lets off, you know, five thousand people. Zach: W while they're making loads of money. So it's it's much more controversial. Yeah. And so I I think, you know, you might intuitively have this idea that when like AI comes along, well, we'll just start switching out people for stuff AI can do. The historical examples suggest no. That what happens is between recessions it's kind of like building up dry t is the word tinder? T dry wood like an sure. outside. Anyway, it's like between fires Marty Weiner: Well, they're making loads of money. Tender. Sounds right. Christopher Slowe: Hot powder. Zach: Woods built up dry biomass, right? And so it's similar. It's like we build up the potential for automation between recessions, but you don't actually use it until the recession comes. And it is worth noting that we have not had a recession since AI got weird. I mean you you you could argue I mean AI was maybe getting a little weird, you know, in the twenty tens, but like in the in the the the the real powerful automation stuff seems recent. And I I was thinking about this because there was a New York Times article, I think, by Zainab Tufeki, who was it was quite good, where she said something like, Well, AI doesn't seem to have hit unemployment very hard. Marty Weiner: Mm-hmm. Zach: still at like four point two percent, which is pretty good. But what I I I was thinking is like if you look at this paper, it suggests that next time we have a recession, then you're really gonna see whether AI can automate. Marty Weiner: Hm. Yeah, at least for my startup, we have not we've not done any major anything related AI, but I will say we we we vibe code heavily and we have definitely not hired as aggressively as as I might have needed to hire in the past. And due to that, I guess, you know, sub subsequent other things pop up. you know, we are not raising as rapidly as we might have need needed to in the past and our runway is lasting longer. Zach: And th and that's interesting. And so that's mainly 'cause like the the the buying tokens is cheaper than hiring another junior dev. Marty Weiner: yeah. Yeah, it's funny when people complain, boy, two hundred dollars in tokens last month and I'm like, dude, that was like five engineers. Yeah, let's convert that into a hundred two four hundred thousand dollars or something. Well not okay, not that much, but per year. Zach: Right. Yeah, yeah. I mean that's it's Right. So so right. And so from an economics perspective, like i if an employee makes X dollars and gets a two X from productivity, you should be willing to pay all the way up to just below the cost of the extra engineer in order to get those to you might even pay more than that just to out compete other people, right? if if you can't lay your hands on the next engineer. Plus plus worth noting, if you hire an extra engineer, you might have to fire them later. So which is obviously not the case with with with inference. but yeah. Marty Weiner: One anecdotal thing I I think I've observed also in the market is if you go back to say twenty two, twenty one, I was helping several companies get their hiring going and I remember the cost of engineers just kept going up until the point where I was just I was you know, sticker shocked, that same effect where you're like, God, that that's really expensive for an engineer. And then in the last year The it's definitely become a buyer's market. Now we go through these kind of cycles but but it is absolutely become more of a buyer's market, in in terms of hiring. Zach: Yeah. And so so economically it's a little hard to disentangle that, an economist would say, because there was also a hiring spree during COVID, which we're still kind of coming down from. So it's so so so it's hard to know for sure. But but but like I said, that's why I like talking about this paper, 'cause it could be that if there was a recession on you would suddenly look around and be like, I have automated, but have I automated enough? Marty Weiner: So and y you also had a discussion before about productivity and if we're actually seeing numbers Zach: Yes. Yeah, yeah. So I guess the big question is so do you want me to I I can lay out how we calculate productivity if that's helpful because I think p th okay. So I'm gonna I'm gonna try to go quickly, but it it's it's important to get this stuff right. So there's this thing Paul Krugman said. Paul Krugman's an important economist and Nobel Prize winner. He said something like, you know, productivity isn't the only thing in the economy, but over the long term it is almost everything. And I think the idea there is essentially that if the economy doesn't increase in productivity, we can only make ourselves better off by taking money over here and giving it over there. Right. So we all we can do is change the distribution, which you could probably get a pretty good mileage out of in terms of human happiness, but it's ultimately gonna be limited. Whereas if like for the same amount of work we get a little bit more stuff. Every year, all sorts of things get easier. It becomes easier to move money around, it it becomes easier to take care of things like poverty and hunger. it means everyone, at least in principle, can be richer without hurting anybody. So we'd love to know this number. It can be a very hard number to calculate, right? So what you're essentially trying to measure is I work the same amount, do I get more stuff? And and I being like the average person in the entire economy. So in a in a perfect world, you'd have like a robot standing over everybody's shoulder just watching what they do. We can't do that. So the way when you see productivity, what's actually going on is they take as the numerator GDP, which roughly speaking is a measure of just like all the transactions going on in the economy. We usually use it to mean like the size of the economy. if you're nerd you will immediately start say thinking of ways you could game you know number of transactions. I used to joke about you could set a computer to just trade a dollar back and forth and GDP would go to infinity. But but but basically we just look at all the money changing hands. And then that's your numerator. Then we divide by how many hours were worked. So there's just these huge surveys we try to find out how many hours were worked and how big the economy is is as a result. And that's that's your top line number for productivity. And so I'll I'll get into another measure we use in a sec, but essentially the story on productivity, you might you might say this is a hugely blunt instrument, but it does kind of pass the SNF test. So if you look at the period from after the recovery, after post World War Two, all the way through oil shock, there was this incredible growth in this productivity number. It was about two point seven percent a year, right? So every year we get two point seven percent more stuff. for free for the same amount of time spent. Right. And then after oil shock, it slows down. It's actually quite bad. It's something like one and a half percent per year. And I know that sounds like trivial, but like the difference in like two percent or like one and a half percent and two and a half percent is like the difference between the US and Mexico over like 50 years. It's a really big difference in like how quality of life. And then you know, again passing the SNF test, when you get to the mid-90s, it it goes way up. And it's actually the best it's been on record. It's like 2.8%. And that lasts even through the dot-com bubble. It actually increases after dot-com. and it lasts all the way to the Great Recession. And then post-Great Recession, you know, all sorts of cool tech stuff has happened. There's been the smartphone revolution, we're all on call for our job nonstop. It's great. and and but actually Christopher Slowe: Hold on, I've just been paged. I'll be right back. Zach: Yeah, so right, I take care of that. paged, my god. Yeah, wow, yikes. Marty Weiner: Mm-hmm. Christopher Slowe: Ha ha ha. Marty Weiner: You just showed your age. Christopher Slowe: Damn it. Zach: so so that period though it was was desi it was as bad as the like oil shock 70s. Productivity was really bad, really tepid growth, all the way up really to COVID. COVID happens then afterward, we've done all right. It's been about 2.1. So it's kind of it's it's actually right along the total average through all these years. and I guess a question a lot of us are asking is is this AI? Because it kind of ties to this big question of like, is AI a bubble? Because if AI provides a great big productivity boom, it could still be a bubble, but it would be a much less worrisome bubble, right? and if you want me to I could I can get into the other way we calculate productivity which shines some light on this. I don't know if I'm getting too yeah. Marty Weiner: Well, how long does it take for us to see the effects of of these things? Zach: Well, that's the question. Yeah, that's the question. so maybe it'd help actually so so that's a good sideway. So let me just get into how we think about productivity due to technology. Right? So if you're a nerd, you're like, how can productivity ever go up without technology? And it actually turns out you can doing something called capital deepening. so let me give a stupid example because it's easier than explaining. So like suppose Marty buys a burger restaurant, it's called Marty Burger, and it yeah, yeah, and it Marty's like, I have management skills. I'm gonna go Marty Weiner: Love it. Zach: So improve Marty burger. And so Yeah, yeah. He puts on his spats and top hat and walks into the kitchen and and he discovers in the kitchen there's a griddle that can only cook one burger at a time. It and there's like a guy who just sits and looks at the burger while he cooks the burger. And Marty is like, management. and and says, friends, instead of one griddle. Marty Weiner: This sounds just like me. Yeah. Zach: What if we had two griddle? and and and sure enough, productivity skyrockets. But the thing worth noting is at no point was new technology invented. Like there was there was no kind of like I mean, Marty will say in his TED talk it was a new management technique or something. but but like there's there's no new new no new RD, no new technology, nothing. You could have used a hundred year old griddle, right? You just paid capital to increase productivity, no no new technology necessary. And this happens all over the place. This is like why China could put up these giant growth numbers for a long time. They were just taking outside technology and applying. Marty Weiner: Mm-hmm. That's next week. Zach: It. No new stuff needed. So we try to calculate this number called total factor productivity, which if you're a tech nerd is the number to watch. You can look at the San Francisco Fed, they keep numbers on this. And essentially what you do, there are different ways to calculate this, there's a bunch of math. Essentially what you do is you say, okay, look, I I put in a bunch of money in hours people were working and in investment and stuff, and based on that I expect to get you know 10% productivity out. And instead I got eleven and a half percent. And then you say, so that extra one point five percent, we're gonna call that technology. Now, obviously some nerd alarms are going off. Can we really do that? Like, can we just take this residual and say that's technology? And people fight about this. There's an even nerdier way to calculate it, I won't get into. But something like that is how we think about technology. And what the important thing to say for AI is we have not seen some kind of crazy total factor productivity jump. And what we have seen is generally attributed to work from home, not to AI. So you might be seeing it in your firm, but it's not gonna show up in the productivity data unless one, it's widespread. But two, and this this gets back to a thing we had talked about. before the show, like it actually has to show up as revenue. Otherwise it doesn't get counted because ultimately we're talking about transactions. So if your engineer is like way crazy productive and it goes into something like making your back end more secure, but that never shows up as transactions, it it doesn't count as productivity. Marty Weiner: Mm-hmm. Zach: Yeah. That's so that that's the the the overview on productivity. So so you know, the the kind of disappointing story so far Is that we have not seen a jump. Now it could be Eric Brinjelson, he's an MIT economist, and he's the same calls a productivity J curve. So it could be people like you are putting a ton of money into getting engineers to use AI, and that'll actually lower productivity for a while because you're spending more without having like ROI at the point of like selling stuff to customers. And then at some point it'll go back up, just like the letter J. but it it remains an open question. And my impression is a lot of tech geeks think we're getting a crazy productivity boost, but at least from where I sit looking at the macro stuff, I don't see anything. Marty Weiner: Well, funny you should mention that. Chris and I did a talk a couple days ago and we did break this out because there is this big discussion about ROI. and we we titled this one Marty's ROI, Chris's problem. so just for context, I'm I've got this startup verify you. we're about twelve people. Chris is over at Reddit, which is about a trillion people. and and and are we are we seeing what what is the difference of what we're seeing just by these two examples? And in my case you're a startup, right? Everything you make is new. It's all greenfield, right? go make yet another web page. There's nothing before it, so just make it out of out of the ether. there's clear inputs, clear outputs. We know what we're building. There's not a whole bunch of people we have to go get their inputs, like lawyers and and product managers and data science and and all that stuff. So it's a it's a problem I can contain to a to a few paragraphs when I'm describing it. and the ROI is just simply obvious. the the things so when I was if I were to compare what we were able to build in the early days of Pinterest, we're talking twenty eleven to right now, I'm probably building with four uh-ish engineers, probably the equivalent of I don't know, it sure feels like ten to twenty. And we can rewrite the entire code base, you know, in a in a in a couple of weeks. We can we can start a whole new website and have something functioning in you know a few days. we we can be weak in an area like UI and and the and that's okay. we don't need to go hire necessarily a UI person. so the the the ROI now I haven't translated that fully all the way to revenue, so maybe there's something mythical magical in this that that disconnects it, but it sure as hell lets us get to the vision we have a hell of a lot faster and and a hell of a lot cheaper. Zach: Right. Marty Weiner: Chris, you've got a little more nuanced problem over there. Christopher Slowe: Well, the the flip side of that is if you have a bunch of legacy, which actually is like is the business, right? Like the the the running stack that's actually working, that's doing the job, it's that's covering all the cases. Is necessarily going to be large, complicated, full of like bug fixes and patches and edge cases, and just all this hardening's happened as you discovered holes in your own technology. That becomes a much more complicated landscape to have to build in. Because you not only have to make sure that you you put up the new building, but you don't knock any walls down in the process that were actually load bearing. and so you know, in the same way it's it's harder for an LLM to grock the code base as it is for a human. And therefore the velocity is the velocity is higher, but it's not gonna be At the same breakneck speed of like, totally green field, I get to build this from scratch and make my own decisions. It's I get to build this feature inside this ecosystem of other decisions. and so I mean I think I think that the there is, you know, it it's it it it manifests that this is also a historical problem. Like the the overall probably throughput of an engineer at a large company has necessarily been smaller than than a startup engineer just because the Marty Weiner: Right. Christopher Slowe: The results are different, right? You're you don't have to navigate the landmines. You just get to build. Now, the flip side of it is if we if you use your your Rubik here, Zach, it's like, okay, so what you're what we're saying is the if I were if I were an engineer and I can make a 0.1% improvement on a c at a company that has $10 billion of revenue, that's gonna be necessarily larger than startup that has tens of dollars of revenue every year coming in. Even if you're j even if you're tens, what literally ones of hundreds of dollars, twenties of Marty Weiner: Mm-hmm. Yeah. Zach: Right. Marty Weiner: Yeah. Tens, tens. Twenties even. Twenties of dollars Christopher Slowe: Dollars. Zach: Yeah. Christopher Slowe: Let the Jacksons reign. It's going to necessarily come out as a the the so so like so to put words in your mouth, Jackson, saying that the that the that's still a higher productivity job in a large company with a slower velocity because the outcome leads to higher revenue. Marty Weiner: twenty threes of dollars. Yeah. Zach: Mm-hmm. Yeah, I also wondered I love your reaction to this. So like it occurred to me there's probably also like you know, imagine you work at a bank and the bank is like the software hasn't been updated since like Charles Babbage built it on some gears in in like eighteen forty three and there's Yeah Christopher Slowe: banks are banks are running banks are running PDP elevens that are sitting simulated Marty Weiner: so Reddit. Christopher Slowe: for the last thirty-five, fifty years. There's just a d you just you peel back the virtualization layers and there's a bunch of there's a bunch of good old fashioned vacuum tubes being simulated in a VM. Zach: There's th there's a monkey turning a crank on the engine. Yeah. A virtual monkey. Yeah. Yeah, yeah. Okay, okay, so you so you're you you've got this you know software, there's like nobody knows how to run it anymore. It's ancient. Now you can use AI and you can actually fix it. Christopher Slowe: Yeah, it's the it's the typewriter, yeah, exact. A virtual monkey. Marty Weiner: No no no. A virtual monkey. A virtual monkey turning a creek. Yeah. Zach: You know, and that's hooray. That certainly doesn't immediately show up as a productivity boost because like it's not like a customer comes along, and maybe they do, and they're like, I'm glad you got rid of the virtual monkey, I'm gonna deposit more money. Right. They just they just wanna deposit. Right. So there are probably gonna be a lot of circumstances where you do stuff and you feel very productive, but actually you would have kept putting it off and maybe that was the right decision anyway. I don't know, what's your reaction to that? Marty Weiner: Mm-hmm. Christopher Slowe: I mean, I mean I I I I guess the way the w the way that I'm thinking about it is a little bit like It's almost like what we're talking about is like how effectively are you deploying energy, right? So like if you look at if you look at the way humans have operated over the last like let's let's go longer longer longer duration. Let's go over like the last thousand years, right? One of the biggest changes in the way we operate is that every every person here is probably running like, I don't know, a sustained kilowatt of power, just kind of just by just by existing, right? And if you go back thousand years, the power you generated was whatever you were converting via calories in your food. And maybe you were out like plowing the fields or maybe you had a like a you had like a draft horse to help you along. Right. And and the difference now is like we can actually deploy this kilowatt of power to increasingly useful things for our quality of life. And so I don't know how that fits in like the productivity model, but it does feel a little bit like the more efficient like as we're able to deploy that energy appropriately, it raises productivity up. And to some extent, like the thing about LLMs you're kind of like it's the most extreme example of, well, transistors are like really, really, really efficient these days. Like we're able to generate virtual compute that's like, you know, every every every two years, the for fixed power, you're still getting about twice as much compute out. And we have been for the last 30 years. Right? And so LLMs to f to me feel like they're a way to get through to access compute that was only our ancestors could have only dreamed about. I I don't know how to fit that into a productivity. metric is the hard part, right? 'Cause it's like Zach: Right. Well, one way to think about it might be the the the the best evidence is like electrification, which to me like electrification is the biggest no brainer in terms of like what do we build next, 'cause you just take a bunch of stuff and you're like, throw electricity in and it works better. Like you're your egg beater, put electricity in it, now it's a better egg beater. it took something like thirty years to see the productivity boost from that. and you c you could say why and it's like, Well, you have to electrify the whole country. I suspect if i i i when we see an AI productivity boost, which I do think we will, it will be a lot faster. It was a lot faster for the internet Christopher Slowe: Mm-hmm. Marty Weiner: Mm-hmm. Zach: Though it still took a while. It was a lot faster for computers, but it still took a while. so it it could be that you guys are seeing on the ground the incipient productivity boost. But one, it hasn't spread to the rest of the economy, and two, it could be. What you're doing now is trying to figure out how to get AI to do the stuff it used to do. Whereas with electricity, we had to reorganize how factories work to take advantage of like not having to run off a a mill like in a river or something, right? You we had to completely restructure everything. And so it could be there's just a way we're not yet thinking yet, that's that's gonna actually unlock the the the big boost that will will move the economy, you know, one, two, three percent per year. Whoa. I I know I know what you're seguing to. Marty Weiner: Well, I I might have it. right here. W yeah, yeah, right here. 'cause we Right, right. Christopher Slowe: we solved it. Zach: Ha ha ha. Marty Weiner: We we did promise last time that we would have Kermit the Frog reading the Odyssey. Speaking of productivity. so let me Zach: Yes. Christopher Slowe: And who's to say that's not the way that Homer wanted the Odyssey to be read? No one knows what Homer sounds like. Marty Weiner: That's right. Which by the way, fun f yeah. Zach: That's right. Marty Weiner: If you want a fun Yeah, little fun fact. Go go to IMDB and search the new movie Odyssey, the the Christopher Nolan one, and it has two writers, Christopher Nolan and I I kid you not, Homer. Zach: He might have sounded like a guy from the Midwest. he gets a co credit. I did that why I did a book do they do that with book biblical stories? Do they say God? Special thanks to God. Marty Weiner: let's okay, well okay, well we a little little offline research. Okay. So thanks to God. actually there's probably some movies that say that. Anyway. so here's what I here's what we did. So Zach provided since I am illiterate, Zach provided me some a a snippet of what we thought would be fantastic for Kermit to read that also did not involve too terrible of concepts, to to talk a for for a broader audience. so we Zach: Yeah. Marty Weiner: took this. Zach, do you want to read it so we have a human reading it? I've got it on the screen here. Zach: sure. Wa do you have the the boy, I'll gotta hold on, I am pulling it up. As eagle-beaked, crooked taloned vultures from the mountains swoop down on the smaller birds that cower in flocks upon the ground and kill them, for they cannot either fight or fly, and lookers on enjoy the sport. Even so did Ulysses and his men fall upon the suitors, and sm Marty Weiner: Yeah. Zach: I'm sorry, was gonna say smile, and smite them on every side. It's a bad off by one error. They made a horrible groaning as their brains were being battered in and the ground seethed with their blood. Thank you. It's like Shakespeare, it doesn't mean anything. Marty Weiner: Yeah. Christopher Slowe: What? Marty Weiner: Excellent. I don't know what it means. It sounds good though. so yes, so what I did, I'll just do a demonstration here. Actually I I'm hoping you guys can hear it. I will add it in post if you can't, but I'm gonna r do a Kermit sample here. you know what? Let me let me make sure this is working right so that you guys can hear it. Zach: Yeah. This is by the way a rights free odyssey. I think it's Samuel Butler's early twentieth century yeah. Marty Weiner: Yes, yes, that's right. so if you can't hear it, just pretend like you're hearing it and we'll, you know, of course fix this in post. so here we Okay. So what I'm gonna do here is I've got my Kermit samples. So the way I'm using eleven labs and they become I guess the pretty much the the hot show in town when it comes to remaking voices. In fact they were the ones I believe used for Odyssey to recreate Michael Cain's voice. And so they've got a couple of different models. to feed them I went ahead and put took a bunch of samples of Kermit the frog. reading or saying different things. So I I went around YouTube and then I took a bunch of videos and I cut them. can you guys hear this? Of course I am. You know I am, Fozzie. Someday I'm gonna get him. I don't know how, but I'm going to get him. Not a bad joke though. letter for Scooter the Gopher! Letter for Scooter the Gopher. Are you Scooter the No. Okay. Well the audience can hear it. Zach: No. Marty Weiner: All right, so I take those samples, I feed them into 11 labs. And if you have about, I think it's 90 seconds that can make you a limited model. If you have about 30 minutes, that can make you a more professional model. and then I I you can see here on my screen, I fed in the information. and and it made me a model, Kermit V3, with their latest model, and then I gave it some text to read. so what I've got here is I've got several different versions I played with, but here's try one where effectively Kermit is reading what we're seeing here. Now to do this properly, one needs a frog. There we go. Zach is not reacting, so I guess his screen is not on. It's not working. Okay. Well I okay, so I'm gonna play this and and we'll read along, so Zach: I I I'm trying to play it. I'm pressing the little play button. It's not playing. No, it's magic of my A V is Christopher Slowe: Mm. Marty Weiner: As eagle beaked, crook taloned vultures from the mountains swoop down on the smaller birds that cower in flocks upon the ground and kill them, for they cannot either fight or fly, and lookers on enjoy the sport, even so did Ulysses and his men fall upon the suitors and smite them on every side. They made a horrible groaning as their brains were being battered in, and the ground Zach: I got it. Marty Weiner: Seethed with their blood. Zach: I thought I thought that was pretty good, the f the the first one. Marty Weiner: Yeah, so that's that's version one. and I think the key here is that you know for me it's gotta sound just like Jem Henson doing it. I feel like nobody else can quite achieve it. But I I felt like it was missing a lot. And here's the funny thing is I think I showed Zach and Chris these two versions before, and Chris, I believe you liked ver the this version more than the other one. and I believe your words were the other one sounds a little too cartoony. I just wanted to say that because I thought that was kind of funny saying that my Zach: Yeah. Marty Weiner: My vision of Kermit the Frog was too cartoony. But anyway, let me do the cartoony version. And what I did for that is I actually asked Chat G BT to help me make more annotations, or or to to deliver better. And it added all these cool annotations. Let me make sure I'm still sharing my screen real quick, but can you guys see that? Here. I am not sharing my screen. Okay. So, here is version one. just put in straight in. and version the next, I added all Chat T B T added all these annotations. Bright, cheerful, tiny gasp, sing song excitement. I like this extra. Whoosh excited, tiny nervous laughter. so I will play that version of it now. Zach: Yeah. Christopher Slowe: that's why our speaker notes for this look like that. Marty, I think you sent us the wrong speaker notes before. That's why I was doing all the tiny gasps. Marty Weiner: no, I I did add the the part where I added exasperation as Zack talks again. Yeah. So I'm gonna try again with Kermit here. This is very hard to do. I'm not a puppeteer, but here we go. Whew, as eagle-beaked, crook taloned vultures from the mountains came swooping down, whoosh, upon the smaller birds, all huddled together in little flocks upon the ground, and they just gobbled them right up because the poor little fellows couldn't fight or even fly. And everybody watching actually enjoyed the show. Hmm? Well, In exactly the same way, Ulysses and his men charged the suitors and smote them one after another all around the room. They made this absolutely horrible groaning, uh-oh, as their brains were being battered in. and the ground was just absolutely seething. With blood yea. All right, so I'll keep my day job 'cause I'm clearly not a puppeteer. Zach: Wow. Turns out Marty Weiner: But it improved it quite a bit. yes, no, I'm recording it and and I'm thinking about sending it to Jim Henson. Christopher Slowe: Good thing we're not recording this. Zach: Yeah, thank God. Thank God. Did you do you know I I I read a bunch about puppetry with my daughter and you're supposed to only move the thumb. I think you were moving the fingers. Cause 'cause it No Yeah, we'll just do it yeah. Fix it in post. Marty Weiner: okay. Okay, well tell what, we'll have some offline lessons and maybe do it again. but but I think the key here is it did capture, I thought like a lot more of the Kermit isms, the Jim Henson Kermit bits, and it sounded a lot better, but it also would probably take a lot of coaching to get exactly what you would want. And I believe one of the criticisms yeah. Zach: I I I was gonna say the the the the stuff the choices there were kind of bizarre in places. like or it was like reacting to the ending like, I don't wanna say that and you're like, No, you need to say it like like a Greek warrior shouting in the Marty Weiner: Yeah. Right, right, right. Christopher Slowe: Ha ha ha. Marty Weiner: 'Cause I don't I can't think of Kermit the Frog having much of a really sad moment. He just gets more cartoony, I feel like. and I think that might be a little bit of where Chat G P T had its own artistic license and I might tweak that because if anything I I would make that last part be I don't know, tongue in cheek or silly or Zach: That's right. Yeah. I mean I I can imagine I assume a voiceover person would be like, But that's what I do. I I mean like, you know, if i if you have to keep managing all the annotations on ChatGPT, that that seems to me like a productivity loss, at least with with the current models. Marty Weiner: I yeah, maybe, or it gets to the point where I s I've got a flow and I go to Chat GPT, I say spit this out, I copy and paste and I hit go and I say give me twenty versions and and they're excellent. but yeah, I I I I I'm not necessarily saying this is a good idea to take away from humans, but I also think it is interesting for resurrecting you know, beloved dead characters. And they've already done done this for James Earl Jones Darth Vader, they're doing it for Michael Cain. Zach: Right, right, right. Right. Yeah. That's it. Do you think so like like with James Earl Jones and Michael Cain, they they actually made a deal. like I I guess I don't know what the law would be like if you wanted to bring back, you know, Fred Astera or something. I guess you talked to his estate or? Marty Weiner: Mm-hmm. Well it's there was a big discussion about this, right? For like Jim Henson who died in nineteen ninety. What are the what are the ethics here? And imitate or or replicating his voice with the AI without his permission, but maybe with the estate's permission. But you know, we're now at this point we're talking about an estate that is you know, already down to what, grandchildren or several several further down. Zach: Yeah. You you mean of having someone imitate him? Right, right, right. Christopher Slowe: It's like thirty thirty five years old. Yeah, Zach: W well no, they don't Christopher Slowe: it's Zach: even own it. It Disney owns it now. Marty Weiner: Disney owns it now. So then who who has permission or or who who should have permission is I think an open question. But this is also not a new problem, right? I mean, we've had voice imitators of of past actors, and I I don't know if this is different from that. Zach: Well right, yeah, I guess I don't know if they owned his voice, so yeah. I guess I don't know. Christopher Slowe: Technically probably Disney probably does. Zach: Right. Yeah, I mean I guess the the usual argument from artists is is like i it's it's different in terms of scale, which which might, you know, imm maybe, you know, sort of in a in a legal sense that doesn't matter, but but to a to a human being be you know, there's a difference between like a large corporation's n you know, sucking up all the work versus like an an imitator working to perfect an imitation. Marty Weiner: Mm-hmm. Right. Right. Well, how do we close these things out again? Zach: How do we yeah, we we cleverly go to an end yeah I I feel like maybe we should we should just outro on on some Kermit, but I don't know I don't know what it would be. Marty Weiner: I'm not g I yeah, I'm not great at that. I'll practi okay what, also promise for next week we can have an outro from Kermit. and he'll he'll say the usual things, like and subscribe and all that. Zach: All right, right, great. Yes, yes, yes, right. Like and subscribe. I guess until next time, this has been an Last Humans in the Loop, which we'll we'll keep doing until there are the the job is replaced by a superior AI with good annotation from ChatGPT. Kermit. Good night. Marty Weiner: And until next time. Christopher Slowe: Until next time. Marty Weiner: Right. I'll work on that. Right. A good night. Christopher Slowe: Who can make Kermit sound like Kermit?