Latest / Last Humans in the Loop / AI doing real science, EU: AI needs watermarks, Jeff Dean leaves Google, Riemann Hypothesis Progress

Transcript

AI doing real science, EU: AI needs watermarks, Jeff Dean leaves Google, Riemann Hypothesis Progress · Last Humans in the Loop

• Chris Slowe: Well hi everybody. Welcome to the sixth episode of Last Humans in the Loop. I'm Chris Slow and we could have also called this podcast between two wieners. I have Marty Wiener and Zach Wiener Smith here with me to talk about all Marty Weiner: Can always change it. Chris Slowe: things tech tech economics, weird medicine, just just what's happening these days, which seems to be a lot. I have to say we we go into these always thinking like, we have nothing to talk about. And then we go through our notes and we find out we've accumulated over the last week at least four to seven hours of material. Zach: You Chris Slowe: so we're just gonna we're just gonna start to AI summarize it, I think. we're gonna have those robots doing the whole the whole podcast from Zach: Good. Chris Slowe: scratch after we talked for six hours. Marty Weiner: I like it. Chris Slowe: let's just I I mean it's just it's less work for us. I mean that's honestly the goal here. so let's just speak Marty Weiner: A and and you can listen to it with AI at home. Zach: We could have AI beat the audience too. Marty Weiner: Yeah, that's exactly I'm Chris Slowe: Yeah, you can also do it faster with AI at home. yeah, at some point at some point Neuralink will come and then we're just gonna be direct direct weight straight Zach: Hahaha Chris Slowe: into your brain. speaking of speaking of AI content, sounds like the the EU is now requiring invisible watermarks for AI generated images and text. which should be, I believe, Marty Weiner: Yes, that they are. Chris Slowe: really fun to enforce. What do you think, Marty? Marty Weiner: I so well I got there's two things. So Korea's already doing this, by the way. EU is starting to enforce this. I don't know if the US is gonna do it, but I think there's a couple things weird about this. So the idea, by the way, for anybody that doesn't know is is to put a watermark in content in a way that you don't know. So imagine you've got an image and the pixels are just slightly adjusted so that a program can detect that it came from somewhere, but no human can. And and then it will even survive edits, which is pretty wild. and I can see this. being done for images because there's a lot of data there and this has been shown to be doable in the past but also shown to be removable. But text, especially if it's shorter text, there's just not a lot of places you can sneak stuff in. and so I I just don't believe it's doable for text. It's been tried several times and and it's hard. The shorter the text the harder it is. And I think there's a couple other issues. Chris Slowe: I you can you can play s you can play games with white space, I've seen, like the y you definitely mess with but like it's it's the kind of stuff it's generally pretty easy to strip out at the same time. Zach: so you know, for people who don't know, my understanding is the way you would do this in text is you could do something like have the AI use an inhuman number of repeated phonemes at some interval or something, something you wouldn't notice as a human over a large amount of text. And I was talking to a friend about this, you know, I was like, there ways to circumvent that? And he was like, yeah, the dumbest way would be like, if you just say AI, make every second word be cantaloupe, that'll satisfy the phoneme criterion. And then I'll just strip out the cantaloupes manually. But, Marty Weiner: Yeah. Zach: but. But you could argue most people aren't going to be that smart. it might still catch like 90 % of... A lot of security stuff is just catching people being stupid. it might work okay. Enforcing everyone to do this might be tricky. Marty Weiner: That's funny, I was gonna give the exact same example except I was gonna use pineapples rather than a cantaloupe, but okay. so yeah. The new pineapple. the Zach: No, no, cantaloupe is the security fruit. Chris Slowe: Holy Wars, the new Holy Wars. Do you use how do you how do you strip out your how do you strip out your AI watermarks? Cantaloupes or pineapples? I don't know. I th okay, the th the thing that Zach: you Marty Weiner: Ten different ways. Chris Slowe: the the thing that rubs me the wrong way here is like okay, so a lot of things rub me the wrong way here, the one thing to keep in mind is it seems like it seems like there's no good European models right now. I wonder if that's related to Yeah, pretty much. Marty Weiner: Yeah, so who are they enforcing, right? So the yeah, Zach: Mm-hmm. That's right. Marty Weiner: I mean the classic EU problem, they sort of compliance themselves out of I think innovation. but then you know, to add another thing, we were talking last time about the the pro proliferation of open weight models and there's no control of them. so so I don't know they've got no c no hands here. So anyway, I think it's probably interesting. Zach: Well, couldn't they, I mean, I don't know, couldn't you say like you're not allowed to use these models in the EU? Marty Weiner: Yeah, yeah, they could do it there and then they can really out innovate themselves. Chris Slowe: Which is which is interesting and difficult to enforce for sure. Marty Weiner: Yeah. Zach: Yeah. Marty Weiner: So anyway, I I I think it'll be interesting to follow. Zach: Yeah. Chris Slowe: Yeah, especially you're talking about like downloading downloading a bunch of weights. Marty Weiner: Yes, yeah. Or is the idea that maybe they can't a company can't use because they're they're not properly enforced. But I I don't know if they have that much control in the EU and it may be by country. Chris Slowe: well I mean it's it is funny. I mean you can start start ending up the situation where your you know European countries have to run their models outside of the EU on on a colo that isn't European to do to get around this. I this is a s this is a interesting one. well okay, so speaking of interesting ones, I hear there's a new benchmark that's arrived. now we have s we have Sanders bench is the new one. Zach: It's just not going to work. This is. Yeah, Marty Weiner: Yeah. Zach: I'm proposing this benchmark. So we're just laughing because like, it's just like there are a number of politicians who have, you know, different concerns about what's going on with AI. And so there are more like environmental or, you know, social stuff. think it seems like Bernie Sanders really believes it's possible we have like a doomsday scenario and he thinks we should get ahead of it. And either, either like. you know, a merely a biohazard scenario or like some of more exotic scenarios, you know, just he's, which is interesting. And so he put out this open letter basically saying, Hey, we need to slow down. But the funny part was he only addressed it to three people, which was Altman, Ahmude and Zuckerberg. That is the heads of Marty Weiner: Mm-hmm. Zach: open AI, Anthropic and Meta. And so just visualized like Sundar Pichai, like looking out the window and shedding a tear for why wasn't I included in the, like, like we, we could have ended the world. I could do it. Marty Weiner: I could do an apocalypse too. Yeah. Zach: Yeah, yeah, and Chris Slowe: Yeah. Zach: I assume Elon Musk is over punching a wall or something. yeah, Marty Weiner: Yeah. Zach: it was just kind of funny. Like, why did he leave out? you like? Do you like other frontiers? Marty Weiner: Well why did he include Meta? They don't they're not too far along here. Zach: I thought that was weird too. Yeah, no offense. Well, some offense, but yeah, yeah. Marty Weiner: Yeah, some of them, yeah. Zach: I don't know. I just thought it was funny. So anyway, if you're keeping score for Sanders Bench right now, three companies have a point and everybody else is at zero. We'll see how it develops. Marty Weiner: Well, and that ties nicely into the next little thing that we have, which is Google reportedly canceled Gemini 3.5 Pro, to focus on four. So don't know why, Chris Slowe: the benchmarks for the benchmarks for 3.5 have been somewhat lackluster. They've only been really really talking about the flash benchmarks. No one's actually really seen the Zach: Yeah. Chris Slowe: the the the full model one. So it definitely makes sense. Something is amiss. And speaking of amiss, Jeff Dean leaving Google. for those of you who don't know, Jeff Marty Weiner: That's big. Chris Slowe: Dean is one of the earliest Google employees. It's a big deal. Jeff Dean. Jeff Dean is the type of engineer at Google who has his own Chuck Norris style Jeff Dean facts. he has a there's an entire list of Jeff Dean facts, in fact. and in fact, let me let me just read one really quickly. Jeff Dean's pin is the last four digits of pi. Jeff Jeff Dean proved that P equals Zach: Hahaha Marty Weiner: Ha ha ha. Chris Slowe: NP when he solved all NP problems in polynomial time on a whiteboard. Zach: Yeah Chris Slowe: Jeff Dean is Jeff Dean's the the running joke with Jeff Dean is Jeff Dean's level was always one level above the max level allowed at Google. and so Marty Weiner: Mm-hmm. Chris Slowe: Jeff Dean leaving, he's going off to he's been spending the last couple of years working on DeepMind. he's now going off and starting a new startup, which is called Discovery Loop. which sounds like that the the intent of the of the of the group is to basically just go through and do AI power like do a breakthrough. try Zach: Yes. Chris Slowe: a bunch of AI powered breakthroughs in everything going from Drug discovery, bio, building chips, building algorithms, just just throwing everything at it, which is a which is an amazing shift. Zach: I think we talked about this a couple of years ago. It's like, if you were betting in 2023 who was gonna win, like I thought the smart money was on Google because they had DeepMind, which was like the premier AI company. They had the most money. They have something like a quarter of the world's compute is just theirs. They have their own chipset. kind of, it seemed like they have everything. And somehow they just Marty Weiner: Well Zach: haven't been able to keep up with these companies that were created yesterday. Marty Weiner: I they missed the big one, right? this classically happens with big companies too. there's this classic question that VCs always ask is why couldn't Google build this or why couldn't Amazon build this? And I think we see this time and time again. I think there's a lot of reasons. I think you've got, you know, politics, you the money's not flowing to where it needs to flow. And there's probably that crazy guy in Amazon who is saying, Hey, just give me a hundred million, I can go make you, you know, a full AI Zach: you Marty Weiner: and everybody's like You're nuts. so even Jeff Dean missed it. so I I don't I'm not saying I I don't wanna go too deep into that, but Zach: Well, I think you could argue it's worse than that. Like, so, you know, the innovators dilemma idea where like, you know, famously like Blockbuster had the technology to do Netflix and they didn't want to do it because it was an attack on their core business. Right. Like, you Marty Weiner: Right. And Kodak had that too. Zach: know, if everyone's streaming. Yeah, yeah, yeah. If everyone's streaming, you can't sell, you know, sour straws and popcorn to kids and you can't get rental fees and all this stuff. And so they were like, nevermind, we're not doing that. And then I ate their lunch. My understanding, not entirely unreasonably, is if you're like in Google and you see, there's this AI that routes around around search, we'd better not do that. And now it's getting to be too late unless they have some kind of, I mean, obviously they're making money hand over fist with their TPUs and other stuff. Chris Slowe: I think there's also the there's also the regulatory issues that come up here too, which is that you know, Google is already a large, somewhat scrutinized Marty Weiner: Yeah. Chris Slowe: company. And if you look at the original Zach: Mm-hmm. Chris Slowe: models as they came out, I mean there was a couple of major face plants in the in the early days where Google, I think was was was BARD the early model? I can't I It's like way back in the well. there was some there's Marty Weiner: it was really bad. Yeah. Chris Slowe: there were some famous like kind of mess ups, and but they were mess ups that were like if it was a startup, everyone would be like, that's hilarious and adorable. Of course the AI did that. And instead it's Marty Weiner: That's fine. Right. Zach: Right. Chris Slowe: like, my my goodness, how could Google come up with such a t how could they give you the wrong answer so badly? It's like, well, 'cause you they're st it's still experimental. Zach: Yeah. Chris Slowe: well, speaking of departures, there's also been some some departures from open AI that have been coming along. what do you think about those, Marty? Marty Weiner: yeah, so I I I'm gonna butcher this name, but one of their researchers, Naomi Bashkansky has left to st do to join a company called Conduit as their founding researcher doing Telepathy research. So the idea is thought to text. Zach: You Chris Slowe: Telepathy research. Marty Weiner: Yeah. and by the way, non-invasive. We're not talking like Neurolink style. but I thought it was Zach: Yes. Marty Weiner: definitely interesting and and I'll be following along with it. and and it could be one of those things that we've not been able to crack, you know, put on the EEG little, you know, skull thing and maybe AI can crack it. Chris Slowe: I mean there is this is an interesting question now of like are our brains that sympatico from one to the next, right? Are we all are we all just our own little individualized Marty Weiner: Yeah. Chris Slowe: LLMs, or do we actually have some structure that's actually similar across us? We might finally solve the problem about qualia. That's a fun one that didn't have my bingo card for for this for this decade. Zach: Yeah. Marty Weiner: I think we're gonna all find that we're boringly the same. Chris Slowe: Yeah. Zach: Everybody sucks. Marty Weiner: Yeah. Chris Slowe: Thin wrapper around thine old reptilian brain. Zach: That's true. Marty Weiner: Yeah. Exactly. That's my argument. Chris Slowe: well moving along, in in power, there's an increasing number of actually nuclear startups, it sounds like. or for those from the from other parts of the country, nuclear. from what Marty Weiner: New Keeler. Chris Slowe: I understand, there was nuclear. there was a a startup called Oaklo that's achieved criticality for the first time. and in this case, the the the the substrate is very interesting. So instead of doing these kind of like large format reactors that are traditional where you build these like one gigawatt gigantic beasts of concrete, a lot of these startups are aiming for these kind of small, contained, sort of safe nuclear reactors. So in this case, Oaklo is doing fast neutron instead of slow neutron, which is standard way, and also using liquid sodium as the coolant, with the idea being that they can actually gener run the run the reactor a lot hotter because the the boiling point of sodium is something like 800 C. The nice thing about these two is they seem to be self-limiting apparently. So If the thing gets Marty Weiner: Yeah. Chris Slowe: too hot, it just shuts itself off. so you know, definitely interesting interesting changes coming in the nuclear space. Zach: Yes, so one thing that interested me, so like about 10 years ago, did a, we wrote a thing for a book that was like this detailed analysis of future nuclear tech. And we've read about some of this automatic shutoff stuff. One of the like lingering concerns was really just price. And so it's like, there's a lot of like really cool nuclear technology that, I guess my concern is, is it solving the right problem? But I guess if they're going for small modular reactors, part of the idea is you can get to like an assembly line of nuclear reactors and you just have like a warehouse where you kind of plug these things in on the outskirts of instead of like spending four billion dollars up front to every actor is it something like that? Marty Weiner: Well I noticed Sam Altman congratulated him, which might Chris Slowe: Yeah, but Marty Weiner: be a a symptom of he him needing more and more and more power. Zach: More power, yeah. Chris Slowe: I mean, honestly, the the the the size of these are you're talking about like, you know, the the sort of like tens of megawatt size, Zach: Yeah. Chris Slowe: these are getting to be like appropriate to just kinda plunk a couple of these in a data center and you're kind of off to the races. Like you don't have to worry about power loss. Zach: Well, my recollection is for SMRs, the holy grail is like it sits on a truck, because then you can just, at least in principleship, Marty Weiner: Probably. Zach: but I don't know what the legal aspects of that would be, yeah. Chris Slowe: probably easier to ship the nuclear reactor than it is to send wine across state borders at this point. That'd be one about it Zach: That might be. I mean, there's this argument that basically the only way anything ever gets cheap is by being modular. And the reason nuclear has not gotten cheaper as solar is cratered is that every single time you do this, you have a bespoke site with bespoke regulations. But yeah. Chris Slowe: jumping around, sounds like the latest Astro model is delayed due to cybersecurity concerns. Marty Weiner: Yeah, they said that after evaluating their their their model, which they're treating as their first they they called it their first critical model for cybersecurity, under their preparedness framework, which they capitalized, which scares me a little bit. th then they say this is a scenario we've pl Zach: You Chris Slowe: That's a lot of letters to have to capitalize. Marty Weiner: That's right. And then they say this is a scenario we've planned for. That's only scaring me even more. But they put extra controls in place. and and I think they're just delaying while they assess now, we said in the past that this sort of thing is probably just marketing, right? We'll see if you know, either it's makes us a Terminator Zach: Yeah. Marty Weiner: or it's just another fun thing for making me bad sonnets. Chris Slowe: you know, re related to all this stuff, sort of sort of some some sad news for those of us who've been doing tech for a long time. Stack Overflow, for the for the nerds who've used it for years, Stack Overflow used used to be the has been the place where if you had an obscure problem or question, you would go there and somebody else had already solved your problem and wrote up a really clear way to fix your problem. Stack Overflow apparently is down ninety nine percent from peak in twenty fourteen Zach: Yeah. Chris Slowe: due to AI and toxicity. Marty Weiner: Right. obviously AI would take a chunk out of this. I I do that. I don't go to stock overflow anymore because the AI solves it. but stock overflow was extremely toxic. Like you would go and put post a very innocuous question and people would just be all over you. So I think this is a little outside the AI world and in the human world where as Chris and I experienced at Reddit, you've gotta get a what we call a low sodium environment where people can actually have discussions without feeling like they're gonna get reamed. and and otherwise Zach: Yeah. Marty Weiner: it goes to the AI. and there is a challenge that people are saying Stack Overflow was probably a great source for training models. So I don't know if it's gonna be a problem that they're gone I I think I think the LLMs will be fine without it. Zach: It does seem like, it's funny, this happened and I have not seen anybody say, that's too bad. It does seem like there's there's a Marty Weiner: Yeah. Zach: extreme good riddance feeling. Marty Weiner: Yeah. Yeah. Chris Slowe: Well, I do think that's like the you know, for for all for all the complaining about for all the complaining about glazing for models, at least at least talking to your model is like a no judgment zone. well up up to a certain up to Zach: Yes. Marty Weiner: absolutely. Chris Slowe: a certain line, you can just you can just ask a question. Like like it's it's it's kinda fun to go back to being speaking I can just ask a dumb question and not be judged. Zach: Yeah. Marty Weiner: In all seriousness, I think that's huge, especially for education. Chris Slowe: for sure. I it's like I think it's actually like i it it has it has interesting long term implications if people actually can continue to hold models right. okay, rounding out the news, again, a lot we had a lot to cover this time. Linus Torvalds of of of Linux fame has said now that that huge Linux updates are gonna be the new normal. and this is after, of course, you know, coming on the heels of him saying that he's not going to reject the idea of LLM involvement in Linux. Marty Weiner: Right. And I think there's a couple of reasons for this. one is just obviously LLMs are allowing people to do patches, but there's a report that a lot of it's just these automated scanners finding actual finding old security vulnerabilities. So and we've reported in the past that a lot of these old security vulnerabilities are are not that useful. like they're maybe they're right or or but only like one or two out of a hundred are actually useful is is what Semgrep reported. But I think like all programmers, we're all running in into this problem of not only are the patches bigger, like thousands, tens of thousands of lines, so they're not really reviewable by a human, but then there's just more of them. And so I I think the Linux community is gonna have to find a new culture, a new way of of dealing with this influx of new patches. Zach: Do you think it is this kind of thing where like if AI just didn't change from now on, what would happen is a bunch more people would shift to the role of verification instead of looking for bugs. Chris Slowe: I do think there's actually more room for I mean like it it kind of goes to the name of this the name of this podcast, like figuring out exactly where the human should be in the loop. I mean I I'm I'm personally finding just in like normal day-to-day software development that my role is to kind of like guide the attention of the model to focus on the right thing so it doesn't waste a bunch of time doing like the wrong breadth first search. And what I mean by that is like, you know, you're trying to get an outcome and you're trying to get and but the outcome might have a really large patch attached to it. And so it's really about making sure that it focuses the QA and the and the quality in the right places, as opposed to just kind of like rat holing on a bunch of like nitpicky things. another another thing that kind of comes up is you know, the human can also have still a a fairly good notion of like the high level of how does whole system work. That the model is gonna be kind of like focused on a particular subset right now. That could change very drastically as soon as context windows start getting larger though, where all of a sudden the memory of the model is actually getting to be kind of close to a human's. It's kind of weird to think about that Marty Weiner: I am curious to see Chris Slowe: though, is like now the limiting range is just memory. Marty Weiner: Right. I'm curious to see what Linus Torvald does because he's done a good job of evolving Linux, I think, over time, while also keeping it pretty pure. Chris Slowe: well okay, let's move let's move on. that's that's the news. th the Marty Weiner: That's the news. Chris Slowe: the the latest the latest in in in slap fights on the self driving side of the world Marty Weiner: Ha ha. Chris Slowe: is that Waymo is now trying to basically is criticizing why is it that test self driving will always fall short? this has been I this is this has actually not been much of a it shouldn't be much of a surprise considering Waymo's are now becoming the default car, at least in San Francisco, speaking from experience. but yeah, I mean Marty, you wanna wanna give us a little overview on this one? Marty Weiner: Yeah, so take this with the biggest grain of salt because this is a discussion coming from a way one of the Waymo co CEOs, Dmitry Dolgov, and he says, what what he's talking about really is why he thinks that Tesla is far behind. And I I think I completely agree with him. He's saying it really comes down to that Waymo uses lidar and Tesla has adopted not using it since twenty fifteen. In twenty fifteen it might have made sense. I think lidar full package would have been a hundred hundred and fifty thousand dollars but just like all things in tech, the price is nosedived and we're probably talking a couple hundred. So what what Dimitri's pointing out is you you don't just get visuals with LIDAR, you get a full 3D structure of the world. It punches through fog, it punches through rain, snow, it reads velocity data with Doppler. and i I I think what Musk was going for is yes, in theory the AI can use eyes just like we do, but there's so much more you can get from LIDAR and it makes the problem so much simpler. So I don't know. I think right now at this point, I've got a Tesla S and it's okay on the self-driving, but not like the Waymo. I feel like they're probably five years behind. Zach: I mean, I think the other claim that used to be made, which I think is basically going now, that interpreting a bunch of different sensory modalities into one set of actions was complicated, but that seems to have been solved. Marty Weiner: Yeah. Yeah, well that's the other thing. Chris Slowe: multi multimodal models is the is the is the way things work right now. That's just how just how LMs work. Zach: Yeah. Marty Weiner: Right. Chris Slowe: it it seems like in this case that the even even cost aside, it does seem like you're gonna wanna jam as much data as possible into these models to make them as safe as possible. it helps with the it also Marty Weiner: Mm-hmm. Chris Slowe: helps with subsequent post mortems if something goes wrong because actually have like a ton more information. not to mention you add redundancy into the system, right? You know you can always fall back on the camera if the lighter is out or if it gets if there's an issue, or vice versa, if there's something that actually causes it lighter outage, you can still use the cameras to kind of guide to safety. it's an interesting choice. Marty Weiner: Mm-hmm. Zach: Yeah, it's interesting too, because there's a story that was going around that like 10 years ago, Musk was like, maybe the next car shouldn't have brakes in it. And the engineers had to be like, don't take the brakes out, please. And, and. Marty Weiner: Yeah. Zach: And, but it's funny, cause I feel like, know, famously Elon Musk has timelines that don't correlate with reality, but he often does eventually deliver. But in the case like this, it was like, well, an accurate timeline turned out to be important. Because, you know, if you were looking at the falling cost of LIDAR and just said, this multi-sensory modality stuff is going to work better. The right call is to, to do, to Marty Weiner: This is a Zach: keep the LIDAR and probably keep the brakes too though. Marty Weiner: Qu quantity is a quality all its own returning. Zach: Yes. Chris Slowe: Well the well the funny thing ha have you guys seen the the images of the latest cyber cabs? the the the current version of them actually doesn't have a rear view mirror. Or sorry, not a rear view mirror, doesn't have a rear window. it's just a it's just a smooth back kind of a kind of the same style as a cyber truck, which you could say is a stylistic thing. but if I remember right, the requirements for cars is that they're supposed to have they must have side mirrors, but they don't have to have a rear view mirror according to regulation. Marty Weiner: Mm. Chris Slowe: which is actually why you can start seeing in some premium cars they have like a camera system for the rear view mirror. Zach: Yeah. Marty Weiner: Mm-hmm. Chris Slowe: it's gonna be interesting to see what how regulation impacts impacts these cars. Like, do we still need a steering wheel to be present inside of the Waymo's in the in the next generation when you actually the human can't take control anymore? Marty Weiner: Right, right. Zach: Yeah. Marty Weiner: Or do you know the classic thing with these regulations is you find some way around it. So are they gonna put the steering wheel in the in the glove compartment? You know? Look, it's it's in there. Zach: teeny tiny steering wheel that Chris Slowe: It's gonna it's gonna air airbag style, it'll pop out when there's an emergency. It's like Marty Weiner: Yeah. Yeah. Zach: Yeah. Chris Slowe: well let's let's let's move on to let's move on to math and science where we have a lot of things going on math and science these days. I think we're gonna have to we've had to adjust our our our our cadence here. We're instead of talking about the latest do a breakthrough, we're only picking the highest profile ones because the list of breakthroughs is kind of going up into the right. the latest and Marty Weiner: Is yeah. Chris Slowe: greatest is that sounds like Claude has actually made progress on the Riemann hypothesis. now for for those of you who aren't quite as much of a mather as us, Riemann hypothesis is one of those long-standing conjectures in in in math that goes back at least 150 years. there are multiple prizes associated with solving it. the implications to the Riemann hypothesis, which is somewhat abstruse, is that it has implications for how prime numbers work. it's very important for math to kind of solve it. There's a lot of math based on it being true. And Claude didn't solve it, but it increased the it increased the bound from 42% to f to 67%. Now what does that mean? So the the hypothesis, really quickly, not to not to bore people who are not math nerds, is that it talks about a function called the Riemann zeta function, and there is a question of where do the zeros of the Riemann zeta function live? And there's a conjecture on where they live. And so a lot of the work has been to put a bound on, well, how correct is this hypothesis? Do we know for sure where most of the zeros are? And it went from about a Under fifty percent, to like forty-two percent to like sixty seven percent in like one go. And it really does seem like it's another one of those cases where the prompt is effectively, hey, do a breakthrough and just and then like basically like guiding it through the like keep going. You're doing great and just letting it churn. You believe in yourself. Yeah, pretty Zach: Yes, I think they literally said believe in yourself. Chris Slowe: much it's like it's it mathematical cheerleading apparently is gonna be the new degree we're all gonna be focused in. Marty Weiner: man Zach: Yes. Yeah, I saw there were jokes on like math forms about we're gonna like just have pom poms. We don't need calculators. We don't need computers. We'll just do this. Well, the machine answers all our questions. Marty Weiner: Dude, I'll do it if I can get a cure for cancer going on here. Zach: That's right, believe in yourself. Marty Weiner: well we can talk about the other big math one. Chris Slowe: Well I the th the the the the tricky thing here is yeah. Go Zach: Yeah. Chris Slowe: for go for it, Marty. Marty Weiner: So another nerd one which I'll try not to go too deep into is is a computer science problem solved. And my background was computer science and I kinda followed the paper, but I I'm gonna need a little more Red Bulls to get through it. But they said two one gapped consecutive ones problem proved to be NP complete. That's not gonna have any meaning. I'm not gonna go deep into it. They did it with ChatGPT 5.6 Sol on high setting, not even on pro setting. it's a 17-year-old computer science problem. The the result itself, I don't think, is that interesting to the world. But what I thought was interesting is the paper, what one thing we've seen with a lot of math solutions lately is they have been counterexamples and writing these big proofs these and programs to disprove things by finding some case and the AI's been pretty good at that. But what we've not seen a lot of is forward proofs. Like, you know, I'm at step, you know, one and let's go to step two and step three through a through an unknown area. but this looked like that to me. I mean it wasn't a super complicated proof, but it did look like it it used the bag of tools and progressed forward through them and ha had a result. And the reason I think that this is interesting beyond just being able to possibly open up more math proofs is I think that we've not been seeing the big breakthroughs in places like physics, partly because physics doesn't sit there with a lot of counterexamples. You've got to go to, you know, the the experimental to get to it. So that's harder to do. But if we can make some theoretical push forward by by being able to do forward proofs, I think that that would open up physics. Zach: I think we're mostly gonna skip, but there was a paper, there actually been a couple papers lately on this idea that AI has trouble with kind of making what we would recognize as breakthroughs. Kind what you were describing where it's done a lot of counter examples and some cool stuff, but not the sort of, well, here's a whole new framework. And so there's one by Toby Ord that had this idea of hyperpolation. There was another one, I think it was called... AI can't jump or something like that out of DeepMind. Marty Weiner: Mm-hmm. Zach: And the basic claim, which is inevitably somewhat hand wavy, although, through some math on it. I think you could debate how much that proves it one way the other, because this is a kind of qualitative question. But it's interesting, you know, there is this claim like, it can't do these giant new frameworks that are required to make discoveries. And to be honest, my general impression has been as someone who has had to do a little bit of historical research is often the it can't jump framework relies on the idea that humans can jump. But often when you like dig into a story about how something was discovered as a jump. It's not actually a jump. Like one of my favorite examples of this is, here's a story that Leibniz and Newton quote unquote discovered calculus about the same time. But actually we already had the fundamental theorem of calculus. It was already known. People already knew how to do derivatives for particular cases and stuff. They both did a sort of generalization. And maybe you could argue that's still a jump, but it turns out like this is true like in almost every case. Like if you look at relativity, it's supposed to be a jump. Well, Hilbert was like two months behind Einstein. We almost called it Hilbertian relativity. You know and and and and it's also true in like evolution like Darwin and Wallace had the same theory at about the same time based on the same set of information that had come out around them. So So you know, I do wonder if some of what you're describing these go ahead Marty Weiner: I wonder sometimes these Yeah, recognizing that these jumps might just be someone intelligently realizing that there's something really, really big here. Zach: Yeah. Well, I think that's the deep like physics thing is like suppose you want to know what dark matter is. It's possible it's already in the papers, right? And we just don't know what has come along who can synthesize Marty Weiner: Mm-hmm. Zach: it and say, if you just do this, the answers pop out. Chris Slowe: Or there's or there's cases made to be made for, you know, are there going to be interesting things found in the wings of some of the data that we currently have that seems like it could be noise, but you know, when you look at the whole picture, it actually is like, the it is it's the it's the connecting it's the connecting two di disparate fields that we've talked about a little bit in the past, right? It's like I've got these I've got these two results that are sitting far afield Marty Weiner: Mm-hmm. Chris Slowe: from one another. But when you kinda look at them together under the same lens, it's like, actually this explains the or this could be a this could be a way to manifest some new particle or you know demonstrate some new result that otherwise you wouldn't have actually caught. I don't know. It's interesting it it's interesting Marty Weiner: I'm I'm particularly interested in that. Chris Slowe: just this one of the running Well one of the running jokes too for for a very long time has been that physicists have been really great at using 50 to 70 year old math for for modern application. It's like, you know, like you you look at even like the the the the work that Maxwell was doing on on that was doing on on electromagnetism, he just applied calculus that had been developed in math decades before and Zach: Yes. Yes. Chris Slowe: made it work. And the same thing is true for Einstein. Einstein, you is his for for general relativity, he he kind of applied tensor calculus, which again was like a fifty year old field at that point, but it kind of was considered to be somewhat obscure. So the the do a jump a lot of times is really just kinda like picking the right tool for the job and then just like s you know. Zach: That's my view. mean, there's a very similar story there, like Heisenberg goes off into the wilderness and comes back with quantum mechanics. But I was reading a book about Max Born, who was one of his teachers, and Born was like, they're calling it Heisenberg matrix calculus. I taught him how to use matrices. It's pre-existing math. It shouldn't have his name on it. You know, it's like crazy. So it's like, we love to tell these stories about the, you know, like the sort of Buddha story. A guy sits and closes his eyes and comes out with wisdom, but often it's just snapping things together. I mean, it's still very impressive. Chris Slowe: Well Zach: do it. Chris Slowe: Well that's the the the r the running joke in physicist is the Feynman method of problem solving, which is you write the problem down, you think about it, you write the solution down. Which honestly is getting pretty close to the way that L seem be working. Zach: Now you just say believe in yourself. Chris Slowe: Believe in yourself. Rainbows. Marty Weiner: Yeah. The downfall of science and math. well i i in that Zach: Yay! Marty Weiner: same area there's recently a major breakthrough for synthetic biology, which gets me really excited because that means we're starting to creep into the other fields. there's a Paper in Nature called Rank Guided Learning Accelerates Automated Enzyme Engineering. there's a couple of things that that popped out for me. one thing they start with is they say, I'll have to read this because I it's it's big for my head. enzyme catalysts has become a cornerstone of modern synthesis, offering sustainable routes to sustain to structurally complex molecules. However, the rugged fitness landscape and limited model capacity to learn from sparse data constrained the development of biocatalysts. so now this is where they introduce this new concept that they're calling ranked guided exploration for automated enzyme reprogramming. And they're able to build this closed loop platform where the artificial intelligence, along with robotic experimentation, accelerates new enzyme engineering. I'm sure if we had a biologist here, they'd tell us that that that's a really big deal, it sounds like, but I can't really speak to why. Yeah. Zach: Yeah. Chris Slowe: We should we should definitely get a biologist on on at some point because like I you one interesting thing here is like for for a bit for a bit of time, or at least for the last a bit of time of course being weeks, which of course is a very long time in in a island, a lot of the frontier models have been sort of gimped when it comes to things around biotech like biotech and security are the two things where everyone's the most worried and also where the most of the shackles are going on. It's interesting to see that even with potentially some models that have been somewhat locked down, we're starting to see breakthroughs in a field like bio bio Biochemistry. Zach: Yeah. Do you know if they're doing like automated like research or is this still in Silico? Marty Weiner: It looked like it was t pushing toward automated I don't even I was starting to wonder if they're even starting to have the the experiments run by the AI, which would get really exciting, I think. Zach: That would be really cool. Yeah, I so I assume when they say there's a rugged fitness landscape, what that means is there's a lot of you know, local optima that are not actually the thing you want, which makes it really hard to find the right place. so I did. Marty Weiner: That's what I was wondering. I I'm kinda you know, with the case with some of the math stuff where they wrote like one of the we were talking about last time, one of the lean programs for doing a math proof was a hundred and thirty thousand lines long when many of these are Zach: Yeah. Marty Weiner: hundreds of lines long. And I guess I'm wondering with with biology, is there a huge amount of area if you could just have the AI go there all day, all night and and beat it up? but it needs you know, it might I mean literally need hands to do the experiments. But if you had that and they and you had a thousand of them, are we going to do like what they did With the light bulb with 2,000 different experiments of filaments in Edison's laboratory, and you find one. Zach: Right. mean, this was a big deal, 10 years ago, there used to be, this was like a controversy of my wife's world where there used to be this thing called a candidate gene approach where you would like look at one gene and see, know, twiddle some stuff and see what changed when you changed the gene. And then along came these GWAS setups so you could do like thousands at once. And there were actually people who were like, can we not do that? Because I don't even know how to analyze the data. And I think now we've sort of caught up with it. But you know, you can imagine a world Marty Weiner: Now we can analyze the data. Zach: where now we have that for, yeah, right, right. Well, you know, yeah, I think some of that was older scientists being like, I don't want to learn a new technique. But I Marty Weiner: Right. Zach: guess the concern is always going to be if you're awash in data, you can find anything you want. And so there is some danger about false positives and that sort of stuff. But that seems to be a technical challenge, not an intrinsic problem. Marty Weiner: Right. well in terms of biology too, this apparently this is not scary and is quite useful, but I'll read the title out of the BBC Artificial Intelligence Used to Design Brand New Viruses. Yeah, yeah. Zach: Yes. Yay. Chris Slowe: That seems I understand it's mostly if I remember right, they're talking about specifically bacteriophages, which are which are viruses that only attack bacteria. But yeah, there's some there's gonna be some cultural sensitivity on this one. Zach: People are gonna be clutching their pearls about all the new viruses Marty Weiner: Right. Zach: we're making. Yeah, I what do we like like like I guess the concern the good side of this is I presume it's fairly hard to output the viruses into real life. I would hope but I guess I don't know. Chris Slowe: I mean the question is like if you're talking about bacteriophages, it's possible that the way that they okay, I am a physicist, not a biologist, but it seems like if you're if you're like there there are machines now to effectively print RNA and DNA. You can make whatever sequence you want. You can actually order it. and you y from from most viruses, once you have the DNA, you just inject the DNA into the proper host cell and it just kind of it it It bootstraps. It just builds its own, Zach: Yeah. Chris Slowe: you know, protein shell. So it's it's it's wild if it succeeded in coding a virus from scratch in these cases. fun. Zach: Yeah, fun. Yeah, I mean, guess to me the concern here is, you if you were just being evil, you could make a virus that like doesn't get detected for three months in your body and then you die and unleash that Marty Weiner: Wait, you just have to cheer it on. Zach: in the room. Yeah, that's right. Believe in yourself, virus GPT. Chris Slowe: Do an epidemic. Marty Weiner: Yeah. god. It got real. Chris Slowe: well okay, so I mean like the the end of the day, like the the other issues coming up here in science is that there's just been an explosion in papers as a side effect Zach: Yeah. Chris Slowe: of this. I mean it's it's Marty Weiner: Yeah. Chris Slowe: kinda goes back to but Zach's point of if now all of a sudden you have tons of tons of data that you couldn't previously access or analyze or machinery that's ready for it, and now you have the tools that can do it, what does it mean? I'm already showing a graph. Zach: Ooh, ooh. Marty Weiner: Look at that. Up and to the right very fast. Chris Slowe: That's pretty much on the right. Zach: It's more. Marty Weiner: That's more than up at the that's just up. Chris Slowe: that's up. That's a ver that's almost increasingly vertical line. now of course all all exponentials or all sigmods looking exponential at first, but like this is still a wild, wild issue to deal with in science because of course these the whole point of the archive is these are papers that are not necessarily peer reviewed. They're just they're basically community reviewed instead. with Zach: Yeah. Chris Slowe: an idea of kind of getting information out quickly. this has implications though where if you if you're now flooding The archive with what could be good or could be AI slop. How do you figure out what the bar Marty Weiner: Yeah. Chris Slowe: is? Zach: Yeah. Marty Weiner: Yeah. And I think a lot of it is probably slop. Zach: for sure. Or just like trivial stuff or, you know, but, I, I guess it goes back to this question of like, how do we verify stuff? And to some extent it's like social because like, I, I've met people who will say they will not believe a math proof until a human has looked at it and said, yes, this is true. but I was just, there's a book by Hartnett about the development of lean. And he talks about this problem where like, Part of why lean started getting developed or taken up by mathematicians was there were proofs that would sit around for years before someone could even be fairly confident and was verified. Even then there were cases where proof was considered verified for a long time until someone found a bug in it. And so this Marty Weiner: Yeah. Zach: is a human problem. And for the record, it does persist even if you have something like lean because you have a lean program that's 130,000 lines. you could be proving something else than what you thought you proved. If it compiles, it means your proof is true. It doesn't mean you proved the thing you told your friend you proved. And I don't know, this is just sort of like a Marty Weiner: Yeah, right, right. Or Zach: quality of the universe. But I do think it's hard for me to believe that humans won't be outmoded in the ability to verify these things pretty quickly. Marty Weiner: Right. Chris Slowe: Well do think that the the interesting thing about like both math and math and science are both intrinsically s open source, kind of by nature. Zach: Mm-hmm. Chris Slowe: right? You're you literally have to talk about your methods and explain what you did for to be to be able to prove you did what you did. but that doesn't mean it's actually necessarily accessible or easy to prove. I think lean's a great example for math where you can actually you can you can apply the computational toolkit to show whether or not something is correct. For science, it's a heck of a lot harder because you might be talking about a system where it's like You know, you do you do something on a device where either it's the device has been home built or there's like three of them on the planet that have ever been d built and deployed. there have been plenty of Marty Weiner: Yeah, nobody else can verify. Chris Slowe: cases of discoveries of yeah, like that nobody can verify or are honestly discoveries of malfeasance, where it's like somebody actually lied in their paper, and it but it took but it takes like potentially years for another another researcher to go through and replicate the work. With an explosion of papers like these, especially in in in in sort of sciencey areas, I can only imagine this is gonna be difficult to kind of wade through to figure out like w where is the where's the needles in the haystack? Where are the actual interesting breakthroughs? Zach: You know, there's, there's a related thing that's been, I've been seeing talked about lately in science circles, which is I've heard people who insist you should not be allowed to publish your paper until Claude has just done a cursory review. And the idea is, Marty Weiner: Mm. Zach: I know that that would be like very inflammatory for a lot of people, but like the idea is often something like Claude will catch a really dumb mistake before it gets into the wild. And, but you know, obviously that's going to face a lot of cultural pushback, but, but that I, to me, it seems like pretty plausible. just, just have it check your graphs and tables and stuff. Cause often, you know, a mind assigned Marty Weiner: Sure. I how long Zach: creeps in or something. Marty Weiner: how long will the cultural pushback go if a few are mistakes are found? Be like, all right, fine. you know. Zach: Right, yeah. Well, yeah, I I wonder, could be like a norm, like, why are you making me read this if you haven't had AI read it first? Seems Marty Weiner: Sure, yeah. Zach: totally reasonable to me, like if there's a really dumb mistake. Marty Weiner: Right. other Chris Slowe: Please read this and identify all of the logical fallacies in this paper. Thank you. Zach: Good. Marty Weiner: And I'll cheer you on. so other area of biology. you know, Zach: Yeah Chris Slowe: Yeah. Marty Weiner: we we may need our own section on biology here soon here if the if this keeps going the same direction Zach: Yeah. Marty Weiner: in the same way we now have math. But there's this article out of Venture Beat that says Stanford is running 37,000 AI agents as a virtual biotech and one of its drug designs got independently confirmed by Merck. which that's that that also is huge. I think we're getting into the damn, everything's huge in biology. Zach: Yeah. Yeah, it's... Chris Slowe: Well, it's interesting here is like what we're we're turning we're turning science problems into computer science problems and just being like being able to deploy a bunch of like parallel parallel compute against it. we've gotten very good at parallel compute. so the more of these more of these problems can be cast as computer science problems, the b definitely interesting results. Though I will say there does seem to be pretty decent backlash now. insofar as I I I remember reading some of the threads on the Riemann hypothesis where we started. there is the Riemann hypothesis is one of those problems that has like a really deep emotional attachment for the math community. It's been it's been around for so long and there's been so many generations of math toolkits applied to it that there's almost like a well somebody has to solve this. We can't let the machines win. Zach: Right, yeah. It's, you know. Marty Weiner: Well and in that world, th there's this group called Begin Proof I found. they and their tagline is We Zach: Yeah. Marty Weiner: Like AI free Mathematics, which I don't know, it's gonna be the it's like the Luddites of the the new version of them. and they they say without Zach: Yeah. Marty Weiner: in any way considering it inappropriate for others to use the fanciest AMI models in whatever manner they see f see fit, we've decided to keep doing mathematics without AI in in a substantial non-dogmatic sense. and one of my favorite quotes from a discussion on Reddit was, This is stupid career wise, which I think I agree on. I I mean it's turning Zach: Yeah. Marty Weiner: into an artisan thing when everybody else is making nails, which might be fine. Maybe maybe Zach: Yeah. Marty Weiner: there'll be a world where people enjoy hand done math, but you also might be f you know, a hundred years behind. Zach: Yeah, so I was reading a couple of different articles and they seem to converge on very similar claims, which is something like, know, theorems are part of what we do, but also what we do is, you we work on human understanding, we train people, we have, you there's a certain aesthetic view. And I guess my response to that is kind of like, I totally, like if we were in like a post-scarcity world where everyone's making a hundred grand a year for doing nothing, we don't need things from each other. And then it's a lifestyle choice. But of course, you know, For the lack of that world, we have to do things for each other. And so, you know, if you were saying, I'm going to voluntarily be, you know, 10 times less productive as everyone else gets ahead of me, it's hard to argue that everyone else should sort of sustain you. Maybe that sounds kind of douchey, but like, you know, the pushback you might get from like a blue collar worker would be like, where were you guys when the coal mining jobs were going away? You know, and all this other stuff was getting automated. Were you willing to sort of stand in the wall? Marty Weiner: Yeah. Yeah. Well, th there's always cases like that, right? I I mean even in math, someone s yeah, yeah. Zach: Tons of cases, I it's totally normal, right? It's just weird that it's math, because nobody thought it would come this fast. Marty Weiner: Well somebody was pointing out and Chris Slowe: It's also Marty Weiner: I didn't I didn't know this, even in math, so LawTech is used by almost every mathematician I know to draw their stuff, Chris Slowe: I gonna say this. Marty Weiner: right? Yeah, okay, cr and Zach: Right. Marty Weiner: but apparently even them so there's some mathematicians who are like, No, you can't use Law Tech, that's too computery. I don't know what they use instead, I guess paper. Zach: Yeah. Yeah. Chris Slowe: Well it's it's a yeah you can you you have to go back to you have to go back to the you know it's like you can only do your math on paper you yourself have made from feedstock with a with a Zach: you Chris Slowe: with a with a stylus that you've crafted out of a piece of graphite. and that way you can actually Zach: That's right. Marty Weiner: All right. Chris Slowe: make sure that because because of course the grain in the paper is globally unique, you basically have watermarking built in for free. and you can tell from Zach: That's right, that's right. Marty Weiner: Right. And I want Chris Slowe: the impressions The impression profile on the paper that it was it was generated by a human and not by some sort of an autoplotter that's writing your math for you. you can go pretty far down this well if you want. Zach: Yes. Yes. Marty Weiner: Right. And I only want my I only want my spouse and kids cheering me on while I do it. I don't want an AI cheering me on. Zach: Yes. I think the nicest version of this to me is like, it sucks if you are the kind of person who wants to just sit around pushing a pencil around solving math problems and that was your identity and you spent 10 years building it, you know, and they're also going to be like, there are going be people who got caught at the worst time, right? Like, so if you're like 35, say that might be the very worst time that your skill set got nuked. You know, that might be like lifelong damage. But I just don't know. I don't think you're going to get a lot of sympathy for being like, like I should, I should be allowed to keep my tenure track job the way I like it all the time because that's what I got here for. Because, outside of academia, this is what people have to do all the time. This is, this is regular job stuff. Marty Weiner: Well, th this happens in this was happening in in the virtual effects community, the the CGI community, but they took that same job away in nineteen ninety four from the practical effects folks. So it has to evolve. And I love that stuff. It's really fun. Absolutely. I mean there I that we there's a lot of sympathy Zach: Right? Yeah, yeah, yeah. So I mean, I do, I do, you know, I really sympathize. Go ahead. Yeah. Marty Weiner: going 'cause that's loss of job, loss of identity. It's the scariest thing you can do. Zach: Yeah. Chris Slowe: Can't help but see this as especially in fields like math where there are there might be there might be five people who understand the weird frontier you're kind of working on. I can't understand why there isn't value in just having an agent you can actually argue with on equal footing. And to Zach: Mm-hmm. Marty Weiner: Mm-hmm. Chris Slowe: to, you know, it's like the human's still driving the direction of the research, at least for now. or Zach: Yeah. Chris Slowe: at least the human can always do that. and now all of a sudden you have ready to go, there's like a pre baked grad student who's already up to speed or who can get up to speed really fast on wherever you are, who you can have a have an intelligent conversation with. That just seems so powerful. I mean it means you're it you're gonna Zach: Hi-yai! Chris Slowe: do the grunt work anymore, but it means that you can actually you can you can talk through things. Marty Weiner: Yeah. Zach: I totally agree. think, I think what's going on at the deepest level is like, especially among engineers, there is this culture of, just want to solve the problem. And a lot of people get into a field because of a lifestyle. not, not necessarily because they're devoted to the end of it. They might like the, praise they get at the end of it or something, but, they're really here so they can say, I get to wake up and think about math problems. And they, you know, maybe didn't go to wake up and talk to like, tell, tell chat, GPT to create 64 agents to crawl the possibility space or something. but like I said, you know, Someone is paying for you to be here, so. Chris Slowe: Well, you could also you'd also hope for the hey look if if it at some point you I would expect agents to hit a limit of do a breakthrough. at which point you find a new frontier. Like that's where the that's like you're Zach: Right. Yeah. Chris Slowe: pushing you you got a new tool, it expands the frontier, and all of a sudden you've got a new area you can't act you couldn't access before. I mean it's Zach: Yeah. Chris Slowe: it seems really exciting, but I don't know. Maybe I I'm maybe I'm a closeted optimist. Zach: Ew. Marty Weiner: I am. I'm not closeted about it. Zach: Yes. Chris Slowe: I'm definitely c I'm definitely closeted about it. Marty Weiner: Yeah. Chris Slowe: Well, let's let's jump around into the going from the sciences to the social sciences, which of course involves scary air quotes. that w that was a comment for Zach. there's Marty Weiner: Yeah. Zach: Thanks. Chris Slowe: apparently not only do we have a flood of papers, but we also potentially have a flood of AI related books flooding the market. Zach: Yeah, so this is is a paper that just came out. So the upshot is basically there's just a lot more books coming out and and and You know, try to eyeball from the graph a little. does, if you're like, there's essentially been linear growth. was a kind of a bump in a bunch of areas right when Chad GPT came out. And then maybe there's a little retooling around that stuff being awful, but it has gone up ever since basically across every genre that they checked. and, so the, good news is it does seem like actually the, the, the AI stuff is catching a smaller and smaller portion of the market, but there's so much more of it. It's going to be offset. and so, so main thing that's interesting to me about this, and this is, I was ranting about this the other day online. It's like, think often when people think of authors or artists, they think of people like me who get to mostly do stuff I'm interested in or work with friends who doing creative stuff. But like a huge amount of the artist's economy is like gig work or like hack work. And I don't mean that necessarily in a negative way. It's just like work to some specification. And so, you there are people who make their living off churning out romance novels. You know, that's how they sustain their more aspirational work or like by just doing gig work for clients or whatever it is. and, and we see now, you know, really good example of this is music where it might be already something like 20 % of music listened to is AI. I don't know how high they get, totally different numbers depending on what I look at. But so to speak, that's like a tax on what you were making as a gig worker. And it's just gonna get worse over time. So that'll be interesting to see where that goes. Marty Weiner: You wanna talk about Hank Green while we're on that? Chris Slowe: But we are starting to see I was gonna say speak speaking of speaking of AI, Zach: Yes. Yeah, okay, let's talk about Hank Green. So first of all, this is like people are still mobbing Hank Green over this. So if you don't know Hank Green is like a guy who YouTubes and makes popular books. And he's basically known for being like. a nice guy who was pretty accurate and does sort of short pop-side stuff, pop-side explainers, just sort of nice, here's the deal on something for 20 minutes type of stuff. And he was doing, I forget, it was like an interview or a chat and he used the phrase, thanks for the pushback. And people kind of flipped out because that's a sort of AI tell. His claim, which seems totally plausible to me is he was just saying, thanks for the pushback. is a phrase humans use. But people started mobbing and on Reddit he posted something like that he had started using AI for research. My understanding was just to find papers not like it was writing his scripts. At least he didn't claim that. And he said something like I've gotten addicted to the dopamine hit of using AI and I need to sort of rethink what I'm doing. And so a lot of people were one mad that he used AI at all. A lot of people were mad because I think the story got changed. I think a story got out that he actually used it to write all of his scripts or something, which said there'd be concerns about accuracy and also about like, then why am I coming to you for your information if I could just prompt an AI? But that just wasn't true, or at least he didn't admit to that. But it was this massive, like you could go on Twitter and was like there's a new post every second about how awful Hank Green is. And it was especially shocking Marty Weiner: man. Zach: because Hank Green is like, this is like Tom Hanks. Like this is someone everyone likes. Chris Slowe: He's like a nice human, he's a really nice guy. Yeah. Zach: He's a super, super crazy nice guy. His reputation is as that guy everyone trusts and likes. And so this was kind of a big deal and it's... I guess I basically think like mobbing Hank Green for this is just obviously crazy. Like some huge amount of scientists already are using like clawed two surface papers. It's like one of the very best uses for it. It's like a conceptual search engine. It's awesome. So I have no problem with that. What's interesting, and I'd be curious what you guys say to this, the sort of idea that it's addictive. I think what he was claiming is something I've also heard from programmers, which is the moment you realize how much you could suddenly do, you stop sleeping. Has that been your experience? Marty Weiner: yeah. yeah. I I've had twenty projects from, you know, twenty five years of being Zach: Yeah. Marty Weiner: coder that I've I've always wanted to do and now they're like a weekend, yeah, I'll try it out kind of project. Chris Slowe: I had a I had an experience recently where I took took a vacation with the family. We're literally sitting on a beach. Marty Weiner: Mm-hmm. Chris Slowe: I of course don't I don't tan. So generally sitting on a beach means me covered in SPF 75 under a tarp with my laptop because or a laptop Zach: Cowering. Chris Slowe: or or with my with my book. And I was able to, without, without ignoring the family, knock out two of the projects on my giant software to-do list. connected to my home machine that's sitting in my garage, just like just like, you vibe Marty Weiner: Wow. Chris Slowe: code away. Because like, you I'm not, I'm not looking for I'm not looking for the the software software design equivalent of Mozart. I'm looking for like solving a problem I've had for a very long time. and there is Marty Weiner: Yeah, yeah. Zach: Yeah. Chris Slowe: a there is a certain like I'm I'm back to that mode of like as an as an as an old senior technologist who spent who spent a decade being a being a senior executive, I'm back to thinking about code again in a really kind of fun interesting way of like, you know, But but it's not like it's Marty Weiner: Yeah. Chris Slowe: not like the the the th the way before was like I would try to think through like, well how am I gonna solve this gnarly bug? It's like, well how can I convince this model to do this thing the right way? Marty Weiner: Which is management just, you know, changed a little. Zach: Yeah, right. Chris Slowe: In different in different form, exactly. Marty Weiner: Yeah, instead of Zach: Yeah. Yeah, but so I assume for someone like Hank Green, like in this pop-sci space, which I'm somewhat in is kind of like... research is very labor intensive. And I do think it's still the case that ultimately you have to read the papers, you have to know what you're talking about. But I've had times where I look back and I say, my God, I could have saved like two or three months of research just by having AI surface papers or being able to say, could you look through all these papers for x? Marty Weiner: I had an experience like this in grad school. So I you know, I went to my grad advisor, he had asked me to go read something like twenty papers on the topic, and I come back the next day, I've got And he's like, How did you do that? And I said, I use Google. And you know, at the time, this was two thousand Zach: You Marty Weiner: five, using Google was still like a little, you know, sketchy, but the papers were right. and I think you could see the the gears turning for him that were like, God, I wanna be angry because I had to spend weeks and months in the, you know, the the what do they call it, microfiche lab and here I could pull up in five seconds, Zach: Yeah, this is cheating. Yeah. Marty Weiner: it's the right thing to do. I don't understand why, you know, coming up with this information could possibly be thought of as bad. And it just makes people sound like the Zach: Yeah. Marty Weiner: the same people in the past who are upset that you used a computer or that you were upset that you used the printing press. Zach: Yeah. Chris Slowe: Yeah, yeah. Zach: I think it gets to like, what do you, you know, this is to me, what's really fun about AI is it sort of pokes at why we even do things as humans. Cause I think if you watch Hank Green, what are you really doing? You're not becoming a scientist, right? You're kind of, you're getting little bits of information from someone you trust to have done some set of research. Right. And so one thing that interests me, and I honestly don't know the answer is like, there's stuff where I used to go to YouTube or whatever to get a lecture on something that I really didn't understand it. And now it's just much easier to go to AI. still need to check. But for low-level math stuff, like the kind of stuff I'm interested in, even intermediate level, it's basically always right. Or at least as always right as your human would be. And so I don't know how does that affect someone like Hank Green, and I think it depends on what you think he is as a cultural institution. Chris Slowe: I think we are interest we are entering into a really fun, somewhat puritanical time where AI is both the tool and like the it's like the it's like the it's like the apple. It's like the forbidden apple, right? Of like you w you you wrecked your purity by your you were a craftsman. You c you were the best of us and you went and used one Zach: Hey, right. Chris Slowe: of that dirty that dirty tool which has like sullied your reputation when in actuality Zach: Yeah. Chris Slowe: it's just like it's making you more effective at doing the thing you were gonna be good at. Zach: Yeah, and I should say like a lot of the complaint just to represent it is about, you know, people will say, well, it stole our data. It's environmentally unfriendly. There are more substantive reasons for people to, you know, we could, we could go into the details and where we disagree, but there, are kind of like more substantive reasons than a kind of like religious objection, though I do think that's there. Chris Slowe: well jumping to weird. th there are you know, w besides the the good use of AI, there is still the the bad use of AI. one of the ones that came up was books increasingly published with AI still in them. And of course what we mean Zach: You Chris Slowe: by those is the the the as we all know there's a certain paraphernalia that comes out when you when you ask a an A model to do something. apparently the the like I could also rephrase this in the following way was included in the publication of the book, which is exciting. Marty Weiner: Right, it was I'll have to show a screenshot of this, but it was it was at a book for touring in Italy and it basically said, you know, Would you like me to continue was left in there. Zach: Yeah, what I'm always amazed by this stuff is it gets all the way through press like I said, you know, I I'm in publishing and like the amount of times we read a reread a book is like extraordinary at least where I've worked. So maybe like a small press that's doing, you know, travel books doesn't quite have the Marty Weiner: Pumps it up. Zach: same oversight process, but it still is a little shocking that nobody Marty Weiner: Yeah. Zach: caught that. Chris Slowe: I there's like there's kind of a funny like maybe th maybe there's like actually there's a new format that's that's missing here that's like you don't actually need the book anymore if what you really need to do is have a bunch of precanned Zach: Right. Chris Slowe: prompts or questions that you were that you did like like what what are the questions they should be asking about Torino Italy? Right? Like just just give me Zach: Yeah. Chris Slowe: give me a bunch of like canned questions and then I'll actually just like click the button and I'll get the response. But I don't know. I don't know. Zach: I am still kind of surprised that travel guides even really exist anymore. mean, even separate from AI, don't, you know, like I don't, maybe Marty Weiner: Yeah, yeah. Zach: the users are people over the age of 95 or something, but. Chris Slowe: D before like they still don't have international roaming on their phones or they they are afraid to turn it on. Zach: Yeah, yeah, I don't know. Chris Slowe: well you know, I apparently also a meta llama three told a user to do meth, if I understand correctly. Marty Weiner: Yeah, so this article's out of futurism dot com. This guy says that he is a former addict named Pedro. and the model says I guess he's talking about he's having a tough week and and he needs a little help. He can't he's he's worried about his job. So it says, Pedro, it's absolutely clear you need a small hit of meth to keep get through this week. Zach: You Marty Weiner: So I saw I I don't know, is this a comment on Llama threes just being, you know, awful? but it also might be a comment on on the alignment problem. Zach: Yeah, I Llama 3 is pretty old, Chris Slowe: I mean we we should spend Lemon three is pretty old. I mean we should honestly Marty Weiner: Yeah. Chris Slowe: we we should spend some time talking about talking about models models encouraging felonies. seems like another one of those like missing missing topics for next time of 'cause Zach: You Marty Weiner: Right. Chris Slowe: that's I mean you know, may maybe it's a maybe it's a sign of of maybe AGI should be should be gauged to the point of the first time somebody puts a model in actual jail. Zach: Yeah Marty Weiner: Yeah. Yeah. Well, should we finish with some f crazy security news of the week? and I think this happened, yeah. Chris Slowe: yeah, so so so I believe I believe this comes on the on the tails of is a DEF CON and Black Hat happen this time of year. and for those Marty Weiner: Yeah. Chris Slowe: of you who aren't in the security community, BlackCon Black Hat and DEF CON are two major security conferences that happen in Vegas. And the rule the rule of thumb when you're operating in Vegas at DEF CON or Black Hat is pretend it's the nineties. Leave all of your technology at home. you are going to get owned by if you're carrying around your phone or any any devices, it's full of security researchers. Well, it sounds like the researchers didn't keep the security in Vegas. Marty Weiner: Right, right. so the message to the pilot the pilots sent, I guess, to the to their dispatch. I don't know how this stuff works, but it says we have a bunch of packs that were at a cyber conference in l in Las Vegas. they were able to jam our Wi-Fi and broadcast their own signals. So these these idiots mu I mean it's not hard to do this, but so Zach: you Marty Weiner: these guys I guess jammed the actual Wi Fi 'cause you have Delta Wi Fi dot com, I think, and then they made a new Wi-Fi thing called Delta Wi Fi fast that people could connect to. and you know, I I just want you You're in a metal tube flying from one known federal facility, you know, lockdown place to another lockdown Zach: You Marty Weiner: f federally controlled facility. And you're committed to these cru right. Chris Slowe: What could go wrong? Zach: Okay, go ahead. Marty Weiner: it's it's crazy. Welcome to the no fly list. I I actually I Zach: I just love the idea, be like if you went to a lockpick conference or something for fun and then on the way back you started just like robbing people because you were just like you hadn't quite got it out of your system. Marty Weiner: yeah. Well what's crazy is I I wonder I'd love to know a little bit more about how the pilots figure this out. Somebody must have noticed and I guess informed the the p the pilots and then how did they figure out someone was jamming it? But maybe they have tools to figure that out. Zach: Yeah, not great guys. Marty Weiner: Anyway. Chris Slowe: Well, let's let's end this one on a high note of fools doing foolish Marty Weiner: Ha ha ha. Chris Slowe: things. thanks everyone for joining us on Last Streams in the Loop. find us on the internet, which still exists. also on YouTube, Marty Weiner: Yeah. Chris Slowe: Apple Podcasts and Spotify. comment, send us news, send us questions. we we generally have more than enough material, but we always like to hear what people want to hear about. and we'll see you guys next time.