Latest / Last Humans in the Loop / "How is AI changing math?" with Math Professor Jonathan Wise

Transcript

"How is AI changing math?" with Math Professor Jonathan Wise · Last Humans in the Loop

• Marty Weiner: All right, welcome, John. I'm gonna do a quick little intro about John here. I've known John forever. Zach and I have known John since middle school. What I remember most from middle school is how I was just way better at basketball than John. I I I might be remembering wrong, but John is a theoretical mathematician. He his bachelor's was at Stanford, his PhD was at Brown. he's now a professor of mathematics at the University of Colorado Boulder. his research is in algebraic geometry, which means there's only like five people on earth who are smart enough to understand what the hell he does, and I'm not one of them. so welcome, John. Did I miss any bits of that intro? Jonathan: thank you for having me. thanks for that Zach: Ha ha ha. Jonathan: that introduction. I've known I've known Zach: For whatever that was. Jonathan: Marty for long enough that that he's he's one of the few people who still calls me John. I everybody everybody else everybody else calls me calls me Jonathan. Marty Weiner: what what do you go by now, John N the h Professor, sorry. Christopher: sorry I'm sorry, Professor Professor. it's nice to meet you. Jonathan: You know, I tell my I Marty Weiner: okay, so Jonathan: tell my students to call me Jonathan and which is fine and some of them can't handle it and they call me professor, but the worst is when they call me Mr. Wise. That's just That's exactly yeah. Yeah. Here Professor Doctor. Yeah. Zach: It's doctor wise. Marty Weiner: it's doctor Professor Wise. Yeah. All Christopher: Yeah. It's at least Doctor. At least Doctor. Zach: Ha ha ha. Marty Weiner: right, well Herr Professor Doctor, thank you for joining us. what courses are you teaching this semester? Jonathan: What courses? I'm teaching one class. Well, one and a half ish. Marty Weiner: Well huh. Jonathan: yeah, so w we're we're running this this like AI class where we're all trying to to learn and teach AI at the same time, and several of us are doing that together. so that's one class. And and Marty Weiner: That's awesome. Yeah. Jonathan: the other one that class is awesome. I'll tell you more about it later. the the other one is this like introduction to proofs class. So like students need to learn how to write proofs. It's kind of the bread and butter of mathematics. And and we you know, if you come from calculus, you have next to no experience with that, so you have to introduce people Marty Weiner: Yeah. Jonathan: to doing it. But we're doing an experiment there. We're trying to do it with formal verification, with with lean, which is a new thing to try. That's taking a huge amount of time to figure out. But that's what I'm doing. Marty Weiner: I really wish they had taught a course like that at my at my grad work. You know, they sort of just assume that you know how to to write a proof and nobody knows how to write a proof un until they're taught. Christopher: Well they well they kinda they kinda jam it in like in high school in geometry. But geometry always felt like the Marty Weiner: barely. Jonathan: Kind of. Christopher: side effect is being this kind of like off ramp, right? It's like you kinda like you do geometry in between two two years of algebra, which are completely different like in in terms of style. And then you then Jonathan: Yeah. Christopher: you forget about it 'cause it's like s it's like freshman or sophomore year. Jonathan: Yeah, I mean some people remember it, but most people just forget. And and and the I mean the style of the proofs that you write there is just like I don't know, it's it's a few it's a few sentences and they're direct applications of of some you know, side angle side or whatever. And y y an actual proof that a mathematician actually writes looks more like a an essay that you would write in an English class. And you have to get people used to doing that when they're accustomed to writing a sequence of equations in a to solve a calculus problem. Marty Weiner: Right. Well let's let's delve in so I I I wanted to have John on so we could talk about how AI has been changing the field of mathematics. We've been talking about every other space, including mathematics, but I really wanted someone on who who is deep in it. so first question, John. first I wanna say I have a bone to pick with you. I sicked Chat G B T at your PhD dissertation and a corollary three point five I'm sorry, I'm using the wrong damn tab. I'm supposed to be using Jonathan: Mm-hmm. Marty Weiner: this is the berate the shit out of John tab. Let me find the John is my buddy tab. Okay. Alright. Sorry. Hey John, thanks for joining us. It's good to have you. Jonathan: So I so Zach: That's Jonathan: you could seriously do that. I don't maybe you actually did it, but like it it's it's full Marty Weiner: I I d well I I well, okay, I I Zach: Ha ha ha Christopher: Yeah. Jonathan: of errors, I'm sure. Yeah. Marty Weiner: Well okay, I I actually did do it but it was it was I couldn't read half of it 'cause I didn't even Christopher: I mean honestly Marty Weiner: underst Christopher: honestly earlier versions of earlier versions of of the various LLMs, one of the one of the fun tricks to do is to actually take somebody's paper, run it through the LM and say, identify all the logical fallacies. It's a hoot. Jonathan: Yeah, yeah. Zach: So Marty Weiner: Okay. Yeah. Well actually that's we'll get into that 'cause that's one of the questions I think that pops up around how AI is helping and not helping math. so real quick, for for those listening, I picked out like three of what I felt like were the three big touch points in AI math for me. and then I I'd love to John if you want to add some, but the first one I had was back in January of this year. that was that was sort of like what I call the the holy shit moment. the AirDish seven twenty eight problem dropped. this was like the first widely regarded as an autonomous AI resolution of an open airdish problem. And that was that I think there there were problems solved last year and before, but I feel like that was the one where everybody started freaking out. then July was the other one that we that I that we all found amazing and fun. the Fable Five provided a counterexample to the Jacobian conjecture in the most amazing way possible. This guy wrote a tweet that said, hello there. The Jacobian conjecture is false, thanks. You know, with an X. He didn't yeah, he didn't even spell Zach: Thanks for the next. Yeah. Marty Weiner: thanks out. Thanks to my close friend Akil for asking about it, and my other friend Fable for working during the World Cup final. So, I don't know how many other big math proofs have been solved by, you know, you know, pushing a button and going and watching the World Cup, but that was that was kind of amazing for how little work that guy did for for the for the proof. And then the other one I pulled out was very recently was the Riemann Rieman Riemann hypothesis thank Jonathan: R Riemann. Yeah. Remann. Marty Weiner: you. the the making a little bit of progress on that. And I think John, you sent me that one because you thought it was interesting because it was more of a case of the first or one of the first positive proofs rather than the others, which were more counterexamples. Jonathan: yeah, that's true. Marty Weiner: Yeah. Jonathan: so your question is what are the what were the big touchstones for for for map? Yeah. Marty Weiner: Yeah, I I I I think yeah, I asking if you haven't got any others and I think toward the question of how big is it that these conjectures are getting knocked over? Like w how is this changing things for Jonathan: Well Marty Weiner: your field? Jonathan: I mean th there are a lot of things happening and and the maybe the more interesting things are happening a little bit below the surface. Like to me th there's there's a paper recently that showed that the the period index conjecture is false. and it's very different from the three things that you just listed in that it was more of a a centaur kind of achievement. It was somebody working with an LLM and to help produce examples LM was making doing things wrong but having ideas and there was some going back and forth. Yeah, yeah. Marty Weiner: So is it human plus AI back and forth? Yeah. Jonathan: I mean the like all of these conjectures like they're notable, they're they're important. People couldn't solve them. or people had not succeeded in solving them and AI was succeeding, right? And that that has to be noteworthy. But but it also makes us reflect a little bit on like what we really care about with math. I mean, because like you're seeing this news, like Airdish condu you know, it's not even in the news anymore when an Airdish conjecture is proved or or disproved by AI. Yeah, but Marty Weiner: look, another one disproved. Yeah. Jonathan: but But a lot of the time the AI is succeeding by some sort of grinding. it's not necessarily producing new ideas or or new understanding. you know, what what would what when a new problem is solved by a mathematician, usually it wasn't just by grinding away with established techniques, usually they had to have some idea, some new insight, which Marty Weiner: Some inside. Jonathan: which which by studying you will just get a better sense of how things work. And it's not totally clear that we're getting that from AI solutions. to I mean arguably there there are traces of that. but but it's you know it's it's not as inspiring in the same way. so I mean like if you take the the Riemann hypothesis thing okay, so I don't know, I I I'm not an expert in this, but my understanding of what happened is is it took some technique that was that worked conditionally you know under an assumption of the Riemann hypothesis for doing some estimates about the zeros, and it you know assuming some quadratic form was positive definite or something, and it it just worked hard to remove the conditional. hypothesis on the proof, right? So the proof worked if some some ult exterior information was true and it said, well we can just remove that hypothesis and maybe the proof still works. And so it was really like everything seems to be in this in the this kind of grinding away with established techniques and Marty Weiner: Mm hmm. Yeah, so s to expand on that, so you're saying that a lot of the techniques are more about the L L kind of tireless tirelessly working through all possi or enumerating all possible boundaries to a problem or something and then just saying, Hey, let me go check every single one of them. Jonathan: Yeah, I mean like it it's something that that, you know, a reasonably intelligent mathematician with infinite patience could do, but it doesn't necessarily require a new a new insight. Marty Weiner: I remember one of the lean programs somebody wrote was about a hundred and thirty thousand lines long and I just that sounded long for a for a lean proof, but I I wanted to Jonathan: Yeah, yeah, but that doesn't that doesn't necessarily well so so what are we talking about? Like for one of these verifications? Yeah. I yeah. Yeah, well, I mean Marty Weiner: Yeah, I think his counterexample was a hundred and thirty thousand lines and that sort of s just sound like grinding a thous you know, millions of Jonathan: it it could be or or or or not, right? Like a hundred and thirty thousand lines could be could be good lines or bad lines, right? but Zach: Yeah. Marty Weiner: Ha. Jonathan: yeah, I heard somebody said that the I don't know if this is true, but the that the the the verification of the Airdish conjecture, the the unidistance counterexample was a million lines of of lean code, supposedly. but Marty Weiner: huh. huh. Jonathan: all all of math lib, which is the the like foundations of math that we we in theory are all building on, is two million lines of lean code. So I mean if if you have experience vibe coding you know that the the the output of the LLMs is not always the most efficient or elegant solutions Marty Weiner: Mm-hmm. Jonathan: to the problems. And in math, like that's all we're looking for is is like elegance. that's that's what we really want. And that, you know, if it just grinds away, I don't know, it it's it maybe we lose something. Zach: Ca can I ask a question about that? Like, so we we've talked about how, you know, w when Alpha Zero, started winning every time at chess, part w what was going on is that in a chessboard, we know how to say at any given point, like, what are my odds of winning? And I don't think we have the equivalent in math. Do you think it's like even like like to so to that would be like can we rigorize what we call mathematical intuition? And d do you think that's even in principle possible, or is it just something we haven't done yet? Jonathan: I I mean, I don't think you should rule anything out. I mean, if you're if you if you talk to a human who didn't know how to add a couple of years ago and now it's producing research level math, I think you would expect this human Zach: Ha ha ha. Jonathan: has a bright future. And so yeah, I I I think it's pretty likely that AI is gonna be doing everything we do. But I think it's very hard to like like AI right now seems to to really emphasize correctness, like just producing a correct proof, and it doesn't seem to have a lot of intuition for, okay, this is a good strategy, it's likely to work. you know when a human works, when I work personally, I usually write a lot of nonsense to begin with, but I try and and like get get an idea that I'm of of a of a proof strategy that I can refine until it eventually becomes correct instead of pr producing lemma after lemma that's correct and hoping that I get there. Marty Weiner: Mm. Jonathan: and yeah AI it feels a little bit like it's over indexing on on just getting something that's a hundred percent watertight rather than pursuing something that's intuitive and and that has some idea behind it. Zach: And do you do you have a a a sense of like when when you when a mathematician has taste or or in intuition like do do you have a guess of what that really is, like what kind of operation they're running? Jonathan: So so I have a completely ill-formed conjecture, which is that you know, on the subject, right, of doing things intuitively and and not precisely. Zach: Yeah. Not in general. Christopher: Ha ha. Jonathan: it which is that the like the AI has a working context of a million tokens, and and I have a a working memory of like seven items. So so A human explanation is just forced to be more efficient and more conceptual. I just cannot hold as many things in my head as the AI can. Whereas the AI it you know, it may feel conceptual to the AI when you have when you can think about a million things at the same time, but it's just not conceptual at a human level. Christopher: Wh which brings it very similar to being the the kind of the the alpha go solution or alpha chess solution, right? Which is that it can just outthink and out grind and kinda get deeper into the problem by its very nature. That doesn't mean it happens to have a ton of insight about strategies that work really well in chess. So you're saying it's kind of the it's kind of the same thing here where it's able to do basically this kind of very kind of plod through the linear argument kind of step by step, in a way that a human could would kind of leave at lose their attention or lose their context because Seven slots opposed to a million tokens. Jonathan: Yeah, right. There there's tons of these things that no human would have the patience for and no human would have the the attention for. Christopher: Or the grad students. Marty Weiner: Yeah. Zach: Yeah. Jonathan: you know, grad students don't work that way in math. Zach: Ha ha ha. Christopher: I'm sorry, undergrads. Jonathan: No, undergrads don't either. No, one does one Marty Weiner: Mm-hmm. Jonathan: doesn't get assistance in math. It just yeah, that's not how it works. Zach: It's a myth. Jonathan: No, I yeah. Christopher: I w would you say that I mean, the other thing we've talked about actually in the past has been, you know, we we've wondered if this is a case of the L LMs have access to effectively the entire corpus of human knowledge, like just kind of at their at their equivalent of their fingertips. Have you seen any examples where there's been kind of like cross field synthesis that's been interesting? You know, it's like it's basically taking something off the shelf from another field that wouldn't otherwise have been applied and then use that machinery to grind through a problem. Or they still kind of fo you know, are Jonathan: Alright. Christopher: they acting like algebraic geometers, not just like general mathematicians? Jonathan: so I think we're definitely seeing examples of this. I I haven't seen that kind of thing in places where I have a lot of expertise. but you know people point to to the unit distance conjecture as as an example of that kind of bringing in knowledge from a lot of different places. but I mean let let me let me give you another kind of a counterpoint to that idea. So I was trying to to work on a problem with Fable a few months ago when when Fable just came out. And you know I was it it this problem should have some relatively simple conceptual solution if if it's correct. And Fable just wanted to compute. It just wanted to compute example after example. And I had to like push it away to to to like think try and get it to think conceptually. But it just wanted to to grind and grind and grind. And it started bringing in all kinds of things that I had never heard of. a, you know, using this I can't even remember the names of things, but it but it had so much knowledge and it was bringing in so many different things to try and inform its c calculations and different things to try. But it just would not think about the thing, about the the underlying idea of the problem and try and sort it out. so yeah, it brings in all that knowledge, but sometimes our advantage as humans is not having all that knowledge and getting to the core Marty Weiner: It's overburdened with with too much. Jonathan: of the problem. yeah. so yeah, it's a blessing and a curse. Marty Weiner: So w w w what are you seeing AI is good at? What is it bad at? I think you mentioned to me something about AI being really good at reading papers. Jonathan: Yeah, you mean like what is it good to to use it for or what is it good Marty Weiner: Y y yeah Jonathan: or or if you just set it loose on math, like what will it succeed at? Marty Weiner: what are what are you seeing good results from using AI Jonathan: yeah, so if you if you can Marty Weiner: And I don't mean just, you know, paper or wha how how have how have you worked it into your work your your day to day, I guess? Jonathan: I'm still struggling to. so the but i that's a complicated question, like how it comes into Marty Weiner: Yeah. Jonathan: my day-to-day. the places where I mean what really convinced me to start think taking AI seriously was I I don't know, six months ago or something. I was at a conference and a a colleague was doing some experiments to see how good AI was at math and he was soliciting problems from us, you know, things we had proved but weren't weren't solutions weren't Marty Weiner: Right. Jonathan: out there. And he just you wanted to see how well it did. So I you know I'd used the the web interface of ChatGBT and and maybe Claude, but but I hadn't really given it the chance to grow to to work for an extended period of time. Right? The web interface will never work for more than a few minutes. And so I put a problem in there that I had solved. you know it took me maybe half a day to write the proof or something. And Chat GPT Pro, you know, whatever the pro version was at that time, worked away for 30 minutes and it produced a correct solution. And and I think my solution was nicer, but nevertheless, it was it was correct. And Zach: But it took thirty one minutes. Christopher: Yeah. Jonathan: it it was correct, and that made me realize it was it was really Marty Weiner: Something there. Jonathan: it was really pretty good. But but and those are the kinds of things that I see the most success at is when I have already solved a problem and I put that same problem in, it can do it. Like even if the s solution's not on the internet. But if I can give it the precise problem that I was able to solve, then it can also produce a correct solution. But the thing is for the mathematician, the major you know, the majority of your work is not necessarily that half day I spent writing that proof. It was all the work I did before to figure out what was the right statement that I wanted to prove and how that fit into the project that I was working on. And a lot of the time I'm you know, most of the time I spend being confused is trying to like figure out the right way of organizing ideas so that I can find statements that I can prove that will that will fit together into the right structure. And when I try and engage with an LLM for that part of the process. for like the organization of the ideas. It just magnifies my confusion. And Marty Weiner: Uh-huh. Jonathan: it's like all the confusion that's in my brain is also on my screen. And yeah, so I've I've I've really not found it useful for that kind of thing. Marty Weiner: I'm curious also about you mentioned that your paper was was or your proof was better. I'm wondering if in one of the ways it was better, was just in the ability to communicate with other humans, the ideas or the the intuitions. Jonathan: I I j Marty Weiner: Or was it about even? Jonathan: I just feel like there there was a little bit more you know, it's like it's like good writing, you know, you it has some balance to it, you feel like you feel like you you see some rise and fall to the structure. of the the org Christopher: Lester. Jonathan: the organization of the presentation. so my writing to me felt felt just felt better and cleaner and the ideas appeared you know they were used and then they disappeared. the AI solution maybe pulled a lot of things from from different contexts and it it it felt a little bit like it was like it was grinding through it a little bit. rather than than than flowing with the the the concept if that makes any sense. you know when you read a nice proof it you just feel like that just makes sense. you don't feel like you have to struggle through it. Marty Weiner: Mm-hmm. Jonathan: yeah so but I I wanted to add to you to the things that it's good at Since it's good at like like if you have precise statements, it's very good at like assessing is the proof I wrote correct? Did I make some minor assumption that that that was not actually part of the assumptions part of the hypotheses? Or y y so proofreading is is can be very good. It kind it kind of like overdoes it. You know, sometimes there are little gaps the mathematician could just fill in and it's like, no, this is this is you know, it hasn't Marty Weiner: He You missed out. Scrap it all, throw it away. Jonathan: Yeah. Zach: Yeah. Jonathan: Yeah, this that's right. But but in the proofreading process I've I've found it very useful. and and then there are other Marty Weiner: Okay. Jonathan: things like like visualizations that are not like in the production of math that I use AI for all the time. But Marty Weiner: Uhhuh. Does it help I I'm curious also about just more the mechanics also, like writing LaTeX equations or or hunting down papers, you know, finding good papers to reference Jonathan: yeah, yeah. So if I'm trying to learn some subject, right? I mean that was where the first thing I ever did with AI was effectively was just trying to learn stuff. So this is like back in I don't know, January, February. by then AI maybe even a little before that, AI had been really good at ingesting all the literature. And if there's some area it's not something I'm necessarily a specialist in anymore, but it or or it's not something that I'm necessarily a specialist in. but it's is something that there's a lot a lot of sp of specialists in in the world. the the knowledge is out there and AI can like very quickly tell me exactly what I need in order to to learn what I need about that subject instead of having to dig in dig through the literature. I Marty Weiner: huh. huh. Jonathan: I mean I asked some colleagues the other day like how much do you actually read papers anymore? Marty Weiner: Yeah. Jonathan: And 'cause for me personally, like d I mean it's sad, but but like digging through someone else's ideas to to find exactly what I need versus getting the AI to present it in a way that that is suitable for me. you know, I almost always go to the second for for Marty Weiner: Yeah. Zach: That's it. So I know we we were just talking last time about a paper. Terry Tau just had a paper out and he put a lecture out. I don't know if you've seen it, but he he he had this rule of thumb which was essentially if you the human can't w write it in a way that is interpretable to another human, like if you don't understand your own proof, you shouldn't publish it. and I wonder if you think that's Jonathan: Yeah. Zach: a good rule of thumb and and more importantly, perhaps is it a good rule of thumb going forward? Jonathan: Well it's a very humanistic perspective on what math is. I mean the community is starting to debate, you know, what what do we really care about? we had we had a lot of things that we cared about and writing math papers was a good proxy for all of those different things. And now those things are starting to separate. They're different different ways you might try to evaluate those things. So one point of view is that math is about human understanding. We want it it's about being able to explain things to each other and it's a very social endeavor and we we just want to to to share our understanding in our community. And So AI can have a role there for like helping the humans to understand things, but ultimately it's the human understanding that counts and that's what should go in papers. I mean that makes a lot of sense, but it's one point of view. I w another one is that what we're trying to do is advance knowledge by any means necessary. And, you know, if you can lean verify something, then maybe it doesn't matter if you or any human understands it. Zach: That's do you think it's like it's possible there's a whole like universe of proofs that we're just not going to understand and we just have to trust the Jonathan: No, okay. I mean there has to be a universe of proofs that we're never going to understand, Zach: Well, yeah, yeah, right. Jonathan: right? But whether those are ones that the AI Zach: Like interesting. Jonathan: AI does understand, yeah, I mean sure. but I think we're all used to that already. I mean, you have to make peace with the idea that there are there are millions of proofs and concepts in math that other people understand, that you will never access. And now there also Marty Weiner: But Jonathan: be tons that the computers understand that That we'll never access. Marty Weiner: Do you think we'll be able to make peace with the nobody on earth can understand Zach: Yeah. Marty Weiner: them? Jonathan: Yeah, sure. Marty Weiner: Yeah, yeah. Sure, yeah. Zach: I do do you think there's there's a distinction between like pure and applied fields here where like you know, in an applied field you i it it's very obvious that you could take the approach of look if it works, I don't care. whereas, you know, if if if if if it's like, you know, pure math as I understand it, sometimes something is interesting just because it's interesting, it's fashionable, and that's why it matters. And maybe in that case you really do care more about human interpretability. Jonathan: I I don't necessarily want to put the divide at at pure versus applied, because that divide exists within pure math, I'm sure it exists within applied math. but I and and I think like it's healthy for that divide to exist. Like you need both approaches. I mean I'm sure we will see the the humanistic side of math continue w and we'll see the like pushing knowledge forward by any means necessary side of math continue and that will probably involve AI up and down through through the the process. but those can kind of exist in parallel and they probably will have separate journals. You know, there'll be the humanistic journals and there'll be things that are, you know, the somebody called it the journal of true facts. Christopher: Yeah. Jonathan: you know, it's just like Okay, so this is true. I mean it's it's useful. You know, there are things like I mentioned the period index problem a while ago. Like people have spent substantial amounts of their career working on that problem. And it turns out it's just not not true. Marty Weiner: Mm. Jonathan: and it's it can you know, just having the guidance like of of where you should direct your attention because knowing what What the landscape looks like, even if you don't know all of the details, could be helpful. I'll say that there's an essay, and I'm sorry, I forgot who wrote it, I read it just yesterday. where someone else points out that you know struggling on something, even if you don't prove it, even if it's wrong, struggling through it like is how you get a sense of the landscape as a mathematician. And maybe Marty Weiner: Mm-hmm. Jonathan: maybe this process that I just described of the AI laying out the landscape for you a little bit is also leads to some loss in your ability to to understand what's out there. it's really it's really not clear what what it means for us. Marty Weiner: Well, I I wanna ask some of the the fun, impossible to answer questions like w what do you think yeah. Well what do Jonathan: Great. Yeah. Zach: Yeah. Marty Weiner: you think things will look like over the next two years? And we had a question about when you think something like a millennium challenge might fall, if ever. Jonathan: Well, okay, so for the millennium okay. Okay, two years is too hard. let me just st start with the millennium problem one. Marty Weiner: To okay, okay. T five f five Christopher: Two two months. We'll start with two months. Five months. Marty Weiner: five to five months, five five five days, Jonathan: Yeah. Marty Weiner: hours, yeah. Christopher: Post post fable five point one. Marty Weiner: Right. Jonathan: okay, there there's a big change happening. Like I think people are adopting things much faster. All of a sudden, like there's a ton of interest Marty Weiner: huh. Jonathan: in the math department in AI. We're running this class, as I said. I it's the most attended clo grad class we've ever had. you know, a typ Yeah, Marty Weiner: Yeah, by you were saying like by five X Jonathan: it's a typical grad class, you know, we'll have Minimum enrollment is like four or something. And what you know, a typical might be might be like five to eight. And on the mailing list for this class, we have fifty. You know, there are people the room is the room is full, people are sitting on the tables and all you know, faculty are there, grad students are there. and Marty Weiner: A lot of excitement in it. Jonathan: yeah, so I mean people know like a lot's going on. We people just want to understand what's going on and and people are I think starting to adopt it more quickly. I think I mean I'm sure we're going to continue to see these these news making things where AI proved this or you know disproved that. d to me the more interesting ones are the ones the where there's a lot of interaction with the human and And s you know, some new ideas are appearing. and well you know, we're gonna probably just like in chess, we're probably going to have a period where there's there's a lot of that. I I think it's probably going to be a very productive time for math. I don't know how long that'll last before the AI completely surpasses us. and whether human humanity will still exist at that point, right? But Zach: Yeah. Marty Weiner: Doesn't matter at that point, yeah. yeah, cause yeah, just to Jonathan: Yeah. Marty Weiner: index on that a little bit, one thing you said that keeps sticking in my mind is it does feel like I I don't feel like I I'm not an expert at all, but I don't feel like I've seen a lot of new creative solutions from the AI. And I I've seen I I think we've seen, like we were talking about before, the the grinding part. But Jonathan: Yeah. Marty Weiner: have we have we seen any clever shit, that's And nobody's ever thought about, you know, taking that little tweak, that little approach differently. Jonathan: It's so hard to say. I don't I Marty Weiner: Yeah. Jonathan: I mean it's hard to say with with humans too. Like you feel like you made even even personally, I feel sometimes like I made some leap some of of insight and I made some progress on something. And then gradually I become famili more familiar with other other people's work, or I just understand what they were doing better. And I realized the idea they had those ideas too, and they had gotten into my mind without my realizing it and I just thought I had a new idea. And like Marty Weiner: Yeah, yeah. I'm sure that yeah. Jonathan: right so so with with AI2, like it it's really hard to to draw a clear line at like where a new idea is and where just using old ideas is. But I mean, I've had this experience where where I give it something, you know, some proof that I've written and it's like, no, no, no, no, no, that's sorry, your argument is wrong, you should do it this way. And you know, sometimes my argument is more or less okay and it just needed some some tweaks. But sometimes when it says you should do it this way, you know, there's something there that I didn't know. And maybe it's a standard technique, maybe it's not, but I had one Marty Weiner: Mm-hmm. Jonathan: the other day and I, you know, it was an awesome technique. I think it was pretty standard, but I just wasn't aware of it. And you know, now I've used it for other things. so Marty Weiner: Right, right. And so I guess I'm wondering about like the those bag of techniques. do we know if AI has come up with a new I mean that's kind of a that's kind of a difficult question to answer, but Jonathan: Yeah. Marty Weiner: like it sounds like all the techniques we're talking about came from humans and the AI is able to you know, like we're saying, can keep a million of them in its Zach: Mm. Marty Weiner: head and we can only keep seven. but I'm wondering Christopher: It has the full it has Jonathan: I yeah, Christopher: the full grab Jonathan: it's Christopher: bag to pick from out of it too. It's it's not just like it's it's Marty Weiner: Right. Christopher: not restrict a field, it's look this this is actually an applicable tool that we can Marty Weiner: But has it has it added to it? Jonathan: I yeah, I don't I don't know I don't know of an example where AI has come up with the like a really cool new definition. Like Marty Weiner: Okay, uhhuh, yeah. Jonathan: like Okay, so I th this is a little bit in the weeds maybe, but you know there's this new thing called condensed mathematics, is this idea of Peter Schulze and Dustin Klausen. and I I mean I'm no expert in this at all, but but it begins with this idea of a topological space, which is something we all learn about in in undergraduate math. And and the topol the definition of the topology is this concept of open sets, which is some sort of abstract notion of distance or measure. It's very hard to explain intuitively what an open set is, but you have this abstract definition. And I think every undergraduate When they encounter this definition, they say, but I already know about sequences and convergence. Why can't I just like formulate this whole idea of topology in terms of convergence? And like every undergraduate has this idea, and everybody Marty Weiner: Yeah. Jonathan: fails to come up with something. And and then and then Schultz Schultz, so Schultz is a Fields Medalist, Klaus is a brilliant mathematician, and but they thought about this and they figured out how to do it. with I mean it was it was with through deep understanding of of like profinite sets and and how to think about that but but it just allowed them to to say, you know, a space is is is described in terms of what stuff what sequences sequences in scare quotes converge in that space. And it's amazing. It's just so beautiful. And And it's like totally just rethinking the things that we've all been thinking about for Marty Weiner: Yeah. Jonathan: for so long. And a like I've never seen anything close to to that kind of elegance from anything that AI has produced. So Marty Weiner: Right, right. Jonathan: that's maybe a way off, but I I wouldn't roll it out. Zach: There's a there's a question we wanted to get into because I know we don't have a lot of time left. which is so as as I'm sure you've noticed, like some people Encounter AI that's very powerful and they get excited about what's coming. And then some people have genuine existential breakdowns. like they're Jonathan: Yeah. Zach: extremely upset. There are a lot of these cree de cour posts we've been seeing on like Substack and elsewhere. And part of what's particularly interesting to me about mathematics in this area is mathematics might be like the first people in sort of aspirational fun jobs experiencing being superseded, except for maybe like, you know, people who played chess and or go or whatever. And and so I I mm one of the things we've talked about is maybe part of that is just it's like identity obliteration. Like you think of yourself as the person who does this kind of cool work and other people in the community know you as that. And then if some machine can do it, your sense of yourself and your place in your community is is messed up. So I guess I I just wonder if if you could talk about how you've reacted and how other people around you seem to be reacting, and i is the way I'm talking about even even sensible. Jonathan: I no no it's Christopher: Yeah, like so far we haven't actually heard you say, you know, I can't wait to start my second career doing laser cuttering and woodworking as a plumber. Marty Weiner: Yeah. Zach: As a plumber, yeah. Jonathan: I I have tenure, right? So Christopher: Yeah. Jonathan: actually there there's there's a lot of truth to that, right, right. the the there's a lot of excitement among older generations and a lot of f fear and and and existential dread and just like the whole range of emotions and the younger generation because you know they have no idea what things are going to look like. but yeah if I could take exception to to only one thing that she said, Zach, it was it was the division of people into the two groups. I mean I think there's there's an enormous overlap. Like I've experienced I experienced the dread all the time And I the excitement all the time. And yeah, I mean and and definitely I think you're right, this feeling of I I think you put it very well, the the f feeling of of your identity and your relationship to the community. You know, people put some stock in mathematicians as being smart people and and and you know, it's like if if you're a if you if you tell s if you're on an airplane and you tell someone you're a mathematician, t they tend to be kind of impressed and they say like, I was never any good at math and i if you tell someone you're a woodworker, they're like, that's a cool hobby. but they're Zach: Mm-hmm. Jonathan: still gonna buy their furniture at IKEA, right? if it if math is is just a hobby, then you know, maybe maybe we lose something of of our cultural cachet and I don't think that's where it's going, at least not immediately, but but but there is some part of me that's like you know. afraid of losing that and and afraid of losing the story of what it is we do as mathematicians. Like we're supposedly working on these these hard problems and only we can like bring the the beacon of of truth or knowledge forward Zach: Yes. Jonathan: and and suddenly the machines are racing ahead of us. yeah. Zach: It's it's it's interesting 'cause we we have a parallel thing in the arts, but the artists in in my experience are mostly angry. Not I mean, I'm sure that the identity threat is part of that, but but I I think part of why it's different is you can always say, No, the AI art is bad. Once I know it's AI art, I can insist it's bad. Whereas in in math, on some level you you can get an authenticated feeling and you can't you can't argue. And Jonathan: Yeah. Marty Weiner: Well th there's a funny flip too with with math. If someone uses your work and cites it, you're you're you're honored. Jonathan: Well and and cites it, right? That's the Yeah, I Marty Weiner: It cites it. Incites it. Yeah. Yeah. Yeah. Zach: Yeah. Jonathan: I I always seen the same the same thing in math with you know AI using people's ideas without attribution. Marty Weiner: that's interesting. huh. Jonathan: and I mean maybe one difference is that you can see it more explicitly. Like I don't know, with art, like sometimes if somebody draws Saturday morning breakfast cereal comics, like you think you can clearly tell that Zach: Yes, yes. Jonathan: that that Marty Weiner: Mm-hmm. Jonathan: was Zach's style. Zach: Yeah. Jonathan: but in math, you know, there's this first first proof project where they try and assess how good AI is at certain problems. And there was one problem, I think it was by Rick Schwartz, and he gave a problem that was sort of similar to to some other problem that he had solved earlier. And the AI solved it, but using his techniques. using his notation with no citation. It was very clearly came the ideas came from from his paper, but there was no no attribution. so yeah I think people are when this happens people are rightly upset about it. But I think it's also a r relatively easy fix, maybe. You know, they they're just putting Marty Weiner: Yeah, Jonathan: a it it it Because the literature is all out there, you and citation is all you need. Like if I if I copied Zach's drawings and then I said copied from Zach Wiener or Zach Wiener, sorry, Wiener Slin, Zach: That's right. Jonathan: then you know I don't think Zach is any happier about it, right? But but in in Zach: No. Yeah. Marty Weiner: Ha ha. Jonathan: In math, if you if you say like this came from from my paper, okay. what more can can you give me? Yeah. Marty Weiner: Yeah, it's awesome. Zach: Yeah, but but but but I you it's funny though, I do I do think part of that has to do with the thing you said earlier, which is like financial precarity, which is like my being able to make a living is predicated on my being the guy who does the thing and if somebody else But Jonathan: That's right. You d you don't have tenure, right? Zach: no, I but I I mean Marty Weiner: Yeah. Zach: I d I do think like like like just anecdotally, I feel like I I might be wrong about this, but pretty much every post I've seen by a mathematician having existential dread about you know, what they are or anything is by someone who's pre tenure. I don't think I've seen a tenured professor say the future is bleak. And so I d I I do think we're underestimating Marty Weiner: Well Zach: precarity as a kind of as as what's actually underlying a lot of the existential 'cause and and like like like yeah. really? D and and that's just identity stuff Jonathan: No, no, no, no, no, no. Like there are definitely tenured professors that say the future is fake. yeah. yeah. Zach: or? Jonathan: I mean, honestly, I mean if you asked me to to to to come down on one side or the other. long term, yeah, the the future might be kind of bleak. I w I wouldn't I would not rule that out. but Daniel Litt wrote this essay. I he's a great mathematician and definitely tenured. Marty Weiner: Mm-hmm. Jonathan: you know it was a it was a th thought experiment. I don't think he he entirely believes that what he suggested will happen but but but he argues that maybe maybe we're seeing the end of mathematics in and you know as as a ai supersedes you know takes away the story of mathematics and takes away the interest in explaining things to another to to one another you know maybe we just lose the motivation for doing it. And you see some of this stuff happening already, like math overflow. Do you know math overflow? It's the the Zach: Mm-hmm. Jonathan: math version of Stack Overflow. He Marty Weiner: Yeah. It's w it's where you yell at a people because they're idiots, right? Losing life. Christopher: Using using LaTeX mostly though. Marty Weiner: Losing life, yeah. Jonathan: it's very similar. Zach: Stack Marty Weiner: is it? okay, 'cause the the the equivalent Zach: overflow, yeah. Marty Weiner: s stack overflow was yeah. Christopher: It's not civil. Jonathan: I mean th there are moments, but but it tends to be very simple. Marty Weiner: Okay. Yeah. Jonathan: yeah, mat math has always been sorry, I'm digressing, but but he points out that the number of questions there has been declining and also the number of answers. Like people are just getting less Marty Weiner: Yeah. Jonathan: interested in communicating with each other. And you know, maybe that that has something to do with the future. But but Christopher: I for w for the Marty Weiner: So I Christopher: record it's it's the same for stack overflow. It's down I think it's down ninety nine percent from peak. It's it people just aren't using it. It's actually pretty drastic. but it's a it's the Jonathan: Yeah. wow, wow, wow. That's that's that's way more than enough. Marty Weiner: Yeah, even more. Zach: Yeah. Christopher: same it's the same kind of reason though. It's it's it's that instead of actually needing to reach out and find another human who knows the answer, there's an agent sitting not too far from you that can just answer your question directly and in a way that you don't Marty Weiner: W without snark. Christopher: without snark and what or you you can turn this you can turn the snark level on and up on and off, but you know, it's like yeah. Answer the question as if Marty Weiner: You can, you can, but you don't have to. You you yeah. Zach: That's yeah, you Jonathan: Yeah. But also the interest in Christopher: it were coming from Stack Overflow. Marty Weiner: I want to try that. Christopher: no, sorry. Zach: Ha ha ha. Jonathan: But the the interest in posting answers has also declined, right? I mean, and even though Marty Weiner: Yeah. Jonathan: we can get the answers from AI, so it's just like the the whole communication pipeline is is kind of Zach: Ecosystem. Yeah. Jonathan: yeah. Marty Weiner: Well, I w I wanna we're kind of a time, but I wanted to end it more on a high note 'cause one one thing you told me I Zach: Yeah. Marty Weiner: thought was was interesting. R yeah. Let Jonathan: Ha ha good good luck. Yeah. Marty Weiner: me let me try this. we'll see if this works. you were telling me one of the cool things that's happening right now, you know, we're talking there's a little there's a a fear of of the long term, but in the medium and short term, there's a lot of excitement that that you don't see too often in fields, right? and you were telling Jonathan: Well I Marty Weiner: me that the mathematicians are all talking a lot more, at least within your department, than they ever that they did before. Jonathan: Yeah, I told you I told you about this class, right? Marty Weiner: Yeah, yeah. Jonathan: and it's really it's been really fun, being an amateur again. and you know, lots of us are trying to learn about AI, to learn how to use it, to learn whether we should use it, to learn what it all means for us. And we're all like grad students again. We none of us really knows Marty Weiner: Yeah. Jonathan: what we're doing. The The gra you know, everybody started in twenty twenty two. so it's not like in algebraic geometry, I've got a fifteen year head start on my graduate students. You know, we all started at the same point. Sometimes the graduate students are the experts and we learn from them. Sometimes we know stuff and they learn from us. and so we're all Marty Weiner: And tw twenty two is charitable. I I think we're back then it was more like you know, y you'd ask it a question it would hallucinate or everything. So really it's more like you guys all started in twenty twenty five almost. Jonathan: yeah, yeah, sure. Sure. Zach: Yeah. Marty Weiner: No, I I I but I I I can almost feel it like it's fun when you get everybody together and everybody's i in the same boat together rather than off in their own little spaces. Jonathan: Yeah, so so yeah, for like for this AI class, you know, none of us knows anything. so th three of us are trying to teach it and then we're all working together to try and to Marty Weiner: Learn it learn together. Jonathan: learn it and then and then teach it and it's super fun. Marty Weiner: Yeah. I mean it sounds like a really fun time and and I'm I'm hoping physics kinda gets there too. Like it just really unlocks a whole new area of to play. Jonathan: Yeah. But I mean there there is some sense that it's it's limited. You know, go going back to the negative Marty Weiner: Uh-huh. Jonathan: thing, like like it's it's like really it's we're like Zach: Ha ha Marty Weiner: Damn, I almost had it. Jonathan: we're really we're really like learning AI, we're all getting used to it. and and that does feel like maybe it's a one time one time thing. I don't know, we'll see. But in the me y you know, it's it's really hard to predict the future. But my job is basically to to understand things and I have some interesting thing here that I can try and understand. And you know, I'm just gonna enjoy it while while I can. And the you know, who knows what tomorrow brings. Zach: you know, just to close out on a book recommendation. I don't know if anyone's ever read Aikenfield by Ronald Blythe. No, Jonathan: Is this is this as reclined now? Yeah. Zach: no, no, Jonathan: Ha ha ha. Zach: Fri for I'm not even gonna yeah. no, I was gonna say there's a book, there's a guy named Ronald Blythe, he passed away recently, and in the late sixties he went around Britain interviewing like people. and they were undergoing the Green Revolution where all the old farming ways were just finally getting removed and like all the hedgerows were coming out, all the old techniques were going away. And he had this guy he interviewed, this young man, and he gave this whole thing about how the old man they're always fussing over little details, they're just making sure to get everything perfect, it matters so much to them. It's all pointless. And and and we don't do it anymore, because why would you? And the the thing he said when he closed out, which Blythe might have artistified a little bit, we don't know, but it was something like you know, the old men had art because they had damol. young men have efficiency. and I think about that a lot now. Like that sentiment is going to come for every community. and it's it's it's it's Jonathan: What w what a blithe sentiment. Marty Weiner: Well we well wait, let's let let's end Zach: beautiful. Thank you. Perfect. We did it. We ended Christopher: Nicely done. Nicely done. Marty Weiner: on a on a positive pun. Good. Christopher: Yeah. Zach: Yes, good. Marty Weiner: Well thank thank you, John. We should ha do this again in six months and and talk more about your your tenure track. Jonathan: thank you for having me. It's been a pleasure. Zach: Yes.