Latest / The Information Bottleneck / Can AI Replace Doctors? Zachary Lipton on the Future of Healthcare and AI

Transcript

Can AI Replace Doctors? Zachary Lipton on the Future of Healthcare and AI · The Information Bottleneck

• Ravid: Hi everyone and welcome back to the Information Button podcast. And our guest today is Zach Lipton. He's a associate professor at the CMU and the co-founder of Average. It's a healthcare AI company that wants to be the intelligence co-pilot for doctors. Hi, Zach. Thank you for joining us. Zachary Lipton: Yeah, thanks for having me. Great to great to be here. Ravid: And as always, ALN. Allen Roush: I'm happy to be here and happy to meet you, Zach. Ravid: So today I think that we can start with with healthcare or like what you think are the problems and the solutions in this in this field. So tell us like why d do you think like it's more difficult or like what are the problems that we actually want to solve in this area? Zachary Lipton: I mean, we could probably spend the entire podcast like ruminating at that level. try to compress. Ravid: We we have time, that's fine. Zachary Lipton: We got we got we got time. Let's pull up a chair. Healthcare is an interesting area because it's it it's defined by a set of aims, but like within that set of aims, you have you sort of every kind of machine learning problem arise, right? Like you have speech problems, you have vision problems, you have decision-making problems, you have cost optimization problems, you have business problems. I I'd say that generally, like what in particular what in particular makes healthcare so difficult, I think there's maybe a few aspects of it that are that are particularly Challenging. one is just the sensitivity of the domain. bad decisions have big ramifications or clerical errors can result in misinformation being propagated. Patient could result in a treatment mistake, like especially in a very litigious country like the United States. like the stakes are really high and it's very you don't you have you know, you can't casually experiment in the same way as you might in another domain. I I'd say another thing that makes healthcare so difficult, especially in the United States, is that the the system is sort of fractured in this very complicated way. The economic landscape of Medicine is not straightforward. I think a lot of us that are like technical people who want to build something and then maybe find a way to commercialize and bring it to folks, you know, I think we tend to thrive in a setting that is like I know who my customer is, and the customer is the user, and and and and ideally they often are like someone that I know something about. And you kind of say, here, like I built this thing, I think you will like it, here it is, you can use it. And in healthcare, it's like actually, maybe you're delivering a product to the doctor. in order for the doctor to use it, a purchasing decision has to be made by the CMIO and approved by the CFO, but actually You know, the sis and then the system has to be like implemented by IT, but actually some of the economic value of them using the system might actually accrue to the insurance company that is a separate entity, which then reimburses, you know. So there is just a tremendous amount of like sort of like organizational and business complexity to working in healthcare that makes it very difficult. I've seen so many talented researchers come up with like interesting technical ideas. But I like to say that like healthcare, like we all look at it as like this incredible opportunity. We want it we feel good about the impact. We know it's like twenty-something percent of GDP. We know there's we know there's there's good stories, there they're saving lives, there's there's economic opportunity, but then like those are the sirens, but there's a lot of, you know, sorry to be topical with the the Odyssey references, but there's a lot of there's a lot of ships like moored on the rocky shores. Of of healthcare because there's just so many ways to die, even before you even get a single person actually using the product. And then I just add that like also there's complexity because so much of what we actually care about isn't just things like recognition. Like I think I think the holy grail for healthcare ends up being in places like, you know, can we can we make better decisions, save people's lives? But these are these are sort of causal questions, not just like recognition questions. And actually You wind up in this area where it's very, you know, hard to demonstrate that kind of value definitively in the absence of clinical trials. And so you have a, you know, a very a very hard path towards demonstrating clinical benefit and one that is tightly regulated and involves randomized trials. And and that's a force that drives a lot of the innovation, interestingly, away from direct clinical impact and more towards like the operational side of healthcare. You know, it it's a little bit safer. And in fact, that that was the path we were on at a bridge. It was we were able to get sort of modern AI technology inside the flow of healthcare by delivering operational value and then start figuring out how do we tiptoe our way towards clinical value because it's sort of a safer place to play. So there's there's so much to go into there, but I hope that paints a little Ravid: What well? Zachary Lipton: bit of a picture of like the the messy the messy Ravid: What is the goal for you? What is the goal for you? Like what do you actually want to achieve? Zachary Lipton: look, I I think me personally I want I want people to have better care. I wanted to have it more readily, I want people to have better outcomes. I actually, my my personal journey is I actually went into PhD in machine learning because I wanted to make the healthcare system better for patients. so so that like I I can I can dig up someday, it's a little bit embarrassing to read it, but I can dig up my Ravid: W when did you do your PhD? Zachary Lipton: like statement of purpose from when I applied to PhD and and that's truly what it's about. Ravid: When was it? Zachary Lipton: so so I was actually originally a musician and I graduated from undergrad 2007. I was like living in New York, I was a jazz musician at the time, like playing the saxophone, and I actually went through a really like difficult and kind of like debilitating personal like autoimmune experience. And so I'm trying to figure out like I kind of fell in the diagnostic cracks between it sort of looks like this, but it also sort of looks like this, and you have these antibodies, but not those antibodies, and it's I don't know. it seems like unfortunately like it's becoming more common these days for people to have like you know, especially in their twenties, I think like weird bouts with like chronic illness and autoimmunity. So I was I got kind of knocked out of like I wasn't I wasn't really active playing music for a little while. I had I lost a little bit of my life and I was spending all that time reading medical articles and just making basically making all of my like grad school friends like send me because I didn't have access to like, you know, PubMed or like any of the I I couldn't get any of like the the sorry, get PubMed, but none of like the paywall gated articles. and so I was just trying to make sense of, you know, you have some of these diseases where it's like, you know, you know, maybe maybe there's an off-label use of a drug that is like originally approved for one autoimmune disease is being used for another one. Probably there's a hundred thousand people across the country doing something like that. You want to understand how's it going. You look for journal articles, you find like here's a report about 10 patients at this one health system that this one doctor saw. And you know, for each of those patients, it's like, you know, there's no there's no structured data, it's very anecdotal. and so it felt like there were all these problems that in healthcare that were like both we're not measuring everything we want to be measuring, we're not able to aggregate the data across the kind of like cohorts and populations that we want to be. And then even if you had all the measurements that you wanted and could touch all the data for all the patients in the country, like looking at this from the vantage of 2012, you know, I wasn't I wasn't yet a grad student and I wasn't that specific. I like an undergraduate degree in like math and economics. But it felt obvious to me that like it was or it felt obvious that it wasn't obvious. like what even if you had that data, like what is the query that you would run? Like it wasn't like, like if only I had this data, I'd do this magical SQL query, and then now everybody's better and everybody's healthy. Like even if you even if you don't have all this crazy confoundedness and missingness and inability to access data, causal inference from observational data, like figuring out how to treat people is fundamentally hard. And so like in 2012, I I kind of came together with like a little bit of a thesis of like, I was also already 27 years old. Like I was like, I'm gonna go to pre-med and then try to do an MD PhD and become a doctor. I was like, I gonna, Ravid: Mm. Zachary Lipton: I'd still be in med school today, right now. So, like, my my kind of mindset personally was like, you know, I I look at a lot of these things, like measurement, systems, infrastructure, statistics. It it felt like there was kind of like a computational meet statistical angle to have an impact in healthcare. And I wasn't, I'm not the only person who thought. something like that. But it was a relatively niche community. Like things like the Machine Learning for Healthcare conference or Chill, which is like a sister conference. Like they didn't exist at the time. There was like a a very small ragtag group of like machine learning scientists who used to go hang out at children's hospital Los Angeles once a year. And they had a workshop called Muckmed that stood for Meaningful Use of Complex Medical Data. But at that time like that was my kind of thesis was like, hey, if I like actually I'm not gonna become a doctor. I'm not gonna become a life scientist. Like, I had a good friend who was a biology professor. He's like, by the time you learn how to be useful in a lab, you're gonna be retired. It's like it's it's too late for Ravid: Mm-hmm. Zachary Lipton: you. but but the thesis was like actually machine learning was actually gonna be this really interesting angle to to have something, something different, something new to say about healthcare. So that's that's kind of, you know, I I I I I use Ravid: Yeah and Zachary Lipton: that as like inspiration to like write to a bunch of PhD programs and say, take a chance on me, even though like I You know, I've never done this Ravid: And Zachary Lipton: before. Ravid: did you believe that like machine learning can be such a a huge deal? Because like right like in traditionally like it was like the role of like statistic like like statistica and like how to you know like to understand like now how to do this like meta review of of papers and how to handle all like I don't know missing values and things like that. But it wasn't like I don't machine learning problem most of the time. Zachary Lipton: Yeah, I mean there's a deep rabbit hole there about like where where do we think this the fault line lies between statistics and ML. Ravid: That's true. Zachary Lipton: I probably have like a weird view because you know I don't know. I I think I like at the time I think I was just like pretty naive. I I think it was like I don't know that I was so deeply aware of the different cultures. I I think I just think I knew that it was like to me, I I was like viewing this as more of like a Venn diagram of like, you know, there's like computational problems and like statistical problems and like machine learning is the intersection. I think that was probably in my cartoon. There obviously there are different some some cultural differences in like what are the What might have been like the the kind of statistical problems that the like machine learning community versus the core statistics community was concerned with. But you know, you also had this long history of like, look, go back to like early AI community, early expert systems. They're they're focused on medical diagnostic pathways and I'm trying to I'm trying to like rack my memory for like some of those like early pioneers of AI, I'm trying to remember. course I read, but and and I don't know, like you read through like a lot of like you know, like Uta Perl, and like I think a lot of AI community was, you know, sort of, you know, they might not have been as much they focused on different things, probably less on asymptotic analysis, less on measure theoretic rigor, but they were concerned with probabilistic thinking on decision making. Yeah. Ravid: But but like you you you the power like is maybe like a good example, right? Like he's brilliant and like he he changed like the the the paradigm of the of like the entire like field but like at the end today when you're you want to do like I don't know when you want to have better model you're not going to to to this type of work. Like you're going like to to another like line of of works that just like you know, kind of like High climbing on on on some data sets, right? Zachary Lipton: Yeah. So so I guess Ravid: It's much more empirical question, right? Like how to build models that give them better results. Zachary Lipton: Sure. I mean, I say a few things. One is like, I mean, my my the landscape looks very different in 2026 than it did in 2012. I think in 2012, like there weren't a lot of neural networks in production either. So it was very much, you know, if I'm speaking to you know my mindset getting into the field, you know, things were very wide open back in in twenty twelve of It wasn't obvious which methods we're going to find purchase. you know, n none of it was practical. No nobody was shipping machine learning systems inside a a clinical care context. And I mean, the other thing that I'd add is that like I think there's always a bit of a dance between between theory and practice and people that are trying to give a cohesive, like philosophical view of a universe of problems and how to think about it, versus people building practical systems and You know, I think one is often like an inspiration for the other. and then I think I'd finally say that like one thing that I think makes me a little bit still always keep I think the more the the neater, the the more the more like kind of philosophically and like theoretically ambitious sort of strain of causality in my mind is that I I think that like Treatment outcome type problems are very different from recognition problems because you can't directly observe how you're doing. And so like benchmarking on patient treatment is not the same, like benchmaxing on patient treatment is not the same thing as benchmaxing on something where you have a a perfectly simulatable environment or something like a, you know, classification task. Ravid: Okay. So you Allen Roush: We had a we had a so so we've had a previous guest, who and I'm I'm forgetting their name unfortunately, Ravid, you might be able to remember, who talked about doing almost this, like continuous clinical trials all the time, where the idea would be that that whenever an AI helped propose an int like a like a a treatment plan in some way, it would actually give you two or three very slightly different treatment plans. and they might only differ in like one particular p drug like for example, Tylenol versus ibuprofen for pain relief in this context, right? And then through this kind of a continuous optimization or continuous trial could be implemented. And this person also had some healthcare experience and was saying things like this. I'm curious what your thoughts are on a proposal of of implementing this in a healthcare context. Zachary Lipton: I think like spiritually I love it. and and I'm also I've come up face to face with a lot of the complexity of, you know, I think one problem that you have is on on one hand, if the experiment is is tiny like this, like I'm giving someone Tylenol versus Advil in a specific set of circumstances. My my guess is that the like like the the big health care the big health outcomes that you you actually care about measuring are probably like indistinguishable. with this kind of intervention at the kind of sample size that you have. So it's not like you know it's not like when you're in code it's you you wrote something and introduced a bug and the code completely failed or is it like, you know, these things are are having, I think, I I think I think very small interventions will have very small effects on patients and and and very hard to actually know what's going on. two and then I think the flip side of that is that Very big and dangerous interventions have huge effects on people. And there's a reason why the system is so buttoned up and guarded. And, you know, like we don't have like these fully adaptive, you know, over the course of a clinical trial, the space of what a model might suggest can evolve in a very organic way according to the whims an AI system. Like right now, it's like You know, there's a very bottom, very buttoned up process for like what does a clinical trial look like and what are the arms of the clinical trial and what are the samples that we're going to collect and what is the comparison that we're gonna make. and and and I I I think what you can run into obviously is like the you know, you're gonna run into on the other side the if we want to if we want to make a lot of systems that make very small decisions, it's like maybe fine, but we can't measure anything interesting. We wanna make a system to make very big decisions. And do it in a very like kind of open-ended way. I think we're gonna run into like incredible and perhaps justified, like sort of regulatory and bureaucratic obstacles. So I I I'm at once, like, look, I I I think we can't. I I would love for us, like it feels like we have so much missed opportunity to learn more, so much missed opportunity to live in a more like dynamic culture of experimentation. And at the same time, I'm like a little bit sober about like. This the system is the way it is for a reason, and it doesn't mean that it's perfect or it shouldn't change, but but it's also like you know, like like there are there are like real problems that you'll run into. And and there is also like a also like a totally sorted and terrible history of medical experimentation, you know, when folks haven't been regulated and have been free to kinda have a more cavalier approach to to experimenting on on on human subjects. Allen Roush: So so so so related to that cavalier approach, I actually I am aware of some of the history of of doing very bad things with medical experimentation, to Shingua or t tunguski syphilis experiments, something like that, I think, being an example historically. But I I also want to point out that the US being in a de facto gray market situation with GLP ones, for example, has I claim led to an enormous positive outcome. Yeah, I'm yeah, there's you know, the Instagram models trying to get clout who probably don't need it, but a huge amount of people who have problems related to obesity or even things like like addiction to gambling of all things have been able to through the poorly regulated Yeah, yeah, that too. Zachary Lipton: Addiction Alzheimer's is emerging as a possible indication, like inflammation and like neuroinflammatory disorders. Yeah, there's Allen Roush: Yeah. Zachary Lipton: a whole a whole universe there. I think it's yeah. Allen Roush: Yeah. Yeah, and I claim poor regulation allowed this to happen rapidly. And and so I actually really love that it's like poorly regulated. And I found similar things I know it's even more controversial to talk about this with small scale antibiotics, but you don't need to get like doctor's notes for antibiotics in a lot of the third world. And it's nice to be able to rule out you know, is my throat? Ravid: Wait, whites nice? Allen Roush: well be well well no it's so because some people still get bacterial stuff like ten or fifteen percent and people talk about but the risk of superbugs and they don't seem to worry about the mass overuse of antibiotics in agriculture, for example. And so I'm always wondering like maybe we should deregulate medical. So I'm curious what you think about all of these things are, especially in the context of AI. Zachary Lipton: I mean, I I think there are I mean, certainly I think you're sitting in a very different place when you have treatments that for the most part have cleared a very high safety bar already. and I I think that's where that ends up being, I think where a lot of the interesting place to play for discovery and statistical discovery ends up being, right? It's it's because like you once once you have something that's out there in the population. you you have a lot of room to learn and and doctors have a lot of discretion to to prescribe off label. now I I mean It's just look, I mean, I would say in in the first place, like I I think that's a great thing. And I think it's quite possible that like we're all gonna be on GLP ones in five years and you're gonna look back on it and you'll be like, my, my, my psoriasis is better. And you know, it just turns out to be a weird kind of it sits in a category with like exercise and like not being a drug addict or something that is just like things that are really good for you. Yeah, so so I look I'm I'm absolutely for it. I I think part of where things get interesting is our language around like what is a trial and like what is what is a formal experiment. And and you know, it's one thing to look at the look at cases where people have full autonomy and are making decisions on their own, about their own care, which include Somewhat like anecdotally informed decisions to try off-label use of drugs, and then us trying to like back up into the data and say, can we learn anything from this data? Can we can we control for the right set of variables such that we could kind of interpret this as like a natural experiment and say something? Versus like when are we like actively trialing something versus some other standard of care in like a high-stake setting? And then I think al also in general it's like that there's I think an interesting question of like what is the what is the in what is the intervention? You know, whenever you want to interpret something as like we're doing we're doing a study, it's like what is the intervention? And and I think like one one idea is like we're gonna say, the AI just gets to decide and we will do that thing. And the reality is often like, you know, people have a lot of discretion about what they wanna do. And, you know, yeah, I I think there's a lot of room that you can, you know, and I I I I I would also add that less like I think we We're we're not necessarily in always in a totally concordant or like totally consistent set of views, but I I think we do tend to like clam up and think very differently once we talk about sort of like experimenting on people. You know, and and and maybe maybe we should be a little bit looser or should be a little bit looser, at least in in certain kinds of contexts, but You know, I think we do have a very different thought about like the AI system is gonna randomly flip a coin and tell you to do this or tell you to do that, versus hey, we have a bunch of people who have access to the same information and can make what they think for themselves in that context is the best decision for them. with the available information. I mean, there's a lot to talk about. Ravid: So so I I I want to ask so like there there are a lot of subdomains that AI can help here, right? Like summarize the like the meeting with with my doctor versus like I don't know, design like new drugs and like I don't know. Understand what treatment to give to to who. where do you think is the the the most like the best direction to go and and where do you think it's like, you know, wasteful. Zachary Lipton: I don't know about I'm not trying interpret like the wasteful side of it. You know, I I I think that there certainly is this question of like Ravid: L let's start from the b let's Zachary Lipton: I I think you like it. Ravid: start from the beginning. Where where do you think we we actually can use AI for for it? Wh where what are Zachary Lipton: I I think there's a lot of parts of the Ravid: the problems that we can solve with AI? Zachary Lipton: Sure. I mean like I I think it's a lot of parts of the pipeline that like sit in front of like the what is the big gate? You know, like I think drug discovery is a great one, right? Where you're like, ultimately like like there's no regulation around like where does an idea come from? Right? Like where are you allowed to come up with a hypothesis? how do you come up with a candidate? And and once you do come up with a candidate, like we have a kind of mature pipeline of like How do you ultimately establish safety? How do you establish efficacy? So I think like like those zones where it's like, hey, like this sits like sort of in front of the gate, you know, or it sits like upstream of the gate. I think that they're they're very safe places to play. and I think you you there are places where you see action, you know, moving really, really, really fast. I I'd also say that Yeah. Ravid: Because I just want to say like we we talked like like I don't know two or three weeks ago with Daphne, Daphne Koller, and Zachary Lipton: Sure. Ravid: she has she has like a a drug discovery company, right? And she said look like AI it's great, we are using coding agents and we are iterate over like their ideas, but like at the end of the day, it doesn't save a lot of time, it doesn't give me Like this, I don't know, like ten X, one hundred X progress in the end to end drug design and and and discovery. so do you think these are like like all the cases are like that or do you think they're like they are subdomains that we actually can get like this like huge improvement when we use AI? Zachary Lipton: No, I I I I do think there are are cases where and and I'm not gonna sit here and pretend to be like a world expert on drug discovery, but I'm sure even there, you know, I th I think it's a question of like what's the bottleneck? And and I I I think I saw a summary. I I don't think I saw the full interview with Daphne, but if if I if I remember correctly, correct me if I'm wrong, I think the argument was something like, hey, like, we weren't bottlenecked on the proposal or something, we're bottlenecked on the speed of like executing experiments, and these experiments are like They're they're they're bottlenecked on like the physical world and Ravid: Yeah. Zachary Lipton: the time it takes between like doing the experiment and when the result is ready and things like that. And look, I I'm sure there's cases where that is the bottleneck, and I'm sure there's other cases in drug discovery where the surf space is like positively enormous and and having proposals that, you know, you know, like give you you know, like one tenth, one hundredth a noise, you know, or increase increase your batting average dramatically and are gonna make a huge difference. for sure. Ravid: So yeah, so what do you think like but what what do you think like in healthcare are the the things like the I don't know for example like probably like transcript, right? Now now like everyone like I'm going to my doctor, like everything is like transcript and like and and I can get it and and probably L L L can get like the important topics and like next steps and all these things, right? Like automate my my my meetings. But where but What other problems you think we can actually can can use AI? Zachary Lipton: Yeah, I mean I think there's a a neat like progression there, right? Like so when we started off, there was a lot of technical risk associated with building an end-to-end scribe. Like we we were in like twenty eighteen thinking like, hey, we want to go from raw doctor-patient conversations all the way to like fully completed documentation. We want we wanna be able to generate the notes and the history of illness and the doctor's plan and you know, and whatever like documented informed consent and prior authorization, like all you know, And and at the time, like we were talk we were like looking like BERT models that had like a hundred-word context window and were really good for, you know, or like T5 models are really good from going from like sentence to sentence, but not handling a, you know, here's a 10,000 word conversation together with a messy, like sort of prior history now generate a a complete write-up of what happened. And so like we've gotten there. And obviously there's still like this in you know, there's always a next level of detail. So like that that project, you know, and that product is continues to be refined and to dial into, you know, deeper and deeper into workflows and different specialty needs. But like that, that that kind of like, you know, that that's now accepted as like a fundamental part of like medical operations. And and part of what made that space safe to play, like I was saying before about sitting behind the right gate, is that Look, it's always been you know, we've always trusted doctors' judgment about what goes in a final document. And we've also always been okay with, for example, human scribes who are like sitting in a call center to play some role in writing a first draft because we trust the doctor to be able to set the gate for what actually is the the final truth. We've been okay with medical students writing the first draft. So that's been like one place to play. But like now we're getting into this interesting zone, which is you're going beyond describing and you say, well, now that you have AI system that's actually party to the medical conversation, they're able to tiptoe into the clinical content, not just from a standpoint of summarizing it or documenting it, but also having interactive clinical decision support. now I think the question is like, you know, so so the the first big step in like what we call, you know, to to us, I think like Us in the machine learning community that have been training models on large medical data sets, training statistical models, trying to predict outcomes and treatment effects. We view that entire practice of like dis you know discovery of patterns from statistical data and turning that into prognostic and diagnostic and sort of treatment guiding tools. Like we've called that all clinical decision support. But in the industry, what's fallen first. Has been a s a specific meaning of clinical decision support, which is actually mostly like applying the knowledge in the medical literature to a particular circumstance for the doctor. And so it's become like tools like a bridge has a clinical decision support product. There's a there's a another company called Open Evidence that has a tremendous amount of success. Allen Roush: Right. Zachary Lipton: What what they do is basically They th you know, the the what's made it sort of like a safe place from I think from a regulatory standpoint to play is they say, Hey, we're not we're not inventing new medicine. We are we're essentially a a very sophisticated rag system that has access to all of the canonical reviews of the literature and and the medical articles. And in the context of this particular patient encounter, we're gonna query in the background These are the relevant medical, you know, these are the the up-to-date surveys that that summarize like kind of standard clinical care evidence. And if you ask a question, we're able to apply that to this particular circumstance and say, here is here's, you know, you ask a question, like, what is what are the likely diagnoses? It can give you an answer with a set of citations back to the article and some explanation of why. So now it's this interesting space where you're you're moving from a pure clerical product into a space where Yeah, like may maybe you're you're sort of outsourcing that hey, the medical decision making was actually already done by the the medical insight was what came from from human researchers and was put in the article and we're just we're just searched. But of course there's that the there's a lot of room there and a lot of play and in what are which which information are you deciding is relevant to which circumstance? There's somewhere under the hood, like there is a real medical judgment that is taking place. And of course, like the next step could be. At what point do some of these systems not just sort of surface information from the medical literature, but actually essentially are are actually doing statistical research in the background and can actually say not just, hey, you know, according to this article in JAMA, this is the case, but to say, look, we're looking across every single, you know, 70% of the patient records across the entire United States, and we know in this particular circumstance this is what happens. subsequently and are applying like novel clinical insights. Like at what point do you cross from you know, like kind of dumb AI note taker through to a more hyper-involved intelligence scribe through to interactive help you take pool the the existing knowledge but of all the world's experts and apply it contextually to like actually fabricating new knowledge and And making treatment recommendations. And I think there is a very, there's a very gray spectrum here. And and I think what the community ends up doing is I I think we wanna put these in neat categories, but I think overall what's happening here is is there's there's I think an organic process of people building technology, finding how it's used, starting to see creative ways that people are using it that are maybe you know pushing the envelope. allowing some time for society to catch up and norms to change and then moving moving things a little bit further. Ravid: And do you think we will say it in the future? You know, like fully autonomous system that make the decisions or like it will be kind of like kind of co-involvement and like decision like that involve both like the the humans and the machine? Zachary Lipton: Like are we gonna be augmentative or s supplement supplementative or Ravid: Like I don't know, like do like do you think I will I I I I now I I'm going like to to to the clinic and like the the system I have like Antho I have all my like test results, like whatever, and I will get like you have this and this and you need to do this. and do you think like we will see doctors? Like do you think like maybe it will be kind of like Antamo interface that the doctor like asked them questions the the system, the system answers and like In some shared decision they they come to a conclusion about about the situation. Zachary Lipton: I I'd say that look on one hand, I think you have systems like a bridge and and others in the ambient space that I know but I mean that are that are very directly augmentative. but look already, two things I'd point to. One is you already have a whole bunch of startups that maybe in very narrow domains are trying to like fully automate something. And you know, I think You know, you have these kind of like more specialty single purpose type type companies that are trying to like narrow down the domain and then what you know, some combination of software and AI, not necessarily super AI laden, but they're trying to sort of fully automate and take the human out of a decision making process or or reduce their role as small as possible. There are those companies, and I'd give like hims and hers for like kind of the like sexual and reproductive health. There's the GLP1 only, sort of like specific pharmacies, or maybe they have a human signing the prescriptions, but really like your your interaction is almost entirely through through chat and through web forms. there's already a couple companies. There's one called Legion Health that focuses more on like psychiatry, especially when it comes to like medication renewals, where they're trying to basically provide it, provide an experience that takes a human out of the loop. And Doctronic is is another one that also has a sort of like, you know, they're going much more for the replacement angle than for the augmentation angle. So this is already happening. the other thing that I would add, by the way, is like look, open AI is it they're they're they're also like kinda like like like edging the Overton window here, right? Because they're they they've sort of, you know, they're the They're they're able to kind of put people in this position where they're like, Chat GPT, they've been explicit about their healthcare aims and this this idea to like normalize as long, you know, and the thing they hide behind is like, hey, like we we don't have the we can't write a prescription. ChatGPT can't send the order to the pharmacy. So the the the I think the norm they've tried to establish is say as long as we're not the ones who write the prescription, anything goes. Right? And and so what's happening is like kind of. I don't know about you. Like, look, I've I've had plenty of totally straight-up medical conversations with Chat GPT. And may maybe I take it with a grain of salt or not, but it's very often like I'm trying to figure out like I'm I'm feeling this way. Is it like I have autoimmune thyroiditis? Like is it is it because my thyroid levels are too low, too high? What what should I do? Should should I schedule, you know, get my labs done, whatever. And then, yes, I like I I text my doctor afterwards and say, hey, Can we call it in? But like I'm we're already engaged in this game where like sometimes I think we're we're using we're using like chat GPT as almost like a test run for a much more like automated style of care. And and both both for good or for bad. But I mean I don't know who who can you talk to that hasn't like uploaded a picture of their child's rash or I don't know, like we're we're we're expecting now, so I'm expecting a baby girl in November. Ravid: Congratulations. Yeah. Zachary Lipton: Thank you. Very, very, very excited. But I am I'm I have like a nonstop, you know, like anything, anything happens if baby's kicking, the baby didn't kick for twelve hours, you know. just like hey, it's twenty five weeks, five days, like tell me how many, you know, how many fingers does the baby have. so like I I I I do think that the the reality is that we are testing the waters on all fronts, as we are as we are for you know So many other professions. I I'm doing it with legal too, right? What what legal question do I ask my lawyer that I haven't asked ChatGPT first? Ravid: So but but w what do you think are the problem? Like why you think we we we still don't see right that like I don't know, like as for example in coding agents, right, like we see that kind of like software engineers like you see a huge progress and like everyone are using coding agents, d no one actually like write its own code anymore. Like what why it's not happening in in healthcare? Or like what are the problems that we need to to solve first? Zachary Lipton: I mean I might push back to at first, like you know, for the f first pushback would just be like, Ravid: That's okay. Zachary Lipton: look, the token spend is not equivalent. But I wouldn't say we don't see it. Like I I say like, you know, a bridge went from zero people using like AI scribing product in in January 2023. To like, you know, a bridge alone is sitting in a footprint that is something like half of the United States health system is engaged. And the other half is also in between between Microsoft and the long tail of like other startup competitors. I'd say like 85%, 90% or something of all of healthcare is in is using ambient technology. and and and they went from using just for primary care, just for scribing, to now using for primary care, surgical, emergency department, inpatient. And not just like, you know, it used to be they just used it for draft a history. Now they use it for the history, the plan, the medication orders. They use it for the building coding workflow. Now they're also using like in the loop, clinical decision support and that product is taking off like like like wildfire. So I would just say like I I wouldn't say we we're not seeing it. And then and then on the software side, what I'd say is like, you know, Like yes, the software tools are taking off and so many people are using it. But you know, like for the most part part, like yes, you have other people who are who are vibe coding a bit. I'll put include myself in that category that I've been I've been a professor with a 20 person lab and then running a 300 person organization and my coding skills got a little bit, you know, embarrassingly rusty. And they certainly give me like a an on-ramp to get dangerous again, and it's it's a lot of fun. but like at the end of the day, like It's sort of like yes, everyone's using the tools, but you don't have like you know, you don't have like Jimmy from HR pushing like production code into your like you know, like your product. So it's like the set of people that can make a commit or like merge a pull request that's gonna totally like I don't I wanna use a cuss word even though we're all from New York, but you know, could totally, you know jank your your your your product is is still like the same set of people and okay maybe it's like altering some of the landscape of people's hiring plans and way people conceive of you know but like I I I I would just push back and say like you know I think in either way you have like the entire industry in a span of three years going from not using this stuff at all to having like total uptake and thinking that it is like transforming the way they do work. you know, I I think the interesting s difference that I would call out though is like in the case of software, I think we, you know, in the case of mathematics or something, I think we like it's I I think we we're starting to think that because we have these huge, well-contained problems with like the the right kind of simulated in environment and verifiable rewards that we can collect at scale, the trajectory towards going not just like Able to lubricate process, put knowledge at people's fingertops at fingertips, speed up productivity, but actually to do something way more radical, to go like wildly superhuman. That seems to be a big difference. I think in software, I think in mathematical research, we think that like, hey, we could we could solve problems that no one could have ever solved before. we could solve them at a thousandx the speed. There there's more of this feeling of not just not just that this is a thing that everyone will use, but that like, holy shit, the field will be unrecognizable. And I think in healthcare, maybe it's more like we've we've done that first step but not the second. I think I think we've gotten to the point of like, let's put existing knowledge at everyone's fingertips. Let's make the let's let's make the process of practicing medicine less painful. Let's make it say you can review a chart in in six minutes because the AI helps you versus sixty minutes. Let's make us say your note taking takes you three minutes of review versus thirty minutes of typing. But no, like I I Ravid: Bye bye. Zachary Lipton: think I think the the going from there to like actually, you know, patients live forever and we cured all disease and and we're living in the era of like you know, like post singularity like radical medicine. No, I think I think that no, that I think that's that kind of you know, euphoria hasn't hasn't happened on me as as fast or there's less of a feeling of we're on the precipice of I I I don't even recognize what it's like to be alive or to I don't know what health means anymore. Allen Roush: So Ravid: Because like, Allen Roush: so I I have some questions on this. First of all, you know, you had previously talked about open evidence, which which is a fun one for me because in my debate community world we had our own open evidence project completely and wholly unrelated to the one about healthcare that has a name collision. But I'm Zachary Lipton: Right. Allen Roush: very curious about how it's being used, you know, your your open evidence in healthcare and how much in general democratization are you seeing from these kind of models, especially like these open source or open access models we've been getting like recently out of China and to a lesser extent out of like Meta and even possibly some other Western companies hopefully soon. Zachary Lipton: Well we're living in a funny a funny era right now where the word open usually doesn't mean open at all. and we have open AI, which is the the world leading purveyor of or or you know, of of closed models. Allen Roush: Liar. Zachary Lipton: And we have open router, which is a closed source infrastructure project for engaging with models, including open and closed models. and and open evidence, I think, is no exception. It's not an open source project. It is It is it is it is a a closed for profit company that provides a a chatbot experience for doctors that is sort of backed by an engagement in the medical literature. and and it is a it is a wildly popular project product and and also, you know, the the you know the there's an interesting platform, you know, collision. I I think we're living in an interesting moment where like every product is becoming every product in some industries because it's so It's never been easier to build and you know, open evidence is building a scribe, a bridge has a compelling clinical decision support tool. It's a very interesting dynamic. So I would just put that aside as like maybe like that's a separate question. And then the other one is like, I don't know, the the the the independent of whether any of these companies in healthcare are open source, there's a question of like, what should we make of this given the this moment and in in closed versus open models? And I I would just say that I mean I mean I I'd love to kind of like pivot into that topic and you know I I feel very strongly that we are living at like a fork in the road in history of do we wind up living in a world that is where everybody kind of operates, whatever it is that they are doing, whatever software they're building, whatever kind of enterprise they're running, where they are operating at Sort of like the mercy of two, you know, very interesting but very particular like corporations with their with their own profit agenda and their own weird set of beliefs. Or or do we live in a world, you know, and where people just are completely at the mercy of those companies that are trapped in walled gardens and vertically integrated products and, you know, long-term spend commitments and or and and and also, you know, being in the kind of situation where They they have no autonomy of like how can their data be used. Like Anthropic recently changed their terms of service in a radical way that eliminated zero data retention. They gave people 30 days' notice. And of course, who who can who can rip their workflows if you're completely married to their ecosystem? I think the other branch in history that we could be living in is a world where actually Yeah, maybe these maybe a a small number of of a big labs are able to nibble at the very, very, very frontier of mathematical or scientific kind of capabilities models, but that actually the the core is rapidly commodifying and that what we actually experience is an open marketplace of AI capabilities, and then we can unbundle a lot of, you know, both both The consumption layer for for end users, for individuals and corporations, the agent design layer, the workflow builders, the vertical AI companies that they can all truly unbundle their businesses from such a kind of like subservient dependence on the big labs, and instead kind of we could put the power back in the hands of demand. We could live in a model agnostic future where You know, all the labs, the big labs, the open source models, the inference providers, they can all compete for token spend. And we we we re-engineer some of these key pieces of infrastructure to to work for the work for the buyer, not work for the seller. and and that's you know, where a lot of my head has been recently is thinking about, you know, how do we, you know, I think for the sake of for the sake of just like democracy, for the sake of the vibrancy of the startup ecosystem, for the sake of open science, how do we how do we bring that world about? And and I think there's a lot of obstacles that I I I'd be happy to go into, but you know, I I think that's I don't know, I think it's like the the noble fight of our time. Ravid: So do you think it's possible? Do you think a world with a lot of different models for specific use cases is a reasonable future? Zachary Lipton: A hundred percent. I I think I think you're already starting to see it. in some ways, but but I think there's a lot of things that that stand in your way. I I think most end users who are like normal people are trapped in the consumption layer. Like they either use Claude or use open AI. And then the more battle agnostic model agnostic front ends, there's some that are really good, but they tend to be for more power users. They tend they tend tend not to be for like the masses. But I think you're already starting to see it. Dan You know, I I think I think when it gets to things like, you know, there's the the the router companies I think have done like a really interesting thing in this space because I think they have they haven't really solved any of the like you know, I think we can go through all the points of like what makes it difficult for people to like, you know, take power back, move workloads around. And and it's like part of it is like commercial complexity, part of it is like infrastructure complexity, part of it is like UI entanglement. Part of it is the fact that these models are not directly substitutable goods. They're not commodities. It's not like a barrel of oil is a barrel of oil. Like a request that's sent to OpenAI and a request that's sent to Claude and a request that's sent to Kimmy give you back three different responses. And actually getting into the weeds of like understanding are these three that are equally good for a particular use case is inseparable from the content. You ultimately have to be close enough to the content that you're sort of getting into the eval game. I think what the router companies have done that's really interesting is they've created, they've they they've dealt with part of like I think the commercial and infrastructure complexity, right? Like if I wanna be able to touch 300 different models, I shouldn't, you know, I wanna be able to do that with one commercial relationship, one security key. I don't wanna have to maintain 300 contacts, 300 through 300 contracts, 300 keys. I don't wanna have to monitor the health of 300 different endpoints and then also have to be like duplicative and have fallbacks for each of those 300. I don't wanna, you know, and and so I think those companies have given people the ability to go one to many at an infrastructure level. But what they don't do is they don't tell you like where should your traffic go. Right? And so the hard part is like, you know, you develop some lot of trust of like, okay, I I think you know, I have some sense of what to expect from from OpenAI or what to expect from Claude, so I just wind up like by default sending much of my traffic there. And then you have this very ad hoc process of some people saying, like, I tried sending this thing to Kimmy or something and it's doing well, so we could, or deep seek or whatever. So maybe maybe we can save some money for the business by doing that. But every single company is going through their own like nightmarish, completely ad hoc. Process of having to separately figure out for themselves what workloads can move where. And this just doesn't seem like this should be a thing that every single company does in isolation. Every single company has to, like, there's enough commonality in their workflows that it shouldn't be that every single person has to separately like guess about can I move this workflow? And will it be the same? Will people complain? And will processes break? And how much money will I save? So, you know, you do have a bunch of people starting to. Play at this level that I'd call like the the intelligent router. And you know, I think LM Arena is trying to come into there. They're they're making a bit of a move from like, hey, we started off as evaluation, but evaluation could turn into a decision of what to send where. So like let's become a router. You have open router trying to come up in the stack, and they say they're offering an intelligent router, but I don't think that's what any of their current customers are actually paying for at volume. They're really using open router, I think, more as an experimentation platform, and then You know, once they decide what is the good model, then they ship that in deployment and don't use open router. but you start seeing the shape. You know, Databricks had an incredible, a really awesome blog post recently about how they've been able to save money internally through some combination of changing their defaults and implementing smart routing policies. So you start seeing people come up in the stack to try to say, How do we how do we reinvent that part? And of course Anthropic wants to say, no, we'll we'll route intelligently for you. and that was what like Opus 4.7 was, right? Remember, it was like it was under the hood deciding, hey, I'm gonna send this one to send this one to you know Opus and send this request Ravid: Thanks for watching! Zachary Lipton: to Sonnet and send this one to Haiku, and I'm gonna I'm gonna turn reasoning high and reasoning low for this one. But I think the fundamentally it's just the person. Like like yes, I think for if they have a long-term mindset, they might want you to not grotesquely overpay so that you become a long-term customer. So there's a twist to Yeah, Ravid: But it is config of interest to you, right? Zachary Lipton: exactly. At the end of the day, like their job is to charge you more money. Like your cost is their revenue, and you can't get away from that. And and and so it's like at the end just like AWS, they'll send a solutions engineer. Solutions engineer will give you some tips about how to set up your system smart. And ostensibly they'll try to help you lower your AWS bill. But at the end of the day, like AWS doesn't want you to lower your AWS bill. Like, like, you know, every single year they're gonna meet with you. And the the meeting they're gonna have is how can we get you to guarantee that you're gonna spend more money than you spent last year? And I I've been in that meeting with all the cloud providers. Ravid: But... But, like, besides routing, do you think also there is, like, advantage for, like, expertise, you know, when I have, like, the expertise for this specific domain and, maybe the data or, like, things like that that push... Zachary Lipton: Well, I think whether whether whether we're talking about the routing layer, I think we get way higher into the stack about how do workflows move, how do agents reconfigure. I mean, I think these all require deep expertise because ultimately like the question the question of what can go where is a question of like how can I get better better performance at cheaper price and do it faster. And and and all of that discussion of like what can what what can be done cheaper better requires that like it's easy to measure the cheaper, it's easy to measure the faster, and it's extremely difficult to measure the better because it it comes down to like do you actually understand the problem in that domain well enough to be able to to to be able to trust an evaluation that you have for this kind of use case. and so I think there's an incredible amount of use for in some ways like solving the orchestration problem the intelligent orchestration problem requires in some ways like understanding all the domains in the same way that building the frontier model is does in the first place. But I like to think that this should be like an equal and opposite force. And this should be a force that like ultimately is not accountable to the labs. It should be accountable to the people. And it should be accountable to the the individuals and the enterprises that are purchasing. And it should have a goal of giving you a better pro it should of of putting downward pressure in the market, not a goal of you know, juicing the bills and getting everyone to token max and you know, charging somebody twenty thousand dollars because they asked Opus five thinking to you know, do a thorough job of reorganizing their their Google Drive. Ravid: it's a- Allen Roush: But I I w I worry a lot about like institutional xenophobia preventing companies from switching to these much cheaper models, in spite of the high costs right now. Like what do do we just need Western providers to start r releasing open access models or what what's what do we do about this? Zachary Lipton: It's a great point. I I think that I wouldn't I wouldn't be so extreme as to say like there's absolutely no risk about I mean, people in in a cavalier way using a model, to you know for for any purpose or something. I like but but but I I I do think that the risk conversation is tainted. Because I think it's tainted by interest. And I think, you know, it's not like this is this is the the like absolute, you know, I don't know what's the right word, this is like flamethrower like just scorched earth war that's going on on like Twitter every single day right now, right? And it is like it is like anthropic Kind of posturing in a way that seems to be tilting towards restriction of the use of Chinese models, teams to be leaning in the direction of regulatory capture, and of course, dressing it all in the language of safety. And then on the other hand, you have the extreme position, which is like, hey, like we're running this on our own servers, on our own GPUs with our own guardrails, often having fine-tuned them ourselves for our own purposes, having run our own evals. We have a lot more window into what these models are and the ability to like poke and prod and test them for various kinds of alignment than we do with anthropics models. If you're like an enterprise consumer, and like how dare you take them away from us. And look, I I I think I think you get to your really great point, Alan, which is like in I if I would love to live in a state of affairs where the American open source or at least the American open weight community had caught up with the Chinese and also was knocking on the door and like. I wish them all luck. Like I I hope Thinky gets there. I hope Reflection gets there. Because at that point it t I I I right it if if we both believe, which I think we're maybe open-minded, but I think tend to believe that some of the the the use of xenophobia is like it's a red herring, it's a it's a bad faith argument to take away competition, it seems they wouldn't be able to direct that same weapon against American open source community. And and I think it would put the like Overall, I I think if American open source or or open weight caught up to the Chinese frontier, it would it would it would take like the most powerful weapon away from the sort of like I don't know, pro regulation, pro ossification, like anti competition kind of forces. Ravid: I want to talk about different topics. You're a professor at CMU, and you founded, you're a co-founder in a startup, and I think, let's say in the last maybe like three, four years, there is a huge debate. What is, like, where the research, where AI research is going to, right? Like, we have both, like... first of all, there were a lot of people and lot of efforts and in computes went to the industry, right? That like now basically you can't make frontier models in the academia. And now we have all the self-improvement, right? That like maybe in a year from now we actually don't need AI researcher anymore and everything will be fully automate. So what is your take about it? Do you feel like, where do you see AI research goes? And like, how do you see this split and where are the... the border? Zachary Lipton: Yeah, what a what a weird time to be alive, right? Like I think depending on like possible features we're willing Ravid: and to be a researcher. Zachary Lipton: to consider. Like there's we could be having the question of where do any of us go? And like, you know, how how are we gonna get really good at playing the banjo in the like post singularity world? Or or we could believe that we're in this like middle scenario, which is like industry is still thriving, but the Ravid: By the way, do you... Zachary Lipton: frontier has created an existential crisis, but only for academia. I Ravid: And do you support, do you support like the, I don't know, the paradigm, the San Francisco paradigm that like in the next six months all the white car jobs will be eliminated? Zachary Lipton: No look, I I tend to be kind of I'm I'm I'm more of an agnostic than an evangelical in in in religion and in life. I I I do not I do not believe that all white collar work disappears in six months. I think overall the people saying that probably underestimate the complexity of the economy, underestimate the the the the difficulty of large scale change underestimate like just how far behind so much of the rest of the corporate universe is. But on the other hand, like I I also recognize that like I haven't always had 2020 vision in in my forecast of the future. And and in particular, you know, I I I think like one thing that I could own, and I think a lot of us who I maybe I don't know if you you group yourself in that bucket, but you might of like, you know, like like I I I think that a lot of us who part of your identity as a scientist came from a skeptical mindset, not from a a s a mindset of true belief. And so that led me in general to be quite skeptical of a lot of the language that was coming out of open AI in 2018, 1920. And not to make this like a mayor cult or like an a an apology, but but I I would certainly like admit that like I I think there's a lot of things that I was I think my model of the future was wrong about, right? Like I think I I thought, I think that at some point the like it was pretty cool that we could fumble our way to to a flying machine, but at some point we were gonna have to figure out aerodynamics and if we were going to build XYZ and then you know, it turned out no, there was so much there was so much further you could go in a in a fully empirical kind of trial and error incremental accretion of of technique. world and and and I also I think I grotesquely underestimated just how far scale could carry us. And so I I think I approach this of like no like my mode is not like I I I'm I'm definitely in a mindset of of of of supporting and advising and launching new zero to one efforts because I believe that like I believe that the economy and that creativity and just possibilities that it's we always I think underestimate just how much there is to do. And and I I You know, I I like to think that it's a little bit like this world of, you know, I think a lot of people talk about this historical antecedent. of like and and there's a there's a great economist who used to be a colleague of mine at CMU, Alex Eemus, and now now he then he went to I think Penn and now he's at Google DeepMind. I think he's been a Ravid: It was in our podcast. Zachary Lipton: awesome. Yeah, yeah, yeah. I think I saw that. Allen Roush: Yes, yeah, yeah. Zachary Lipton: I think Ravid: You Zachary Lipton: he's been a a very interesting voice to follow on on this of someone who's like sitting inside the the the the the companies, but also has some context of like Agriculture being, I don't know, I whether it was 100 or 150 years ago, being, you know, 90% of the world's employment going to 1%. And yet, like, you know, the the income of the median person is so much higher than it ever was. so so I I at once think like I'm I I think I'm I'm I'm not like a hard cold water, like it's all bullshit skeptic. I actually think like something fundamentally economy-shaking and transformative is happening. And I also think that it's sort of a poverty of imagination to be certain that this means there's no role for any of us or that, you know, and I I like Emos's challenge. I don't have the answer to it, but the challenge he gives is to ask, well, well, when when when X becomes abundant, what what then becomes scarce? What is the new bottleneck? And I like to think that there's a more interesting answer than just like everyone becomes a massage therapist or something. Ravid: Ha ha! Zachary Lipton: but but but I do think there is Th there's a lot of thought and and maybe the economy looks radically different. Like maybe you live in a world where there's, you know, five hundred times as many startups, but the the average firm size is much smaller. I think there's so many possibilities for how the world reconfigures. And and I also believe that there could be a lot of pain along the way. But look, I think there was a lot of pain along the way when manufacturing got offshore from the United States. I think there's lot of pain along the way. but it doesn't necessarily make me embrace like an apocalyp apocalyptic scenario. I I think we need to be thoughtful and understand how the world is figuring out and and figure out like what what used to be good advice for people but now is bad career advice and how does the university need to change. And I think we have to have our eye on it, but I I don't I don't come from it like I I don't have this mindset of like just like have a a greedy, grubby land grab of like hoard your capital over the next three months because after that you're gonna be living in, you know, once 2026 is over. You're either part of the permanent aristocracy or the permanent underclass. Like, no, I I that's not my mindset. But at the same time, I am open to the idea that look, look, whole industries could crater, whole new industries could be created, the world might look very different. and I I think I'm just like a little bit more agnostic about what that would look like. So I I I know that I Ravid: and for it. Zachary Lipton: like I I get to like specifically the question for PhD students, but I'm also happy to go there. Ravid: Yeah, no, like for AI researchers, right? Like what do you think? Like first, PhDs and maybe like even after it, do you think, do feel that like they are still place to do research? And if so, like in what types of research? Zachary Lipton: I mean I I think my immediate answer is yes. I I think there are. If I can continue to like riff on the theme of like historical antecedents. you know, I I think the one that I come to a lot is look search. Like information retrieval was like the big thing in computer science. Like like this is kind of like big data, IR search algorithms. Like in the mid 90s, this was a very hot area to be in academia, and the action was in academia, you know. Larry Page and Sergey Brand published PageRank at Stanford, John Kleinberg, who I mean, what an inspiration and like what what interesting topic hasn't he yet touched at some point in the last 30 years. But he had the like hub and spoke model of like relevance scores that was like a very similar kind of idea. Like this is where all the juicy ideas in search were. And it was like if you wanted to work in like web scale data and like organize the world's information and make it searchable, you you did it at the university. And at some and and of course we had big conferences. We had SIG IR, we had KDD, we had these w they're still around. But they were like they were like bigger than NERPs. They were bigger than ICML. They were where the action was. And it was probably easier to get a job if you were doing like search algorithms than to get a job if you were doing like LSTM, you know, like recurrent neural network researcher. you know, and indeed look, look, Sep Hochreiter in in the late nineties, he published a LSTM paper, like What an incredible feat of imagination. And then he went to go into biostatistics because there were no jobs in machine in computer science for him. He went into like bioinformatics and came back when people suddenly was like, like you you got 700,000 citations yesterday, you're back. Ravid: you Zachary Lipton: So so I I think like we've lived through this before of like an area it this is in the sweet spot for academia, and academia gives birth to it and germinates it, and then The center of action moves and suddenly it's hard to compete if you're not, you know, s like the research moves into a resource intensive the the research moves into a resource intention intensive regime. I'm like a tongue twister, where it's very difficult for an academic to compete. I I think that that I think that that that's happened for for language models, right? Like I I think for many of the questions you want to ask. It's like what does academia do? I I I think one thing we've done is like I have a a student. he's actually an incoming faculty member at Cornell Tech. And he his name is Pratish Mani and he was one of the founding engineers of Datology in parallel. I don't know how he does all the things. you know he he made a really interesting PhD run by writing scaling laws in an interesting way and he and he got very involved in a lot of the questions around how data curation influences models. And he was very good at establishing a pattern and then handing off to a big lab. And you know, w when esta like understanding the way different types of data filtering, different types of data curation strategies influence the performance of of a model, establishing that the the the pattern either sustains or even widens as you get to larger regimes, and this would get the attention of some of the big labs who would then actually take his ideas and run with them. And he did a lot of these works, like rephrasing the web and like different ways of like kind of like eliminating shortcuts from like sort of like sort of joint like clip style, joint like vision text embedding models. So I think that's one thing. I I think there's always going to be probably something of like the on the more idealized theoretical end of the spectrum in academia. You we can always ask this question of like, well, what happens when proving stuff is just nobody can, you know, like all all creative proof writing is is the province of of of language models. And I think a lot of people are having an existential crisis now because of just how good LLMs are getting at proving mathematical theorems. But I think there's still a lot of work to be done in actually coming up with the right the right thought experiment, the right construct, the right setup that I don't know that we've seen language models do in the same way. I think they've been better at like come up with a proof for a crisp statement where that's the part that has like the verifiable reward attached to it. And the like what is an interesting question or what is an interesting model is I think we've made less progress because I think we don't know how to we don't know how to formalize it in a way that's amenable to the same kind of hill climbing. I I think there is always like the the normative side of things. I I think a lot of questions about how What is the impact of these technologies? How do we measure it? And and what sort of policy changes might society consider where academia has a a unique advantage in being like being a little bit more impartial and not just representing the the interests of any particular company. That's a place where academia can play. So I I think there's there's a lot, there's I think there's a lot of terrain, but I also like I feel the pain. Like I Look, I think you and I are similar generation. I might be like three or four years earlier and when I when I started publishing, but you know, 2013 to 2020 or 2010, 2020 is like what like such a golden era for doing bleeding edge, Ravid: You Zachary Lipton: like neural network adjacent, machine learning research, everything from the applied through the empirical through the kind of like theory of and understanding of deep learning was. Such an amazing time to be in academia where if you're a creative PhD student with four GPUs, you could you could become Ravid: Yeah. Zachary Lipton: you could become a a a pillar of the community with just like your the the thoughts in your brain. Like it was a it was a magical time, you know? And like Ian, Ilya, like all these guys came up, they became like found a a a voice, changed a whole field over, you know, with Ravid: Yeah. Zachary Lipton: from their from their lab while like eating ramen. So like I mourn that. Like I I do like of course it's like I feel bad for like my PhD students that like it's a lot harder to to be noticed today. It's a lot harder to also like the conference are overrun. Like how do you even make a paper that someone believes you even wrote it and isn't AI generated? It's it's such a harder time to be wrong. Ravid: Yeah. Allen Roush: Yeah. Ravid: But... But it's still like, I don't know, I just saw like the paper yesterday, did you see this paper from the group in Germany that like they took... They took like the... Like the... The reasoning traces, right? Like the filtered reasoning traces of opus and things like that, right? Like basically it's like... It's... It's not a raw data, right? It's kind of like a simplified, very nice-looking version of the reasoning. And then they just use the weaker models and just ask them to decode these traces to the original ones. And it actually works. And, and, and yeah, it worked and like they found really cool thing, you know, for example, I don't remember like maybe it was opus or like Gemini that like you know, insert some like math question from the from the the some competition like some Olympiad competition and and and like the the filtered version of the of the reasoning like you the what they actually got is like, yeah, this is a very hard question, I can do this and this and this and this, but the unfiltered one was like, I actually know this question, this is from the original map competition, so I will try to do this, and then it started to do the calculation, but it failed and actually I know the answer, so I just returned the answer. Things like that, and they release a lot of traces like that. Zachary Lipton: Yeah. Awesome. Ravid: So I think there is still space for neat ideas. Allen Roush: Well and and I'm very sympathetic to what you talk about with conferences being overran with with so much stuff. I mean, I hear about almost doubling it seems like the number of papers that are being submitted or even accepted in some cases. And so much of that's also driven by all of the remaining good jobs and really AI, even engineering land, still have, you know, nice to have or sometimes required NERTs or multiple NURIPs or whatever publications and until we drive Zachary Lipton: Yeah, totally. it's hard. You know, but I don't know. Ravid: Are you optimistic? Zachary Lipton: Yeah, I'm Look, I I mean like my my feeling in general is like Look, I was a jazz musician, you know? And You got this like beautiful, beautiful language and art and people that have gotten to such an incredible level of understanding and expression and intuition with it and they're fighting to be able to you know, y you wanna get paid like seventy-five dollars to play, you know, for like an evening at like smalls or something. You're competing with like the The five best sex phone players in the world, probably for that time spot or something. Like I I think th that background for me, I come I come to machine learning or something, to AI research, to to startups. It's like, you know, I don't know. Like I I I think all along, my whole time in machine learning, I've always had this dichotomy of like people complaining something. it's so competitive here, it's so whatever. And and I just feel like, like what a Ravid: They are just spoiled. Zachary Lipton: Like what about the other perspective? Like, hey, like there's there's so much action, there's so much demand, there's so much creative opportunity. If you're the s three hundred thousandth best machine learning scientist in the world, you have a job in six seconds paying like you know seventy times what you make as like the you know, like sixth best jazz musician in the world. Like it's just like I'm i I I I I I don't I I don't want to sound callous, like I don't feel so sorry for us. Like I feel I feel like I I I do empathize with PhD students that like felt like, I was getting into one thing and now it's something else. But I also like remember that I don't know if you came a little bit later, but like for me, I caught the tail end of like, you know, Alex Net happened in 2012. It took a long time for people to take their blinders off. And like in 2013, Like I watched a whole community that that had built a decade on Bayesian non-parametrics, on you know, sort of like support vector kinda like kernel learning on probabilistic graphical models on I don't know, like I I watched so many folks like Hold on, you know, or like they built like they were in NLP, but they were attached to like parsing systems and certain kind of more rules-driven dialogue engines and state action, like kind of representation or like intent action for what they call those, sort of like tuple representation of interaction. And like all these people, they had done it, they had done it for years, and like they had stayed with it and they were like. The sting was coming and they were like losing all the steam out of what they were doing. People were just stopped paying attention to their research and they were in a state of like a little bit of existential dread about like, you know, you know, I don't know, there were some people who were pooping on it, saying, like, this deep learning stuff, like they they don't really know anything, and this this is a huge fad. Other people just thinking, like, I don't know, just just being being more anxious. And and I would view it as like, I don't know, like, On some level, if if your job was you went to like you said, I want to do PhD in like technology in like the most sci-fi sounding field, it should be because on some level you're like, I want to be excited by a changing world and I want to see the pat like I want to be able to see the paradigms flip around in my own lifetime. And like on s on some level, if you're not a Like like all of us th like we write these papers and all anybody wants to do is brag about their novelty. And if like you find yourself in a world where like the world is changing constantly and then you're not happy about it on some level, if only like that like my that like do we get to live through something so exciting and at least just like do we get to make a lot of belief updates? Allen Roush: Well it it and and you know, you mentioning kind of the artistic background that you've had is is something I I'm actually have a bone to pick with the whole social class of of artists in general, because they for so long purported extreme egalitarian and and left wing political and social views. which meant circa t you know, the year two thousand, even two thousand five, twenty ten, piracy is great. Screw intellectual property, right. Rights, you know, and even I claim Aaron Schwartz, a proper technologist, existed sort of near the tail end of that and kind of cumulated in this with what happened to him. And now I have watched so many people, I claim the entire DSA, which you know is as an example, Democratic Socialists of America, turn and do a massive 180 because the wrong people started democratizing the information, right? Greedy companies. and I'm not seeing people who who claim egalitarian social views. Anarchists, socialists, and etc. fighting for open source models or doing like what the EFF does or Richard Stallman or any of these people. I'm instead seeing them embrace Ludditism and just saying, you know, attack data centers. I claim that soon enough they will go after AI researchers potentially, or at least anybody else that they can paint a a scapegoat as being, you know, the kind of person like Luigi style, right? And and I claim we had the Unibomber a quintessential example. So I'm curious then like in his own way, yes, MIT professors. And so I'm very worried about all of these things of what Zachary Lipton: Sort of sort of did in his own way. I mean I mean yeah Allen Roush: I Zachary Lipton: ID prison but even like I mean you re you read the manifesto it's it's not even like I mean it's like explicitly about AI, right? Like there's it it's kind of f fundamentally centered on this thing of like, you know, will the future need us? Allen Roush: Yeah, yeah, exactly. And Zachary Lipton: Yeah, yeah. Allen Roush: part of this is I reject carbon chauvinism and I think that's like a core like fundamental belief. I think a lot of people have never thought about like where they feel on like humanism, transhumanism and all of these. It's crazy that, you know, school I remember they tried to beat humanism into my head so much in school, and now that I'm seeing the possibility of life looking, you know, different, I'm I'm ready to abrogate. And I find ev even many people in AI research are not willing to to embrace The machine intelligence. For all the talk of how this is supposedly embraced in Silicon Valley, most people I know, even who are like at OpenAI anthropic, are not okay with like Messiac style, you know, building the machine god type rhetoric that that I'm actually shockingly willing to embrace for somebody on a call like this. Zachary Lipton: I mean there's so much. There's there's so much there, I don't even know Ravid: Yeah. Zachary Lipton: what there was one to first. I I mean, I think that though like there's a lot there and there's a there's a lot that's real. I guess the thing I might push back on a little bit is like I I think like you can really pull out, I think, a lot of these narratives of like who's where and and what what class is on what side, of what issue at what time, but I think it's very hard to lump that many people. You know, you know, and and I'd say that like, you know, what we're like we're artists as a class, like All for like dismantle intellectual property rights or something. Sure, but Allen Roush: Certainly not with Metallica, right? Zachary Lipton: but I I think also just in general, I I think that there was a like like Metallica, obviously like as a like jazz had like what 1%, 2% of like the total, like my my beloved but but but but you know impoverished you know kind of like corner of the musical universe. Didn't have much, but still, like I I think there was this thought of like, you know, the r record sales used to be a business. And and I think that there were a lot of people who were on on the, you know, may maybe we wanted to be able, maybe we thought artists who couldn't afford to buy records should be able to like boot like just for their own didactic purposes, but we didn't think, you know, we didn't think other people should be able to do it. Or or we did we did look with some amount of horror at like, How does something like Napster flip the world on its head and and take away like like like a a a major a major source of revenue and possibly like kind of alter the the the viability of of of music making as a as a career? You know, I I don't know. I mean I mean like a and and there's so much of what you said it's hard hard for me to respond to all of it, but like maybe one other thought that just comes to mind is that like I feel a lot of these like sort of like there there's these like big narratives that come up. Are we building the machine god? Are we transhumanists d there's a lot of like big words we could put on it and and and kind of try to create like a something to hold on to that feels like a you know, like not just like a trillion particles moving, but like like a little like a jet stream or like like some some like hey like this is this is an ideological Allen Roush: Teleology. Zachary Lipton: line that we could talk about and reason about. And and I think there are those and there are people championing my own kind of non you know, it's not that I'm like I I'm I'm neither like maybe messianic nor like anti messianic, but my own kind of like agnostic view is I I I don't know, like I I try to make space for like coming at it where with from a space of like, I don't know, like epistemic humility. Of like, I I I I think there's a chance everything changes and and and I'm I I I think it's possible that my imagination is too poor to even see the contours of what that's gonna look like. And and I wanna wrestle with the different main lines of ideas, but I also I don't know that I buy any of them totally literally, but for as like like little points on the simplex or to like anchor to anchor a conversation and the truth lies somewhere in the cracks. And I just I I I feel like you know, maybe maybe so many like there's probably a lot more people like me out there too who who like haven't like picked a a side or completely like you know or or like a collapse uncertainty on like a certain outlook for what the future looks like, but but are trying to I don't know, at at once get wise and and and also keep their mind open. You know, one thing it's interesting that like, you know, there are certain issues. Like like I think the open models issue and I think generally the like regulation issue has cleaved a little bit along some party lines. And I think you are starting to see something of like, you know, an interesting like party inversion where like I think Regardless of how you feel about social issues, regardless of how you feel about a lot of other kind of political content, probably like the current administration has been overall like more pro startup, more reticent to regulate, although you see like with like Claude and Mythos or whatever, like a little bit willing to like testing the waters of stepping in, but probably like, you know, certainly nothing like, you know, Bernie Sanders' rhetoric. But you know, I actually got to spend a lot of time in Washington. like early days of a bridge, you know, I think a lot of people are trying to make sense of like, what is this new space? How will it be regulated? Will it be thought of as a medical device? Will it be thought of as a productivity tool? How should Washington be thinking about it? And I had a background of like I had worked a little bit on a lot of the more like accountability and societal aspects of machine learning and in my PhD and faculty career and had a little bit of A little bit of background on that side of things, although I think that community has become like maybe like less salient in the in the actual halls of power now as it's become a big industry. I went and, you know, one of my PhD students actually is now leading a pillar of policy out one of the big labs at the time was working at you know between like think tanks like congressional offices. And he set up a bunch of meetings for me and I went and I met I'm at I had like a one of these whirlwind days, you know, where you go and like I'm I'm in the I'm in the Senate office building and I'm meeting with like an aide for ten s you know, Senator Cruz, and then I'm I'm going down the hallway to meet someone who works for Bernie Sanders, and I'm going somewhere else who like is part of like the joint task force on like science and technology policy, and then you're talking to someone else. And and then I think it was actually I I was speaking on one of these panels that they had. They called these the my god, I'm trying to remember the name of it, but it was sort of a joint thing that like Schumer and Rounds was this AI Insight Forums. It was it was a kind of bilateral kind of like like bipartisan like effort to like have some informational sessions and bring experts to talk about these issues. And I think that like struck me like crazy, like this is this is little bit older now, it's like three and a half years ago, is that honestly like I couldn't tell which party like I like I was talking to when I was talking to their their AI policy people. Like like the the political contour is like which side is left and which side is right. Like people had didn't Ravid: You Zachary Lipton: know. Everyone was so confused about the issue. I didn't know if I was talking Allen Roush: I I've been stupid. Zachary Lipton: to someone who worked for like Senator Cruz or for someone who worked for like Chuck Schumer. Allen Roush: Yeah, yeah, and I claim we're in a whole new political party system. You've heard political scientists talk about like fourth, fifth, etc. system. I claim twenty twenty-four and or twenty twenty-eight will reorient everything. Because I claim techno-libertarianism was was at one point basically monopolized by the left. And now I'm seeing the kind of people who I would have never thought would have talked that way, exactly like Bernie Sanders, AOC, Warren, saying things that trigger my I think the term is gel man amnesia effect, right? Where where you go go to somebody who you hold in authority, like for example, a a news publication, like a newspaper, and then you see them read or you see their article about something you know like a a lot about, and you see it's totally full of falsehoods and and completely wrong. And then you just turn the next page and you go on like, well it's gonna get relations internationally right even though it's screwed up AI, right? And so now I'm really worried about so much of the rest of I'm not obviously like as somebody who who's, you know deeply connected to like liberalism, right? Even big L liberalism. I have a hard time ever abandoning, you know, the Democratic Party as it's currently constituted, for example. But I'm very worried that political things are realigning and I'm feeling politically disenfranchised given the failure of Alexander Yang. Or Andrew Yang. my God. Sorry. Andrew Yang Zachary Lipton: Right. do you do remember this one hit wonder from the band Steelers Wheel? Allen Roush: No, I should. No. Zachary Lipton: And it's got this great lyric. Ravid: Yeah. Zachary Lipton: clowns to the left of me, jokers to the right, here I am, stuck in the middle with you. Allen Roush: Yeah, exactly. Well, and I I claim it's hard to even like like that whole thing of like we need new ways on a political axis. Like even on a two D political axis, the whole thing of authoritarianism versus libertarianism and left or right. I claim we need at least maybe not even more axes, but that the first two something else has become more important. Maybe it's pro versus anti technology is one of the two axes. Zachary Lipton: Yeah. God, there's th there's so much there. Ravid: Okay. Allen Roush: Yeah, I feel I feel bad. We're at the near the Ravid: I think... Allen Roush: end time. Ravid: yeah... we're out of time. Zach, do you have anything else that you want to add before we... finishing? Zachary Lipton: my like we we've we've gone we've gone through we've Ravid: You Zachary Lipton: gone through the pragmatic, we've gone through the messianic, we've we've ventured into the political and and I I And Ravid: Anything else that you want to promote, to sell, something? Zachary Lipton: I like to think that we're still friends at the end of it. you know I'm I'm I'm I'm I'm I'm not really selling in in this moment. I'm happily happily building in the dark. but Yeah. obviously, you know, I don't know, I still I still think the world of Carnegie Mellon and obviously proud of everything we built at a bridge and I think I would encourage PhD students and aspiring undergrads to come come study at the school that I think invented so much of of the modern AI universe. And you know, I I think I it's hard hard for me to imagine a better place and a bridge to work. for and and now that I'm on the outside I could say this like a little a little bit objectively. but like whether whether it's that you wanna, you know, save the marriage of a doctor who gets to go home and actually see their kids at the end of the day instead of writing notes until 10 PM or to work more at the like frontiers of how do we extract insights from clinical conversations that actually guide care. So like obviously two amazing places to work and things I'm proud of and I mean I I I think on living on a note of optimism, I guess what I would just say to like whether it's like students or entrepreneurs, in in this moment of like absolute insanity is just that I think like I started my PhD like we talked about before, at that moment when like the world changed and no one knew what to make of deep learning. And I would just say that, you know, the the the mindset I would have towards all of it is like that moment when like the world is changing. when no one knows which way it's gonna go, when the old tools are out and no one knows even knows what the new tools are. Like these are always, always great moments of opportunity. They're great moments to be starting your career. They're great moments to be coming out of left field and making a lateral move because they're level setting moments and whatever anyone thought they knew is is may or may not be useful anymore. And so I would just encourage people, I think it's very easy to despair. It's very easy to think about the being in a permanent underclass. It's very easy to write off. Like, is there anything left for me to do in PhD? And and I would say that like in moments like this, like they will always in retrospect be just absolute, you know, one, you know, once in a generation special, like God sends for people that have their eyes open and that are creative and that I hope that I hope that people entering PhD or entering the workforce or thinking about starting a company can can can let go of the like terminator dread and look at look at this moment for the for for all the all the potential and and and and sort of turn their turn their turn their uncertainty into you know into a weapon. Ravid: It's a great message to end with. Zack, thank you so much for coming. It was a great pleasure to have you. Zachary Lipton: Yeah, thank you so much for having me. yeah, really grateful to be on the pod and yeah, really, really been b been a fan of of of of of the the whole series and and honored to be among the long list of of of amazing people you've gotten to to come on the show. So but best best wishes for everybody. Ravid: Thank you so much. Allen Roush: Yeah, thank you, Zach. Ravid: Thank you.