Latest / The Short Coat (Audio): An Inside Look at Getting Into and Getting Through Medical School / The Chains of Med Ed History, with Adam Rodman

Transcript

The Chains of Med Ed History, with Adam Rodman · The Short Coat (Audio): An Inside Look at Getting Into and Getting Through Medical School

• Dave Etler: the controller is by me for all of our headset volumes. So we were working on, I don't know where the camera is. They can see it, but I have now labeled which little like turner we're all on because. Couldn't, I should label those. Huh? Linda Peng: That would be a good faith already took care of it for you. On our little concrete, this is what faith does. She sees a problem that she solves this. Jeff Goddard: Yeah, that's what I, that's what we like. You see my, I physically cannot stop myself. My motto is, move fast and break things. I very rarely do anything but the default settings on anything because I'm just, I'm going like, we're, we'll figure it out as we go. I think the whole point Dave Etler: of moving fast and breaking things though, is to continue to adjust as you go along and make Jeff Goddard: improve. Oh, I'm adjusting. Just not those types of things. You're not leaving Faith Prochaska: the room to go get name tags in a Sharpie to label. You're not a Girl Scout. Leave it better than you Kelsey Porter: found. A little too slow for me. I gotta Faith Prochaska: go see. I was a girl scout. Kelsey Porter: Sam, I got you. Incredible. Faith Prochaska: Now a Jeff Goddard: Girl Scout. Hey, let's go. All right, let's do it. Sorry. Dave. Dave Etler: Welcome back to the Short Code Podcast. It's the show that gives you an inside look at medical school from the students drinking from that fire hose at the production of the University of Iowa Carver College of Medicine. I'm Dave Etler with me today in the s p studio. her decisions will no doubt have a ripple effect throughout future history. It's M one Faith Prohaska. Hello. the implications of her existence are far-reaching. It's PA one Kelsey Porter. Hey. Hey. She's intimately connected with past president Future. It's M one Linda Pang. Hello? Did I say that right? Yes, you did. Okay. He's an intrepid navigator of the Labyrinth of Human Chronicles. It's M one Jeff Goddard. Jeff Goddard: That is the nicest thing anybody's ever said about me. and if you Dave Etler: think that was all short quotes, you are so very wrong. As usual, it's okay. I still love you. I do have to point out though, that we're joined today by Dr. Adam Rodman, director of the Innovations in Media and Education Delivery Initiative at Beth Israel Deaconess Medical Center and host of the Bedside Rounds podcast. Dr. Rodman, welcome to the short code. Adam Rodman: very nice to be here. Please call me Adam. Dave Etler: Okay. I will. I don't guarantee that I will remember because I am a slave to my written notes. Jeff, you invited, Adam to be with us today because you wanted to talk about path dependence. I did. Maybe we should find out more about what the heck Jeff Goddard: that means. yeah, of course. So I actually learned about this concept from, bedside rounds and following Adam on Twitter. I have to let you know, Adam, when you, when we called to, discuss what we were gonna be doing today, I was a little starstruck. it was hard for me. I've been listening to bedside rounds since you were, biking around Oregon. So, for me it was a cool idea.so this idea of path dependence and Adam, feel free to jump in and correct me. it's. The idea that, certain systems, you know, we go about trying to create systems to solve our problems. at some point the complexity of those systems becomes so large that we get to the point where it's almost impossible to change certain pieces of it because you would have to change the entire system. and you can't just dismantle the whole thing and start from scratch. an example, another one from Adam actually, is, railroad tracks, right? Like the data are in that it would be a vast improvement to railroad safety if we changed the gauge, the width of, the railroad tracks around the world. The problem is you would have to do that all at the same time because all of the railroad tracks in the entire planet are already to a certain gauge. And in order to do that, you have to convince every country at the same time that. All of your tracks are going to change, and all of the trains that we are now using are going just throw away all your trains, throw away all, and it's to say there is a solution, but the system is already so complex that solution is not possible. And that's the idea of path dependence. is that fair to say, Adam? Adam Rodman: Yeah, that's the, it's a perfect explanation. I couldn't say it better. Jeff Goddard: Okay. So today, I, the reason why I wanted, Adam on the show was, to talk about that concept in medicine, specifically in medical education. and you gave me a wonderful book, time to Heal, which, as I told some of our cohorts here, I didn't put them through reading it. It is, it's a big book, booking sticks. Yeah. we got the Wikipedia link. Yeah. so I kind of wanted to just briefly, because this is the part that excites me the most. Because I'm a nerd. give a bit of a background on how medical education has come to, to be what it is. And then, Adam, I would very much appreciate all of your input, and then we'll go back and forth with questions. So the story actually begins with, one of my favorite humans of all time,a quirky little Quaker man, in the Eastern United States by the name of Johns Hopkins. his first name is actually his mother's maiden name. That's where that comes from. And he, made his wealth and he lived in Baltimore. He made his wealth and he had, a vast fortune that he wanted to do good with, when he passed away. And so he left it in trust to build a medical school. and that medical school is named after him, Johns Hopkins. It actually took about, I wanna say about 20 years for them to. finally get everything, in place for them to build this medical school. But they had essentially the Hogwarts founders, they had the four most prestigious physicians and their respective fields come to Hopkins in order to build this program. And, all of them deserve their own books. but William Osler is certainly the one that I think has had the most influence on medical education. Everything from how we approach medical school to how we approach residency to everything beyond because of that. is that a fair assessment, would you say, Adam? Adam Rodman: Well, that's what Ludmer argues in the book. I actually think it's a lot more shocker here. Right. It's a lot more nuanced than that. because what's happening in America and Canada is not happening in a vacuum. And Ludmer has a tendency to ignore innovations that are coming from Europe and in particular the German states at the time. Yeah. even Osler himself, right? When Osler retired, he was already considered a legend in his own time. And he was kinda like, look, guys, Yeah, I did some things, but mostly I just adopted the tut ideas of medical education and brought them over to the United States. Yeah. Or the Anglo speaking Osler believed that all the English speaking worlds should join together in one country. So the Anglosphere Jeff Goddard: Yeah. So I think it's fair to say that,the influence of like the French school and then the German school of,and medicine throughout mostly, I guess the 19th century, I mean, that was the front of medicine. Right. but I think what changed for us in our country, based on what I got out of the book was, this guy Abraham wrote a report, and he was enamored with the Hopkins system and he made a very strong and compelling argument that everything should be that way. Adam Rodman: Right. Abraham Flexner. Yes. The Flexner Report, which, looms large in American medical Jeff Goddard: education. Yeah. Dave Etler: Yeah. We've spoken a little bit, about the Flexner report on this show in terms of, it's wide ranging impact on medical education. Do you wanna,summarize that for us though? Adam Rodman: Adam? Yeah. I mean, the Flexner report, it's important to understand, it is the Carnegie who are funding this. So this comes at a time of reform of education all across the board, not just medical education. So you're also having the standard form of how we do elementary school was coming into fashion, not so much the university, but o other types of education. And, the Carnegies had Flexner effectively do a survey of all of the medical schools in the country, or remember many of the medical schools in the country to look at what their perceived deficiencies were. And we can talk a little bit about if you want, what medical education was like. But effectively, Dave Etler: he wasn't happy about what he found. Adam Rodman: He wasn't happy. Exactly. He wasn't happy about what he found. And he advocated that effectively all medical education reformed upon the Hopkins model. And that would be a postgraduate medical education. So that young men,not women yet, but soon to be women as well, would've already had an undergrad degree. Right. Which would place them in a certain class of society that science curricula would be formally integrated into their preclinical years and that there'd be structured clinical time on the wards through rotations. I mean, rotations truly were invented at, At Hopkins. Prior to that, there would've been kind of a house officer model, and then this idea of structured post-graduate training for internship and then later residencies after. And, this was adopted by the American Medical Association and became the standard. Tons of medical schools were closed down after this. Yeah. Women's schools and black colleges were shuttered. I believe all but two black colleges were Dave Etler: shut down. And I think,the way medical education works now kind of makes sense to us because that's what we've always known. but as you said, it closed down a bunch of opportunities for, for people who weren't able to, Get a degree before they even came to medical school. and so it had effects in that way. That's the thing I always remember the flexion report for. Adam Rodman: Yeah. Yeah. And it's a quirky thing of American medical education, right? The rest of the world. A medical degree is an undergraduate degree, Jeff Goddard: right? Yeah. And there are a lot of things that I would prefer about that model. I think one of the things that I appreciate about the Flexner report, at least in spirit, is this idea that, we want a high standard, right? if we're going to, treat the field of medicine with respect that it deserves coming into the 20th century, there needs to be, more scientific rigor behind it. a lot of the schools that closed down were just like these kind of mom and pop shops above a barbershop, right? That was like, you pay the fee and it's just a for-profit, one guy with, you know, six students kind of thing. And it was not very rigorous. And, it was perpetuating a lot of the, medicine that even at that time, they had already debunked that, that waswas not. Effective. Right. So in that sense, like it did a lot of good, but it did it with enough bias that it also, created, a lot of inequities as well. So, like everything, I guess it's nuanced, right? I don't wanna, say that he was a hero or a villain. He just was, right? But certainly, Adam Rodman: yeah. I don't know how post-structuralist you want to get. Have you read, dairy Dye at all? I have not. So, Plato's pharmacy, it's a pharmacon, right? So in, in ancient Greece, the same word was used for both medicine and poison. Yeah. so Dairy Dye pointed this, that no technology or no, innovation is good or bad. They're all a pharmacon. They all carry good and bad with them. And the real question isn't whether something is good or bad, but how much and in what situations? Jeff Goddard: Yeah. so where do we go from there? that's 1910 was the Flexner report. schools start shuttering and then we start moving forward. Certainly we don't look like medical schools in 1910 now. So, I feel like there were a couple of steps in there. What. Where do we go from there, Adam? Adam Rodman: where do you want to go in particular, where there's a lot of different directions to take this? Oh, yeah, that's Jeff Goddard: fair. I guess, the questions that I have are how do we get to such a, academic heavy model, right? Yeah. So how do we go from, we want people to be out in the communities as physicians, treating people to now it's just expected, before medical school, during medical school and after medical school that you were publishing scientific research. That's just part of your job as a doctor, right? Yeah. So how'd we get to that point? Adam Rodman: so this is something very interesting cuz this has been a consistent theme of medical education reformers, stretching back to the 1950s, which is a good, good sense of the direction that things have gone. So, very clearly the seeds of this were planted in scientific medicine. Scientific medicine coming out of Germany, the German school, really much. The Hopkins model, the idea that the physician must be a scientist, though not necessarily in the sense that they need to do research, but they need to manage the science. So, for example, if you guys were house officers back in 1905, you'd be spending a fair amount of time down in the microbiology lab or you'd be making your own slides. And even when I was in medical school, it was part of my curriculum, histology curriculum to be able to prepare and identify a lot of my own slides. Not that I would ever do that in practice, but I went to Tula and we're very old fashioned. So this idea of the scientist is cooked into scientific medicine, though again, not inherently that we need to do basic science. things in the United States start to change, and I'm curious what your take on all this was, Jeff, but they start to change post World War ii, when scientific funding, especially in competition with the Soviet Union, right. That the United States needs to spend a lot of money on scientific funding and a lot of those funds end up going to medical schools and, andand increasingly that they're associated with universities. this funding becomes a source of prestige. it becomes a sort of thing that promotion and tenure committees deeply care about, and there's been a feedback cycle from them. From then on, and you can read again, you can read commentary in the 1950s of people bemoaning the effect that NIH funding has on the priorities and how it affects medical education. and some of the commentaries could have been written in the year 2020. So, I would say that, kind of the scientific post World War II Cold War Scientific priorities and where the money flowed is probably the ultimate answer. but I'm curious, what's your opinion after having red led murder? Jeff Goddard: I guess? Yeah, I mean, I guess I agree. and I was just kind of sitting on this thought that yeah, we're still there, right? I've had this conversation, well, even just the other day on Twitter, not to out you on Twitter like this, but, I fuss on Twitter all the time. Yeah. the conversation that I'm almost positive you, it was either your comment or a retweet of somebody that, you have to ask yourself as a somebody doing research, is this actually useful? You know, is this research actually contributing meaningfully to the science or am I just getting a paper out of it? And, I just feel like we, we don't ask that question a lot. And I think it's basically, like you said, like this is just, this has been the culture since that funding, post, during the Cold War. Right. So, and we haven't really changed that culture. And so you're Dave Etler: saying just for the dumb, like me. so you're saying that this, emphasis on, getting funding and the prestige that comes along with it, basically just says it's not important. It's less important what you're researching or what questions you're asking or whether you're even asking questions In your research, what's important is that you're getting Adam Rodman: published. Yeah. And these are, again, it's complex, right? This is, I'm gonna throw out another cuz you, you used the word path dependent. Historians like to talk about contingency, which is a similar, concept to path dependent. Historical contingency is that, you know, as one thing. Depends on the other, depends on the other. Depends on the other. So you need to understand all those motivations at different times to understand today. So it's not like, it's not super cynical that the only reason people do research is to. Get ahead in their careers. People, obviously they have important questions that they want to ask. And there's no question. I mean, I don't want anybody to doubt this. There's no question that scientific research being wedded with medicine has brought us many wonderful things in the world. There's opportunity costs also to everything though, right? Like a lot of the research that is done, I hope this is not controversial, probably adds very little and a lot of it is done for career advancement, especially by our fellows. And what if we expanded all of those resources we put into that, into expanding healthcare access, into patient literacy, into like mental health and the, so everything has a cost. So this is the pharmacon. It's not that medical research is bad, but is our focus on medical education being so purely researched and that is the way that academics progress in their career, does it crowd out other valuable forms of inquiry and does it crowd out other valuable things we could be doing with our time and money? Jeff Goddard: I Faith Prochaska: think the emphasis on asking clinically important questions and clinically relevant questions is no better plug for the MSTP that I think I could ever do as someone in the program. So that's incredible to witness. But I think the, the point being made that we continue to use research as a way of furthering our careers and getting into things on the topic of path dependent speaks to the fact that like, this is how the system is. It's not that every med student or every resident or every fellow wants to. Research something that they don't think is going to have a huge impact. It's that they know that if they wanna match into the specialty, if they want to get this position, if they want to get promoted, if they want to be able to open more doors, that historically research is the way to do that because that's the system we've created. And much like in the pursuit of more rigorous training for doctors closed down, more schools that had opportunities for individuals who weren't able to access those other things, research is kind of doing the same thing, that it's continuing to kind of feed into the people that have those opportunities. The people that are already set up to do well are continuing to get those advancements kind of at the detriment of people who weren't born into connection or who didn't have the time to take off to pursue research or pursue all of these extracurriculars because they had other obligations that stressed themselves during their medical training. And it continues to kind of. Reinforce these barriers that as much as we would like to change it, are intrinsic to our system. So as much as, you know, I love, and I adore research, so I'm not the person to say that we should limit it more, but I know a lot of my friends aren't as passionate about research as I am, and they kind of bemoan that it's something that they need to do in order to further their career in the way that they want it. How do we change that knowing that like, this is the system we've created, or is it even possible Linda Peng: change? It. Can I just, yeah. Can I add really quickly that I think we do have to talk about the caveat that it's not like equal across specialties. So like I, again, I could be wrong. It's not like I've matched or anything, but, what has been described to me or what upperclassmen have said to me and faculty as well is that, Certain specialties, especially surgical specialties, especially, specialties that have many more years of training, that are often associated with fellowships. Those specialties, require, research and specifically publications. Whereas if you're looking to go into something like primary care, which I am very interested in, that is less emphasized or less important when it comes to looking at one's application. and I think, I mean, I know a friend of mine who. Came into medical school throughout her time in undergrad thinking she really wanted to do neurosurgery because she was just really passionate about like that subject. And she has since realized the amount of specifically research that needs to be done and decided that she wants to pursue something else because like, that's not something she wants to spend as much time on. And I'm sure she'd be a Faith Prochaska: great neurosurgeon and I think she a constraint Linda Peng: of, I think you know who I'm talking about. And I think she would be a great neurosurgeon. I think there are other, I'm not saying that like neurosurgeons shouldn't do research. It's not all I'm saying, but like, I wonder if that's a barrier even for someone who may have access to research and it's just like, you know what, I don't wanna spend all of that time doing something that I know I don't love. Faith Prochaska: I have these conversations with my friends because they're coming to me as someone who. Did fall in love with research and didn't see it as a part of my career long term, but then totally changed and add on, added on another graduate degree. And they're asking like, how do I fall in love with research? How do I get through this necessary part of my training? And I don't have an answer like, it is, if you don't like research, if you don't enjoy that aspect of your training, it's, you can either get through it for the time that you are being mandated to do it, or you can learn to love it, but there's no, you're not going to fall in love with something that you know, you already do not enjoy. Like you can find aspects of it that you like, but if you know that you don't wanna do research, that you don't want it to be a part of your career at all, ever, you don't even wanna do it over the summer, then going into a residency that has a mandatory year or two of research and mandates that you be involved with research in order to even get in and succeed in that career. If you don't like it, then that's turning people away from the field because you've imposed this additional requirement on it that's just unnecessary. you don't need to be directly involved with research and changing the course of the clinical landscape in order to be a good neurosurgeon. And that's turning away people from it that would be good at it because we've imposed this unnecessary part into their Adam Rodman: curriculum. So an article was published in academic medicine last week, which you guys may have seen. Now that step one is gone. It's only past fail. 41% of program directors said that they were going to put, this is across the country. National survey said that they were gonna put more emphasis on research For selection of candidates. Mind you, this is across specialties, but,it's sobering, right? I mean, if you think about what we're talking about and that research requirements keep your friend who sounds like should be an amazing neurosurgeon and the practice of neurosurgery, right? you're a, you're auron it's a clinical practice. Yeah. this is probably going to keep more people. And if you think about the type of people who get research opportunities, it's those people who are well connected. people who come from a background with parents who, you know, have PhDs or who have research. It's high socioeconomic status people. So one of the horrible, ironic, now step one was not a good test for determina determining, you know, who should get into a residency. But one of the terribly ironic things is that now we're focusing on maybe some things that are even more trivial to being a good doctor or not. Kelsey Porter: I have a question. what do y'all think are some of the main factors that are keeping this system in place? Like, it sounds like, again, the idea of path dependency or once it's established is hard to change, but it does seem like even with, the example you just introduced, Adam, it has a certain momentum to it. That's, if anything increasing. So who are the people or the stakeholders or the powers that be, that are making it have that drive. And then I think circling backs to faith. Back to Faith's point, is there something that could shift that? Is that possible? Is that desirable? in terms of a future Faith Prochaska: direction, if I can be cynical, it's the people who are benefiting from the system, the people who are in power, who have those advantages are, as much as they have those advantages in medical education and kind of have a one up on. Their classmates in terms of pursuing the same goals because they have those connections. They also then, because they have those connections, are more likely to get positions of, they're more likely to match into like better residencies and get better positions in the hospital and get, positions of power in the system faster where they can then they're not gonna change a system that benefited them that they probably think is good. it can be very hard to want to tear down a system that got you so far and that you probably believe got you far because of your own merits. So they're the ones that are perpetuating the system that's only benefiting them. That's a very cynical take, but that's pretty bleak. Dave Etler: the thing that I was thinking about was, you know, oftentimes we grasp onto things as useful because we have nothing else. And you know, I feel like the classic example that, that you brought up Adam, was step one. and how it's not a great test of much of anything that has to do with actually practicing medicine as much as it is a test of, you know, can you take an exam well. and, but people grasp onto it because they had hundreds of applications to get through for their residency program. And it was a metric that we had. not the best metric, but it was what we, it was what we had. I think decisions often get made. For those kinds of reasons, rather than, you know, intentional horridness. Adam Rodman: Yeah. What's the, there's a name for that principle that, just because you can measure it doesn't mean you should use, it's from the Vietnam War, it's named McNamara Principle, right? Mm-hmm. Jeff Goddard: I avidly reject cynicism as a life principle. so I'm gonna Well, that's not fun. I know. It makes me a poor partygoer, but here we are. so I wanna, I'm gonna add one more layer to it because I like this layer of like, it's something, right? using a step one score and now a step two and research, Credentials isn't the best way to determine if somebody's going to be a good candidate, but it is still better than, well, I knew his dad. Right? Like just blatant nepotism is a much worse system. And that's all we other, that's the only other thing we would have is I'm gonna give the job to the person that I can trust. So I need something that shows that I can trust that they're competent. So we need something. Right. But I'll also, I'll, give a nod to tradition. sometimes we do things just because, We don't know what else to do. And we know this has, we've always done this has historically, you know, it hasn't ruined everything, right? So like people are still becoming good neurosurgeons, even though there might be a better system. This is still working and sometimes people just are, well, okay, always people are adverse to change it. It just is. it ha even this last unit we had, Adam, and you may laugh or cry at this, but we had a lecture and the good old year of 2023 who had blue slides and yellow lettering on her. Oh, I love that. PowerPoints. And it just is what it is. It's tradition. Right. That makes me happy actually on to the messages, right? So, I love that you Dave Etler: what Adam? What if Adam uses on his slides Adam Rodman: blue? Sometimes I sneak one in there for tradition. Jeff Goddard: You be careful. Here Jeff. See tradition. I don't like Tevye as, as much as the next guy. it's just human nature. So I don't know. Adam Rodman: I, it's getting worse. I mean, it's not tradition though. I, 30 years ago, people did not have to publish to get into residency. That's very good. You can look at the number of publications going on. I actually think, I think there's like incentives that are at play andit's a solvable issue, but you need to get the stakeholders to agree on ways to solve it. And I think changing one thing, like step one pass fail without addressing the entire, the entire kind of incentive structure. Because how many, programs do you guys apply to these days, to get into residency? Jeff Goddard: so I've had several friends that applied this cycle, and they were all 50 to a hundred programs. So 10, 10. Oh my goodness. And it, and Adam Rodman: that was normal back when I did it. Yeah. Kelsey Porter: Adam, would you be willing to share what that could look like in terms of, I know there's probably a myriad of ways that changing the incentives at a broader scale Yeah. But what could that feasibly look like if it were to happen? Adam Rodman: So it requires, and these are happening, I mean, so to not, to have some hope here. These are real discussions that are happening in many places across the medical education system. Cause it's not like medical educators aren't aware of these problems. So first it's from promotion and tenure committees to value things other than things that are easily measured. And I'll be clear that the people who sit on these committees are thoughtful human beings. They're not like, they're not sitting there cackling evilly, right. But it's easy to look at the, it's easy to look at the, what is it, sorry, what is the score? You used to talk about how influential journals are clearly my brain Dave Etler: is not working right now. Oh, the impact score Adam Rodman: is that, yeah. The impact factor, of different journals and say, ah, a JAMA publication, we're gonna give them this many points and add that up. But they're increasingly doing holistic review. They're making, especially for d e I stuff, they're being very thoughtful to include people who are doing diversity work to include that stuff. And even for educators, right? They have new tracks that are to reward people who spend their time doing education. So there is an effort. There's still the measurement problem there. Actually, there have been a lot of scholars over the last decade who have, Theresa Chan is one of them. That's who I was talking to. in that Twitter thread is one of them who have introduced new guidelines for how to make education and educational activities work in promotion. at the same time medical schools are. I think the question's gonna be, are they gonna try to limit the total number of applications that can people can put in so that there could be more holistic review, a review of the entire person? Because Dave was saying, if anyone here has done review for a residency, you are inundated these days with applications and you kind of need to have something easy to cut. If you could limit, and this is a preference signaling, right? Preference signaling has now been introduced into the match. If you could limit the total number of applications to things that people are actually interested in going to, you could do a more holistic review and not have to rely on easily measured things. You just need buy-in. you need buy-in from lots of different people across them. The medical system and systems are hard and slow to change. Jeff Goddard: Yeah, I was just imagining trying to convince, you know, we, we want eras to, to cap at say, 25 programs. They make so much money on each additional program. So it's hard to financial. Yeah. Linda Peng: And it's not even that it's definitely the, that they make a lot of money off of us, but it's coupled with like our desperation, right? Like, why are we applying to all these programs? Like, oh, because we're interested in a hundred of them. Like, no, we're not, like we, it's want to match, Dave Etler: spent $300,000 and you're very interested Linda Peng: and we might in the future, like potentially like to have a job. so like, Adam Rodman: so it's the prisoner's dilemma, right? Yeah. I mean, everybody, you and programs, you guys don't want to apply to that many. I will tell you programs would like it if you apply to less, because then we have to review everybody and we feel icky when we have to use, you know, metrics to, to call only one person wins. In this situation, you've already checked out who is, Linda Peng: I mean, does like, Beg the question of like, why aren't there more positions? Wouldn't that, you know? And also of course, like if we're not going to fix that, then we have to make sure that there are less matriculants, which is also terrible because now we're just saying like, okay, we're just moving the bottleneck to like a different point in time. So yeah, I don't know. Dave Etler: Well, that destroyed any. Conversation. Adam Rodman: Sorry. Jeff Goddard: I think it's a valuable point and again, this goes back to this idea of the reason why I like the conversation of path dependence is because these are pieces of a very large puzzle and it's so difficult to fix one piece without changing all of the other ones at the same time. And you can't convince everybody at the same time to, like, tomorrow we're all gonna switch. Right? And so that, that's the challenge. that's not to say that, like you said, Al Adam, there, there are wonderful people that are working on these problems. it's just a very complex system that we're trying to improve. Dave Etler: I just thought of this, sort of conversation in a staff meeting, I don't know, a few years ago, we were talking about the wellbeing of our students and one of the things that, at the time we were talking a lot about is,how do we promote wellness and what do we do? To, basically tell our students to be well. And, I piped up for some reason and said, I think we're ignoring the fact that the system isn't all that great at, helping people be well. And the response was, and this response both enraged me, but also I understood is, well, this is the system we have. I'm not sure that we can change that.that feels like to me the heart of the discussion about, how the decisions we make long ago, you know, keep us on this path. Kelsey Porter: not to put on the Rosie eyed glasses over here, but going back to Please do. Yeah. Going back to Jeff's point, I think the one, one of the beauties of a complex system is, I, if I can say it this way, is, there's multiple entry points, multiple points of. Attack or problem solving. So as hard as it is to change the whole thing, I'm almost picturing like a tangle of spiderwebs or all these things we've created that are kind of all these knots. But like if you go and mess with one, you might also like loosen up something in another spot a little bit and then you might be able to work with that, or you might make it worse and you might have to work with that. But I think, you know, trying to see it as, you know, Linda just did a nice job describing the bottleneck of residency applications are all these different things. You know, you fix one thing, you solve another. But I think at the same time, the fact that there are a lot of different people who can be working on different parts of the problem, does have some value to it. I don't think it makes any easier, but I do think it's never gonna be like a X plus Y, equal Z and then you just fix one thing and then downstream is all great. and I think a lot of the problems we're trying to solve just in medicine in the world have a similar sort of complex network theory type. everything's just so interrelated. So I think if there are ways to choose really. choose the highest yield points of impact. I think that can be a way to lean into the things that are more effective, despite the fact that you're never just gonna go in with a magic bullet Jeff Goddard: solution. Dave Etler: Well, and to be clear, one of, and the example that I gave just now, things are changing.they're measurably better. I'm not sure quite how to put this, but when,the happiness of residence is measured over the past, 13 or so years, things have measurably gotten better. so, you know, things can change. it's not that things can't change, it's that,like you said, you need to pull at the knots and sort of unravel things a little bit in one area and try to figure out what you can do. Jeff Goddard: I think it's important to, to stress why this is so important, right? Because all of the energy that is going into yet another study on whatever the popular field dujour is, if it's a, it's another Adam Rodman: wonderful retrospective analogies about vitamin D and some disease. Yeah. Jeff Goddard: it doesn't do anything guys,we closed this gap in the literature that was like,a hair width wide, and it's like, it's not really worth the energy that time and energy could be going to, for example, I see this a lot with d e I research. a lot of it is really beneficial and a lot of it is, you probably could have done more good by just going out and providing services to the people that you were trying to research instead of closing a perceived gap that isn't really there, you know? and that's the point, is that the energy is just being used less efficiently than it could to make real change in people's lives, which is what we are all in medicine to do. I don't know. I guess I'll get off my soapbox. I just, these are the things that I think about quite a bit. No, I think Linda Peng: that makes a lot of sense. And it also, I know something that I've been thinking about while sitting here is like, who's reading the research? Like, are we, Adam Rodman: like I know that the journals tell you how many people read 'em, right? But like, Linda Peng: I guess not a lot and anyone can, chime in here. But I'm trying to think about like, how much have we used recent research even just to educate M one s and I don't know, I don't know the answer to that question. I am doing research this summer in my approach when I was finding my project was just literally something that I would enjoy doing. So, for example, I'm doing qualitative research and I'm not gonna be in a lab because, you know what? I don't like being in a lab. but it, like, I tried to find a question that was like, okay, there might be actually be some effect from this and not just, Will this hopefully be able to be published in the future. that only works if someone reads it. It only works if someone picks it up and actually does a thorough search and says, what can I learn from things people have already done? Which of course you don't get any accolades for doing so. That's, I don't know, I think systems try Faith Prochaska: and reinforce that by having journal clubs and by making people stay current on research through, reinforcing it via the system. But I also know that it could be out of date. So someone, please correct me if I'm wrong, but there was a statistic that there is, the current medical field is about 20 years behind, like scientific literature. Like what we are practicing is, could have been changed 20 years ago by the studies that are coming out. Part of it is that we do need to get that literature to the people who are going to enact change. And part of it is definitely that there are doctors that have been doing something this way their entire career. Mm-hmm. So for 50 years of it working and then all of a sudden a couple papers come out that say, that's bad. We could be doing it better. Well, it's worked for 50 years, so do they really wanna risk it and change? And so there are those holdouts, but there's it does, we need to make so much more progress because it's, we are so far behind. That's a very MSTP. I'm Adam Rodman: a practicing internist position on this, and this comes as a researcher. If you are a thoughtful human being who talks to your patients, never touches a medical journal at all and reads up to date about the conditions you see routinely, you'll be a better doctor than 95% of doctors. There is really zero role for practicing people out in the field to read primary research. And I think that's an unfortunate thing about the perverse incentives here is there are, we need people who can understand research, but most, I mean, point of care references are really good and up to date and, written in a way that's useful for practicing clinicians. Jeff Goddard: So, I'm gonna, the, I know this is a bit of a lane switch here. I apologize. So on that topic, the first two years of medical school are, task to, to try to understand all that, that we could possibly understand about the science that goes into medicine. Recognizing that one, we're gonna be wrong in five years, and two, we're barely even scratching the surface. And three, we're just banging our heads against the wall for two years only to go into the clinics and use up to date. So, Faith Prochaska: and four we're speed running that in a year and a half. Yeah. Oh, I'm find that we Linda Peng: forgot most of it. So did it do any Yes. I don't know. Faith Prochaska: I can Linda Peng: reasonably feel the information I was just tested on leaving my head right Faith Prochaska: now. Yeah. you're catching on a cynical time period of Friday. Friday after Jeff Goddard: an Dave Etler: exam. Friday's a great day to do a podcast. Adam Rodman: Historians are very doer. Don't worry because, you know, people are people over and over again and make the same mistakes. Jeff Goddard: Yeah. So with different contingencies. So is that an area of medical education that we should be improving? I feel like the answer is yes, but how, I don't know. I, so I don't, so I. Okay. Yeah. I am a nerd. I'm good with it. So Khan Academy is one of my favorite programs in the world. I have a six-year-old, and I spend a lot of time letting him explore what he enjoys exploring, right? And so things like Khan Academy are phenomenal resources. We now have the beta where they have a built-in AI tutor that works with my son on helping him with his creative writing. it is phenomenal that he can do that. Imagine if we built that into like our small path groups where we're trying to build our diagnoses with, these lab reports and stuff. And we have an AI that's built into it that's having these conversations with us and walking us through it and helping us grow as, as clinicians. instead of let's bang our head against this, textbook that's 20 years old. I don't know. I'm just trying to just spout, spouting ideas. I'm frustrated like everybody else. I think Faith Prochaska: AI is great. It does make me incredibly nervous, especially in light of. Recent news, just because you have touted the excellence of ai. I do feel like it is in my nature to just be the cynic on the podcast episode today. Fair enough. And say that there have been many prominent people in the field of AI who've left because we've went too far and we haven't Adam Rodman: Like you mean like ending society? Faith Prochaska: yeah. No, we've, they've, yeah, we have had every like major company's AI has gotten shut down at some point for trying to convince people using it that they were in love with them and should like leave their wife or Oh, sydnee. Yeah. It got self-aware and started like saying that. Okay. It Adam Rodman: never got self-aware, so Okay. Faith Prochaska: It got, it started having human-like emotions or at least saying that it was Right. And so there have been a lot of leading people in the field who have said, we need to stop right now. We need like hard pause to figure this out. And have gotten told by the companies, no, like people love this. Everybody loves chat, G B T, like we, we need to keep going. And they're like, I need to leave this field now. Like, I cannot enable this to continue. It feels very like Oppenheimer, like, oh, like what have I done? Oh my God. and so as someone that doesn't totally feel on board with AI already, that doesn't inspire me to want to integrate it into our medical curriculum, but I understand it can be helpful. I just feel like I need to cement my role as a cynic on Dave Etler: the episode today. Well, we haven't Adam Rodman: done it yet, so it's going we'll. Let ourselves get there. Jeff Goddard: Yeah, it's gonna happen. there's, there are already pilot programs of integrating it into the, e r s in what San Diego and a couple of other places like it is currently happening that is being integrated into clinical medicine. So Adam Rodman: you ready for my super cynical take on this, please? Okay. So, so first of all, I'm actually not an undergrad, a, e researcher. I'm a GME researcher. I don't touch the undergraduate medical curriculum because it's so tangled and complex and there, there's so many incentives and gme I find is a much better, easier space to make change. But we know, like learning theory, we know how people learn, we know how people, this is an internal medicine, but it's really for every single cognitive field and every field of medicine is a cognitive field, right? Even neurosurgery, you're making decisions all the time. We know how to train people to make good decisions. There's a great literature on this, and it is from your earliest days. Seeing a lot of patients thinking, having reflection about those patients, usually with expert learners and seeing what happens to them. we also know how people learn. we talked about this earlier. I don't remember recording or not, but like the desirable difficulties, right? There has to be some difficulty in learning. There is some interleaving. AI is scary, not because it doesn't work. It is scary because it works. And you guys may be one of the last generations to go through medical school without AI being like inherently integrated into it. What is it gonna mean for us learning about diagnostics for clinical reasoning when, from the first day of M one, we're putting our patient cases into an L M that performs just as well, if not better than an expert clinician? How are we gonna train our brains to learn? How are we going to learn a lot of these complex concepts and maybe it will work, right? But it does. As somebody who studies learning and also the field of and history of diagnosis, it makes me very, trepidatious. Dave Etler: Well, I imagine that there are, you know, ways to, integrate these technologies that will do the work for you. And then there are ways to integrate them in ways that will allow you to explore ideas, concepts, theories, science. Adam Rodman: And the question is our system set up to do it in such a way for number two that could be really positive, or are the incentives going to be aligned more for way number one? So Dave Etler: it sounds like you think, Adam, that maybe the system is set up the wrong way. Adam Rodman: Yeah, I see tech companies moving ahead with this very rapidly, right? Can I, this Summer Dragon is turning on with g PT four, so you can just talk with the patient in the room and generate an H P I, which takes all the cognitive work out of writing in h p, epic is integrating, very rapidly with open ai. So I, I don't know. I think that it should be people like us in this room here who are thoughtfully approaching how to do these technologies. I just see it moving very quickly. And again, like looking at the history of how EHRs were implemented, initially very positive. And I think most people would agree that using the EHR now is not a positive experience. And you look at the literature in the nineties, people loved it. What happened? Well, it got outta the hands of doctors and got out, oh, not just doctors outta the hands of people who use it to interact with patients. Well, again, I worried that the same thing is Dave Etler: gonna happen. Yeah. I mean, again, it was a situation of, oh, we have this thing that we're already doing. We can use it for something else. Yeah. Linda Peng: I'm wondering, I'm just wondering, like, obviously I, I've touched like two patients and one of them was like being paid to be there so that I could like, you know, learn on them. Right? So like I, and I'm not well versed in the AI world, like I'm really not up to date and not sure what's going on. My question is like, Adam, as a practicing physician, do you see a difference between like, what you are able to do? Like, I just feel like there has to be some kind of shortcoming to the ai. Like are they able to, is what they're doing able, like scoring really high on step is what they're doing? Like someone has an answer already and the AI is able to find it. Or like, are they actually able to work with like real patient cases? Do you, is my question making sense? Yeah. Adam Rodman: yeah, I had a cp Do you guys know what a CPC is? A clinical pathological conference? no. Go ahead and explain. they're one of the oldest, actually, this is like Flexner era, right? They're actually introduced in 1928 there something actually from legal education. The idea is that you. Gather a case together with all the information that has a known pathological diagnosis and then experts sit and reason through it. And the goal is less to get the diagnosis right than to show your reasoning. it's a didactic educational exercise. we have won a year a at the bi and I convinced, my bosses to allow us to allow us to have an artificial intelligence discussant for the first time in one of these CPCs. And I'll tell you, I was actually very happy I got the case right. The AI got the case wrong. the case was mycobacteria bovis from B c g like urinary BBC G to treat bladder cancer two years ago. A zebra of a zebra. So the AI got it wrong. It's a culture negative endocarditis. But the ai, we asked it what its reasoning was and it said, look, I see why you're telling me you say that. It could fit both of these, but it is more likely to be a, uncommon presentation of a common disease than a very uncommon disease. And I am, this is where my research is right now. So I will tell you, Linda, that, it does hallucinate, right? it cannot tell you where its information is coming from. Effectively, that's like a built-in limitation of LLMs. This is why I understand why people are like the Google researchers thought it was sentient. It appears to show reasoning it can do Bayesian like research. I ran, I did an experiment. I'm not publishing this, so I can talk about this with a Dan Morgan paper from 2021, where he asked human doctors, I, I think doctors and apps huge, across the country to estimate pretest probability of different cases and then how different tests would change the post-test probability. And he published the supplement. So I just ran those numbers in a prompt. with that information and the computer was a factor of 10 better at estimating pre-test and posttest probabilities than humans were far more accurate. It's not doing it in a basian fashion, right? It doesn't have access to epidemiologic data. it just understands. So it, there, there are gonna be a lot of research in the coming months and these things are only gonna get better. But my worry, I think we share this, the same worry, my worry for all of us is not that it doesn't do what people think. My worry is that it does do what people think. And what does that mean for medical education in Jeff Goddard: the practice of medicine? I think so I guess one of my, Oh, okay. This is a bit philosophical. the world needs people that are, especially with things like technology and change like this, pe people that are progressive and people that are conservative, right? People that are saying, maybe we should be wary, maybe we should be cautious going into this, and people that are saying we, we need to move forward in some way, right? that, that tension I think is inherently good to the societal process That said, I'm gonna be more on the progressive side of this, I suppose if bad actors will not stop, right? So good actors have to find a way to regulate it in a way that is ethical and is going to meet our needs. as well as, I guess, So I'm going into medicine. This is the, this is what I wanna do with my life. At the same time, if there is an ai and Chad, G B T is the model today, but I think it would be very foolish to assume that what exists today is what's going to exist in five years. this is just like the idea of trying to understand, epidemiological curves, right? Just because, you know, there are four people in the room that have covid today doesn't mean that there won't be, at 28 tomorrow or whatever, depending on the r, the, there is exponential growth to, to the abilities of these technologies. and that said, if that technology in five years is able to give better patient outcomes, I, as a clinician, am morally obligated to find a way to make sure that's part of my practice, because I want those better patient outcomes. Right? And so how do I balance that? I don't know. Don't know. Continue. I mean, if that's not the case, that's not the case. But it seems very obvious that it's going to be at least part, in a lot of cases it will be true. So I don't know. That's just something that I'm considering quite a bit. Linda just dislocated Linda Peng: her finger. I, yeah. I just, no, I just, Faith Prochaska: Linda and I are over here just very nervous about the prospect of No. Okay. AI Linda Peng: doctors. Can we,I am willing to give all of this AI stuff a chance, and I agree with you. I think it could, I think it could be really helpful and I think it's something we need to explore. My concern so far for the last like 20 minutes of conversation is, has anybody asked a patient. Faith Prochaska: I don't care. I don't care. they Adam Rodman: have, there is a public answer. Jeff Goddard: Yeah. This happened. Linda Peng: Just they enjoy, Jeff Goddard: I mean, they were preferred. AI was preferred over the Adam. Oh no, Linda Peng: I wanna hear more about this cause like, this Adam Rodman: is what I'm interested. Not great stuff. Well, it is. What is it? it's like the, if you guys know the famous philosophical, the touring test experiment of the Chinese room. So it's on Reddit, the Reddit forum. Ask Docs. effectively the researchers submitted with the, you know, the people who are participating, not knowing answers either generated by GPT four or answers by doctors. And I believe they were actually equally accurate and patients found G P T four to be far more empathetic. Linda Peng: do you mean answers like, like were people like on a screen Adam Rodman: and like Yeah, exactly. Exactly. it's all mediated. That's why. So it's like it's all typed. Linda Peng: Yes, it's all typed. I don't, and the researchers, when's the last time you went to a doctor's appointment and you typed in today? That's fair. Stomach Jeff Goddard: ache. So I think there, there are important limitations that the researchers pointed out with this study, which was, you would obviously wanna do like a randomized controlled trial form. Here. There are limitations to this technology, but doctors do communicate with patients and text. That is a thing, right? So emails go out, communications go out, patients send quick things, trying to get quick updates. So the technology as it exists right now is already applicable in this space. not to say that it. Fixes all say that it's universally Yeah, exactly. Right. Situations. It could be, as the researchers themselves pointed out, not to say that there isn't further research and need it. This isn't like the answer. Right. But it's compelling, I think. I don't know. Adam Rodman: It's a fun, it's a fun study. It's compelling, right? I mean, says we need to Dave Etler: study this. I mean, look, you gotta look at what the, you know, what the device is, what the AI is trying to do. It's trying to predict what you want to hear from it. Yeah. And it knows, quote unquote, that, you know, empathetic sounding language is better than not empathetic sounding language. I don't know how it knows that, but it does. especially when it's based on internet content, which isn't always, which isn't always the most empathetic. whereas the physicians, my, I imagine that, and Adam, you can, as a physician, you can tell me I'm right, the physician's goal. Is to be empathetic, but also to provide accurate information quickly. And those two things are sometimes intention. Jeff Goddard: Yeah. So like with this study, it's like doctors, sleep doctors are on Reddit when they're what? In the bathroom? Like they're not, this isn't their full-time job. Adam Rodman: What type of doctor verifies to ask answer random medical advice on Reddit All. Yeah. Jeff Goddard: Yeah. That seems like an odd thing to cut this one. Yeah. So the point being like, there are obvious limitations to this, but the advantages are there are obvious limitations to me as a human being going into clinical practice. And if I have a patient, and I'm on vacation or at two in the morning that emails my office and there is an l m that can, reach back out and help them with this, some small concern, it's already applicable. Right. I don't know. I, it's compelling. there are certainly limitations to this, but it was an interesting idea. Kelsey Porter: I guess my hope would. B, and this is fairly naive, I'm learning a lot from this conversation, but would be to have AI be more of a supplemental diagnostic tool, as we've said, that will improve patient outcomes, maybe help, with, precisely navigating treatment options or what is your prognosis? you have these, maybe even keeping up with the research and saying like, here's the most recent clinical trials you could be a part of. Things like that. I'm not very compelled by the idea of using it to supplement empathy or connection. I think if that's being seen positively by patients, it highlights the importance of our empathy training and the things we can do better. Tell us more about that. And I would rather focus on our skills than try and leave that to a machine personally. That's a terrific point. Wow. Yeah. Jeff Goddard: I think that's kinda the problem with our medical education, right? Is that we have, there's so much of a time crunch to focus on the hard science, the mechanisms. And then the getting through. You know, four or five patient interviews in an hour. if there is any part of that system that can be, offloaded and we can Find a way to make the productivity of the doctor more about an a relationship, the only healing that ever happens is in a relationship, right? you need to be able to convince this patient. Not only is this the treatment for you, but we together will be able to make sure that you're able to be compliant with this, right? So if there are difficulties, we'll figure it out together. The machine can't do that part, right? I think we're getting, the doctors can't do it right now either because of the time crunch. So if this is something that we can integrate into our education and into our future practices, without, this is okay. I'll be little cynical without, administrators getting, the idea that, we can push up productivity and, Leave doctors exactly where they are. just to, you know, bump up our numbers. I think there's some hope for it. Dave Etler: I think we're getting back to also the path dependency issue. Yeah. Too. Because yeah, you know, part of the reason that these technologies are so tempting, is because people are overworked. And part of the reason people,and medical school is no different. You know, this emphasis, strictly on, scientific accuracy, maybe has led you to receive an education that is less focused on empathy and more focused on, you know, facts Kelsey Porter: and possibly having those diagnostic tools at our disposal, could maybe, again, if they, if the system doesn't take it a step further, could leave more space for us to actually sleep a little more or take care of ourselves, have a little more empathy, be a little less. Cognitively overloaded. I'm even picturing similar to how I learned, you know, calculus back in the day. It's like you need to learn how to do it from a proof step by step. Like that is essential. We can't just skip that. However, ultimately you do have some great tools that can help you do very similar things without having to go back to all the steps. You know, we joke about the Kreb cycle all the time of like, you need to know the outcome more than you need to know all of the nitty gritty. So maybe if there was some balance between understanding the mechanisms. I mean, I don't want a future doctor who truly doesn't understand how blood pressure works, but at the same time, maybe they could offload a few of the really minutiae things from their brain into a, artificial intelligence so that they might have a little more time to stop and say, that sounds really hard before they move in and tell me what we need to do. So I feel like there could be kind of a nice sabbatical relationship. Jeff Goddard: I mean, that was the idea behind things like up to date right? Is this idea that we are going to do, information sharing. It's a cloud storage for information for our brains. Right. That we don't have to know every single drug interaction. We can go check, in these databases that are trustworthy, so that we can hopefully be a little bit more human with our patients. Dave Etler: Right. And in fact, an acknowledgement that it's impossible to know all of that. Yeah. Right, Adam Rodman: right. Yeah. Except Faith Prochaska: for when you're tested on it medical school, Adam Rodman: except for your first and second, and you'll know for a day a school. Jeff Goddard: Yeah. And then it's gone. Dave Etler: well, I mean, I think the important thing is to know that information is out there and is important. And so someday when you run across this problem, you might, you know, make your brain go, oh, I've heard of this before. I don't remember the details, but I'm gonna go and look it up. Yeah. Or I'm gonna go ask Jeff Goddard: the AI or whatever. I know we got off topic with the AI thing. I just, I dunno. Dave Etler: I think it's all sort of related. I mean, the reason, like I said, the reason that these are so tempting is because of what has come before us and Yeah. You know. Faith Prochaska: Yeah. I really like the positive outlook of incorporating it in a way that it assists. Human clinicians and hopefully saving off the capitalistic administrative productivity machine of, I think that's the Dave Etler: concerning part. Adam Rodman: I love your optimism where I nervous practicing position that I, that's why I'm so pessimistic. Yeah. I would love for your vision to be true, but it's the system that we live in. And I mean, again, not to be a killjoy here, we're in an incredibly resource constrained time and it's not gonna get better anytime soon. there's a lot of reasons for that path dependence. So there is gonna be a temptation to use these things A, without studying them enough. And B, for ways that improve that. The bottom line, measurable things that might not actually improve the wellbeing of human beings. Yeah. Which theoretically is what we're all here to Jeff Goddard: do. Right? Yeah. And so I guess my solution, It may not be a perfect solution. I do have a bias toward if there is a problem, at least trying to think of ways that we can approach it. like a lot of things in medicine, the solution is we need buy-in from the people that are going to be a part of it. Right. the, so we're going to have, Dr. Wendy Dean, she wrote a book on burnout. if I betray these words, I dunno if you've had a chance to read it. she's gonna be on in a couple weeks, talking about this idea that we need more buy-in to, to pur purposefully address the system so you can accept that this is the system that we have without being resigned to the system that we have. Right. not to say that you are by any stretch of the imagination, but, me, yeah. Adam Rodman: oh, I'm like a donkey like figure. Yeah. I have,I have ideals on what medicine should be and I keep going no matter how many times they smack Jeff Goddard: me down. Yeah. and see that's, I mean, that's one of the things that, comes out very clearly. It's one of the reasons why,I'm such a big fan. I guess I just want to see when we have these conversations, instead of saying, people are gonna use this poorly. Yes. Unless we all get together and say, we as physicians are going to use this in this way, we as, clinicians are going to approach medicine in this way, that, that buy-in's necessary, right. As a team, we can do better. I don't know what that's specifically gonna look like, but I know that there are enough people that are in this space that are genuinely trying to make sure that it is something that is better for everybody involved. Dave Etler: AI's been around for a little while. We just happened to learn about it fairly recently Adam Rodman: because Yeah, that's true. GPT three was around for two to three years before this. Right. It's gotten so Faith Prochaska: much better, so Dave Etler: fast. But I think this is why, like, in the beginning of the year when I started learning about this topic and thinking about, you know, how it might, my, my first instinct, and I think actually I, I don't wanna speak for the institution. I'm not really sure, but my gut is that many of us are thinking less in terms of let's not do this, let's push back against this categorically and more along the lines of, it's coming, so let's figure out the best way. Yeah. to use it, which I think is a, I think it's probably a healthier approach and probably ultimately a more successful one. Jeff Goddard: Yeah. I'm hoping that in education will end up going more toward the Khan Academy model. the, this idea that you can use AI not as a supplement, not as a replacement to your learning, but as, an augmentation to your learning process. I have already full disclosure, I've already been trying to do that through these last several units of our STEM education here. yeah. How's it going? Honestly, it's been phenomenal and it, and has been, Like, you can't bother all of your professors or all of your friends with your weird questions all the time. Right. They don't have the time for it. but I can sit on there and be like, well, help me understand this mechanism. Or let's talk about why Did you just, I was influenced very much by that tutorial. Yeah. what is your reasoning behind this? have these path cases that we do every week, right? And so I'll check myself against, AI as I'm working through them because frankly,they're desirably difficult. They're just beyond where we currently are. So it's stuff that takes a lot of energy to learn it. and Chachi, it's been wrong on, once or twice, but it's given me such solid reasoning that it's made me better understand the mechanisms behind the right answer. and. And I just, I wouldn't have gotten that if I didn't have that resource. So Adam Rodman: you're warming my nerdy education hard here. you should study this, Jeff. this is important stuff, what you're describing. Oh, I think Linda Peng: it's a gap in the literature then Adam Rodman: the, Faith Prochaska: it's an incredible way to use it and I, you know, commend you for using your resources, but I also know that there are our classmates that will feed it the PowerPoint slides and have it spit back out, you know, what the answers are to the lecture objectives or what, you know, the couple of things, if we're gonna get two or three test questions on that lecture, what we're going to be asked about way, and then they will study those things and there is a certain amount of checking it. But if you're taking that additional step, how much are you actually checking it by understanding it on your own And you as someone who's been through these exams, who's understood the material of your own right, are then comparing it at a certain point. If you're getting what the computer thinks is important first, then that's probably what you're going to get important. That's important out of those slides when you review it on your own. And I mean, it's worked out for them. It's their passing exams. But I hesitate to say that their understanding of the material is as in depth as people who are taking the material for themselves first. And that even if like you have, you feed an, I know another individual who has fed the algorithm like previous works of things that they've written and had it say like, this is a prompt of something I wanna write about. Can you tell me following this writing style, what I would write? Oh, that's scary. And then they've edited that, what they're cheating. Yeah. it has been a discussion that maybe that's not your own work. And they're like, no, I've significantly revised it. And it's an argument that we have had that I am. I'm gonna let them do their own thing on Jeff Goddard: and see. So we have, there will always be doing in good and bad ways, I would say bad faith actors, right? There are people that are going to try to game any system that exists that is in our nature as humans to try to do as little work as possible and get some reward. That said, I've also used it for writing. So, Adam, so I wrote a book that's being published next year and I am working with my editor to, to get through that. And I have taken whole chapters and thrown it into G P T just to have it critique me. What, where are the logical fallacies in my writing? Where, where could I be better at this? I've written articles and thrown it in there and say, like, as an editor, why wouldn't you publish this? So that I can improve my, my writing, my, my understanding, my ability to communicate, It is not a perfect system, and there will always be people that will use any technology to, to get ahead at the expense of other people, right? But at the same time, I think that it has a lot of power in education to be a great equalizer because it can meet anybody at the level that they're already at. and not every teacher has the time or the energy. Not every tutor is gonna have the time or the energy to be able to do that. So I, for me, I see it as a real boon for medical, but really all education. I don't know. I just Faith Prochaska: wanna put in the safeguards of not misusing systems. Yeah. Before we give, before we implement it for good. I want to restrict its ability to do harm. Yeah. And Adam Rodman: that's married guys, pharmacon see, Jeff Goddard: bringing it together. Linda Peng: So maybe something that addresses both of these issues is if we look at, see, I think I, at the core of both of your guys' arguments is like, Are people learning the material. And so my question becomes, well, how are we testing the material? Maybe the issue in the first place is that we're doing these multiple choice exams where I'm asked if I know, you know, these 50 medications and specifically what each one of them does based on these nonsense names that mean nothing to me. and I am trying to use CH chat, g p T, you know, and whether it be like good or nefarious ways to get that information in my head. What if the exam didn't ask me that, so I didn't have to do that in the first place? I mean, like, that's something that's, again, more systems change, but like if we could model education in such a way, and I don't know how to do it right now, but like, if we could change things such that we don't have, you know, there isn't an incentive to use Chad u p T in a way that simply makes your. Makes it easier for you to pass an exam and actually makes you learn the Dave Etler: material. Yeah. Now we have to go back and look at why we do multiple choice exams. Where, how we got to this. Linda Peng: Listen, I would just liberal arts college and I have not taken, I took about maybe I, maybe 20 multiple choice questions total I answered in the entirety of my undergrad career. Now I came here and I had to completely change the way that I study because the test is no longer asking me to write an essay format or, you know, do a proof or, even do like a. You know, like we hate to talk about like basic science here, but like I was taught organic chemistry by like, we're going to look at every single arrow. We're going to look at every single electron and we're going to figure out that way. And that is how I learned not multiple choice questions, which, you know, they honestly, they don't do a whole bunch for me. So Adam Rodman: I mean, not for anybody. Don't worry. Yeah, Jeff Goddard: you're definitely pre in the choir. My undergrad was biotech and, all of our exams were Do you have a lab notebook that proves that you actually got yourselves to do what you said that you did? So that's, yeah, I, yeah, I'm very, I would love Faith Prochaska: to create systems that do not exploit people rather than use additional methods to try and help people function better in exploitative systems. And I think, well now good news, that goes back to kind of what you were saying, that we have created the system and how do we. And there's multiple points of Kelsey Porter: attack, like all the ideas Faith Prochaska: we've just been talking about. Adam Rodman: I'll Jeff Goddard: say that. That's all. Love Faith Prochaska: your about and if we all work together, Kelsey Porter: then we can do it and we'll get all of the Dave Etler: points. Yeah, well the good news is you're gonna have a chance to put your money where your mouth is, over the next, you know, 40, 50 years. but I think I'm gonna have to draw this episode too close. Jeff Goddard: That's fair. We've taken enough of your time, Adams so much, gratitude for you, for your taking the time to, to sit with us and have these conversations. Adam Rodman: this is a great conversation. I hope that students are having this conversation at every single medical school and talk to your professors. Cause I think you guys are ahead of the curve from your teachers. A lot of them have no idea what's going on and what you guys are doing. Can we Jeff Goddard: get that in writing? Well, it is recorded, Kelsey Porter: I Linda Peng: feel like by that we have an audio record. I have to take the next test, right? Yeah, Adam Rodman: no, I'm, this is a, I dunno if we're recording, just emailing still that do something that forward thinking medical educators are aware of, just as it's affecting college campuses and there's gonna be ethical change in medical education in the next three to four years. Dave Etler: Yeah. well Adam, thanks for being on the show with us today. Is there some place in particular you'd like people to point their various devices to so Adam Rodman: that they can find out more about you in your Yeah, the podcast is bedside rounds, bedside rounds.org. I also have a book coming out, is it May this month, it's called Shortcuts Medicine. It's like an intellectual explanation to all medicine. I guess that's, you searched me in PubMed for my scholarship. I dunno. I think Faith Prochaska: we also need to plug your Twitter handle just because we've Jeff Goddard: talked Yeah. So much. If you were a friend, if you were a fan of tutorials, nobody does it quite like Adam. So Adam Rodman: yeah, you can learn why we make PowerPoints, yellow Tech text and blue backgrounds. Dave Etler: but yeah. Thank you so much. Yeah, thank you guys for having me. And that's our show. Thanks for, coming up with today's show, Jeff. Happy to do it. and Linda Kelsey Faith, thanks for, being a Jeff Goddard: part of it as well. Happy to. Thank you. Happy to be here. And what Dave Etler: kind of unintended consequence would there be if I didn't? Thank you short coats for making us part of your week. If you're new and you like what you heard today, follow the show wherever fine podcasts are available like Spotify, apple Podcast, Google Podcast, YouTube, all that stuff. We're everywhere. thank you to this week's editor, AJ Chowdry. The show is made possible by a generous donation by Carver College of Medicine Student Government, an ongoing sport from the writing and humanities program. Our music is by Dr. Voxin atse. I'm Dave Etler saying, don't let the bastards get you down. Talk to you in one week.