Latest / Oxide and Friends / AI, Materials, and Fraud with Ben Shindel

Transcript

AI, Materials, and Fraud with Ben Shindel · Oxide and Friends

• 0:00
  Adam Leventhal: Is our guest here? Yes. Ben. I see you're listening here.
• 0:02
  Bryan Cantrill: Yeah. I think I think I think I find Ben up. There we go. Henry,
• 0:06
  Ben Shindel: are you here? Yes. How's it going? Although when I accepted it, it just changed the audio output. Can you hear me?
• 0:12
  Is my audio good? Alright.
• 0:15
  Bryan Cantrill: The audio's good to me, but I don't know. Like, mister judgmental over there has got apparently, has got issues with everybody. How does how does Ben sound? Sounds okay?
• 0:23
  Adam Leventhal: Ben sounds great. Ben in comparison to some people.
• 0:28
  Bryan Cantrill: Okay. Know, we're staggering ourselves already. Ben, thank you so much for joining us. I was listening, Adam, to our episode from last week with the the chime. We proposed ringing chime whenever we refer to previous episodes.
• 0:43
  And, you know, you hit that one as a bit of an Easter egg. You didn't actually talk about the fact you've done that.
• 0:47
  Adam Leventhal: I I did. You know what? Here here's the thing. When I have to go edit, like, an hour and a half plus episode, I'm gonna sneak little things in here and there. So, you know, stay on your toes, Brian.
• 0:58
  Bryan Cantrill: It's too provocative. Absolutely. I thought it was a test of the emergency broadcasting system. I hand on heart. And this is surely something that has not left the generational chasm, the emergency broadcasting system.
• 1:11
  Ben, do you have any idea what we're talking about?
• 1:14
  Ben Shindel: I don't think I do.
• 1:16
  Bryan Cantrill: That's like, am I in the right discord? I have am I in some Gen X retirement home? Well, I mean, yes and Yes and the so there was a and I was as a kid, this felt antiquated. There was something called the emergency broadcasting system and they would have these tests of the emergency broadcast system where they would have these long beeps. So my Adam, my question for you is, did you ever see an emergency broadcast on the emergency broadcast?
• 1:46
  Because we got these tests all the time growing up, right, watching TV.
• 1:49
  Adam Leventhal: No, I don't think I did. And it I feel like the, you know, the other day I was driving, I got a silver alert and I feel like it's a similar kind of thing where you're like, you're, you know, your phone erupts or whatever, but it was like, you know, thirty years ago instead. So your TV just starts bleeping.
• 2:05
  Bryan Cantrill: Bleeping. I had I growing up, I actually did have an actual emergency an actual use of emergency broadcast system for a tornado alert in in Colorado, East Colorado. So Well, there
• 2:16
  Adam Leventhal: you go.
• 2:18
  Bryan Cantrill: Very exactly. But so I thought I but that that is not what that chime was. That chime was I'd have something historical relevance. I the Internet needs me to understand.
• 2:27
  Adam Leventhal: It does. Yeah. Yeah. So I used I used the chime, which was my the Mac Plus boot up sound.
• 2:33
  Bryan Cantrill: Oh my god. Die cut. It almost came out my nose. Like, the dog barked right on the word chime.
• 2:39
  Adam Leventhal: She's like she's like, no. She she knows. She knows what's up. Okay. Thanks, pal.
• 2:43
  Thanks. Part of it is, honestly, I think of that chime every day because oh god. Hold on. I gotta deal with this.
• 2:53
  Bryan Cantrill: It's like, the dog is like the dog is like, you're not giving me credit for the chime. I it was my idea to use that sound. I know this guy is gonna pass it off on his own. And the dog the dog's very upset. Ben, I'm so sorry.
• 3:06
  We have a long standing tradition of not introducing our topic for some number of but we're we're running towards the end of the banter, so don't worry. We're we're we're
• 3:13
  Ben Shindel: we're we're there. No. Sorry
• 3:17
  Bryan Cantrill: about that. So what yeah. What is the Chime, Adam?
• 3:22
  Adam Leventhal: Oh, so Chime is the Mac Plus boot up sound, which I think about almost every day, which is a little embarrassing because my son's school, like the bells that announced the start of the class, are exactly the tones that the Mac Plus made if you see that it dims incorrect a sim. Excuse me. It's it's it's sim incorrectly. So if you did that, it would have this kind of, like, nasty startup sound, and I think of it every day. But I did the the the affirmative startup signed.
• 3:53
  Bryan Cantrill: Do you realize that this is the equivalent of someone coming by our school growing up in the eighties being like, you know, your school bell reminds me of the sound that RCA Victor would make as you warmed up the tube. I mean,
• 4:08
  Adam Leventhal: in my defense living fossils. This is the first time I've I've articulated this. Okay? I've I like, I have not even imposed this useless yeah. I mean, even I have not even imposed this useless observation on my wife who who she would say, I impose every useless observation I have on her.
• 4:27
  Bryan Cantrill: Oh, come on,
• 4:28
  Adam Leventhal: There are more.
• 4:28
  Bryan Cantrill: Exactly. There are oh, there are many more. Sorry where this came from. This is there's many, many more. Alright.
• 4:35
  Well, so a a very special sound to you. Sounds like emergency broadcasting system to me, but I was I thought it was great. There was a it was nesting up a little chime when we we referenced previous episodes. But and I think we're gonna be referencing I I just gotta say, get the chime warmed up. We are gonna be referencing some previous episodes because, Ben, you should know that this is not the first time we have picked up fraud on this podcast.
• 4:58
  In fact, I was thinking it I I someone online said like, oh, this is like oxide and fraud. I I kinda like the fraud cast, Adam.
• 5:05
  Adam Leventhal: That's really good. Really good. I love that.
• 5:07
  Bryan Cantrill: Did not come up with that one. Nice. Welcome to the Oxide fraud cast where but we we got our Adam had the delightful idea of a March madness of Silicon Valley fraud where you would fill out your bracket. I mean, and everyone knows that Theranos is going deep. The question is, you know, Theranos is obviously coming in as a one seed, you know, are they gonna be able to take it all home?
• 5:30
  Yeah. But this is a bonkers story as far as I and I have gone down the fallen down the rabbit hole on this one. I think this is just absolutely nuts. And so Ben, maybe you can kind of give us the entree to this because, you are a material scientist who has a newsletter called the BS detector. And this was kind of a story in January.
• 5:59
  And then the and we're talking of course about this paper that was written by Aiden Toner Rogers, a lot of notoriety around it, artificial intelligence, scientific discovery and product innovation. And then there's this MIT retraction that almost, I mean, from my perspective came out of clear blue sky, but did you know that a retraction was in the works or like take us through when you saw that retraction?
• 6:29
  Ben Shindel: Yeah, so I had no insider information before. I saw on Twitter people were discussing that it was, know, MIT had put out a press release saying, you know, we have no confidence in this work. And I remembered reading about the piece when it came out a few months back, it was kind of in between when I
• 6:48
  Bryan Cantrill: had finished my PhD, when I
• 6:49
  Ben Shindel: was starting a job. So I was like, I wasn't actually reading papers at that moment. So I just saw the press coverage. And I wish I'd read the paper at the time because I feel like it would have I I would have noticed stuff about it. But at the time I just saw the press coverage and I figured, oh, okay.
• 7:05
  I guess that makes sense. Like, they, you know, they gave some researchers some AI tools and it helped their performance, and nothing about that was super shocking. I guess if I'd read a little closer it would have been shocking, like the extent to which it helped. But yeah, I saw it on the when MIT put out the press release, Wall Street Journal kind of covered their mistake, And and I was, you know, immediately captivated by it because, you know, like you said, I have I have a blog called the BS detector. I'm a material scientist.
• 7:33
  I'm interested in AI, so it's been a perfect story for me.
• 7:36
  Bryan Cantrill: Right. I mean, just like, am I controlling reality with my mind right now? Is this like, has this been created for my benefit? I mean, it feels like you're in a unique position.
• 7:45
  Ben Shindel: Well, and on top of that, I actually like, not like I have a lot in common with the author of the article, but, you know, he he played basketball and tennis in in college. I played tennis in high school. I played pickup basketball now for fun. You know, he went to McAllister. My my ex from high school went to McAllister College, so I just feel like I had a lot of
• 8:08
  Bryan Cantrill: success with And,
• 8:13
  Ben Shindel: you know, about the same age.
• 8:15
  Bryan Cantrill: About the same age. Right. So you start so you see the retraction, and you're like, alright. Time to actually read the paper. Can, jaw must have been in your lap.
• 8:27
  I mean, can you, so what was it like to actually get into the paper and read it? And at what point did you realize like, mackerel?
• 8:36
  Ben Shindel: Well, because I knew it was getting retracted and it was fraudulent. Hindsight's really twentytwenty in that case. So I was picking up on the things that I think are probably BS in the paper. Again, I don't want to like, you know, I guess there's some small possibility that actually the data is somehow accurate or some of it's accurate, but he just wasn't supposed to have it. Or there could be something else that was the reason they're attracting it.
• 9:02
  But I think that's unlikely. I think most likely it was, completely fabricated data or at least wholesale kind
• 9:10
  Adam Leventhal: of Yeah,
• 9:12
  Bryan Cantrill: okay, so this is something I definitely want to get to because I want to understand kind of where the fabrication starts. I think, do you think we can say with like relative certainty? I mean, you said in your blog entry, is my face going to be red if he's got legitimate data that has just been somehow manipulated incorrectly. But it really does not feel that way. This feels like it's wholesale fraudulent, doesn't it?
• 9:35
  Ben Shindel: Yeah. I I agree. And there's a couple of like I mean, in the blog post, kind of thought that the smoking gun was he has this material similarity plot where he and this is where the methodology kind of just, it doesn't make any sense to me how this could be done. And also it doesn't make any sense how this could be done without like, I don't know, hundreds of hours of work. And also people who really know how to interface with the materials project API and like a bunch of other stuff.
• 10:05
  It's just not, it's not a simple task to do. But he has this plot where he shows, where he's making the case that materials that were developed with the help of the AI tools are less similar from other materials and literature, so they're like more novel. And he has this histogram of the material similarity before and after AI, and it shifts to one side, it's more novel. But the shape of the distribution is weird. There's like gaps between the numbers on the plot, and it just doesn't make sense to me intuitively why that would be the case at all.
• 10:44
  And then I saw one of his citations, which was like the second or third citation, like it wasn't the main citation for the methodology, but he mentions it in other part. That paper has a very similar shape to their similarity distribution, except it's for drug development. So kind of a different use of that methodology entirely. And it has that shape, which might make sense for the molecules they're looking at. It doesn't really make sense for a broad class of materials.
• 11:13
  And I think that he probably just saw that shape of distribution, of drew the data into that shape, and then put it in that graph.
• 11:20
  Adam Leventhal: At risk of I mean, I think we're now past the point of having buried the lead, Brian, but could we maybe rewind a little bit and talk about what the extraordinary claims that the paper made?
• 11:31
  Bryan Cantrill: Sure. Yeah. I Ben, do you do you wanna do the honors? I'm I'm happy to explain my inference, but happy for you to do it if you
• 11:38
  Ben Shindel: Up to you. I I'm I I
• 11:39
  Adam Leventhal: can Go for it, Ben.
• 11:40
  Bryan Cantrill: Yeah. Yeah. I was like, I hear from this guy enough. Like, this guy's just gonna start he's gonna start berating my beloved sounds again, so my beloved tones.
• 11:52
  Ben Shindel: Yeah, I mean, sure I can. Oh yeah, so the article. Offensively, you know tracks a thousand materials researchers
• 12:03
  Bryan Cantrill: at a
• 12:05
  Ben Shindel: yeah, at an unnamed R and D lab that comes, you know, some some big company that could that could support such a large amount of just materials discovery researchers, which I don't think this exists. Like I don't think there's a company in The US that has many people devoted to just materials discovery such that they could all be tracked on the same metrics like this. That would be very strange to me, but you know maybe it's possible that there's one or two companies that have that many material scientists doing this. Don't know. And it tracks, it has a randomized trial, and the randomization is that they rolled it out in three steps to their teams at the company, rather than doing it often
• 12:46
  Bryan Cantrill: does when making changes at a company. I mean, sorry. Is everyone else just rolling out changes and not doing it in three different waves of that align on air. I mean, come on. This this is the way we do.
• 13:00
  Just could see changes.
• 13:01
  Adam Leventhal: That's right. Just just like a scientist. Yes.
• 13:04
  Bryan Cantrill: Just like a scientist. I'm so sorry. And, Ben, I'm sorry to keep it. It's already lost here. Yes.
• 13:08
  Priti, continue.
• 13:10
  Ben Shindel: Oh, yeah. No. Interrupt at any time. It's, that alone was kind of surprising. And then of these thousand scientists, he tracks their age and how long they've been working at the company or something like this and their degree.
• 13:26
  But then he also tracks their specialization for materials and it's very broad, roughly like one fourth in each of the classes of materials he lists, are biomaterials, ceramics and glasses, metals and alloys and polymers. Putting aside, a of materials don't easily fit into those categories if you're working with composite materials that are but whatever. It's just weird that there'd be such a large company with massive amount of researchers, you know, a 18 researchers, a 18 researchers that got the AI tool, so maybe some didn't. And they they
• 14:01
  Bryan Cantrill: would specificity on what the the AI tool that they're getting is. I mean, I was did I miss that specificity about what exactly these researchers are supposedly being granted? I mean, mean, source said it was a GAN, but it seems like it's a GNN. I mean, did you get any details on that, Ben, in terms of like the AI tool that they're being granted?
• 14:24
  Ben Shindel: Yeah. So he spoke about this on, I think someone asked him this question, I don't know if it was NDAR, there's a recording of him speaking at some conference virtually. And he answers this question cause they're like, what was this AI tool? And it's also weird because back in 2022, there wouldn't have been like, certainly the LLMs that were out there when this trial would have started, were not gonna be generating
• 14:48
  Bryan Cantrill: accurate truths at all. Yeah. Would have gone Nazi in a second, right? Mean, this is the in 2022, you've been like, we're having Nazis develop our materials now? Yeah.
• 14:57
  Mean, it's like, it's crazy that it was things have progressed so much since then.
• 15:02
  Ben Shindel: Yeah, but at the time there were like materials discovery AI tools that you know, I mean, it's unclear how good they are, but there was a bunch of companies that were making like they were at the time and still are I'm sure making probably they've got a lot better at it, making these kinds of like artificial intelligence tools for predicting material structures or something like this. But they're like usually for pretty niche applications and guess I'm unaware it probably exists that there is some like broad data tool set for this kind of thing somewhere in the world, but I'm unfamiliar with what it would be or what company would be purchased from if that exists. But he did say that it was, I forgot exactly how he described it, but it was something like you can feed it your proprietary data for the company and that helps it. Basically give it like post training data or something, and that helps it predict materials in a particular domain, I think was what he was kind of trying to imply. Again, didn't get these details in the paper, so I don't know if he really had something super specific in mind, but that's the best guess I have.
• 16:13
  Bryan Cantrill: Yeah, and so and then the findings are that getting the AI, the AI tool, the unspecified AI tool, it's a graph neural network had this unbelievable lurch in productivity, very clean lurch in productivity across these three waves. And just to the point you're making at the top end that you are getting not only are people more productive, but they're discovering more novel materials. Right?
• 16:43
  Ben Shindel: Yes. And and higher quality materials. So more novel and higher and higher quality.
• 16:48
  Bryan Cantrill: Oh, okay. And Wait. Why not? Yeah.
• 16:52
  Adam Leventhal: And more patents and, you know, across the board.
• 16:56
  Ben Shindel: Someone else pointed out that it appears like the patent filings start at roughly the same time as the new materials spike, which that seems like very unintuitive because you'd expect them to discover the new materials and it would take a while for them to actually file the patent, maybe even a really long time. But they both spike at the same time and if some economists were suspicious of that, I guess I don't really know what it's like to you know, what the patent situation is like at these companies, so I I wouldn't know. Nick, could you just
• 17:27
  Bryan Cantrill: educate us a little bit? Because I think part of what is so ridiculous about this paper is to me it has like a very childlike sense of how material science actually works. Could you shed a light on how these new materials are discovered and brought to market? Because I refuse to believe it's as pat as is presented in this work. I've got to believe that there's a lot more nuance to it than that.
• 17:48
  Ben Shindel: No, yeah, you're 100% correct. And that is one of the funniest parts of the paper. It's like, I think it's very much like how an economist or how an econ student might like imagine, or somebody who works in like an industrial organization, economics, something like this, imagine that materials are discovered and patented and products are made. But I would guess only a very small percentage of the R and D work is like, all right, let's have 100 teams of researchers go and just find new materials. A lot of it's in developing materials that they already know exist.
• 18:22
  A lot of it's in working on processing those materials or making them useful for real application or modifying them in ways that wouldn't generate a new crystal structure at all.
• 18:33
  Bryan Cantrill: I got to feel like a bunch of industrial activities just thinking about, we got a computer company, we make hardware and there's so much engineering that goes into DFM for designing for manufacturing. I got to feel like surely a lot of material science is like, hey, can we find a way to make this material more readily manufacturable? Can we just change the economics of this? I mean, that crazy? I mean, feels like that's going to important for material scientists.
• 19:03
  Ben Shindel: Yeah. That's incredibly important. Like, I mean, I even like, the the stuff that, like, say, battery companies are doing right now where they're trying to expand beyond particular battery formations because they have, you too much cobalt in it, so that's expensive. So they go to nickel, and then nickel is expensive, so they go to, you know, iron and manganese. And so a lot of that's like, these aren't new crystal structures of materials or new material.
• 19:24
  You know, they're just trying to make the economics work by, you know, really slowly iterating on what they have. Right.
• 19:31
  Bryan Cantrill: And as opposed to this idea of like, oh no, we material scientists. And I love like the input to the AI are the properties that you want in your new material. It's like, is this a genie? Is it, I mean, do you get, am I not allowed to wish more wishes? I mean, just feels like it's incredibly reductive.
• 19:51
  And then, but the results, so they show a huge spike in productivity, in novelty. Then there's like this also very weird finding that these mythical material scientists are now like less satisfied with their job. They're finding like their craft has been eroded. I mean, they used to get to the point where like, how is that even remotely believable? Don't know, what was your take on that conclusion?
• 20:23
  Ben Shindel: Well, by that point of the paper I was arguing like, this is nonsense. So the survey results were just like, I imagine he put his beliefs on what people would respond or pick something that was like mildly surprising, but not shocking enough to make people really suspicious. It's just funny that like every survey question has a very low key value and it has like a significant finding.
• 20:45
  Bryan Cantrill: Okay, so can we talk about the survey for a second? Because part of this is like he determined how much time is spent on ideation versus construction. And material scientists, the artisans that are material scientists, they love to spend their time in the ideation of materials, not in the tediousness of actually constructing them. Mean, it feels like he's got like this kind of met like, it's like this is a stonemason in your mind. I mean, I got no idea, like, where his mind is.
• 21:15
  I think alchemy. Yeah. Alchemy. Totally. I mean, it probably is alchemy to him.
• 21:19
  So, like, okay. Fine. But then it the but then determines that, oh, they're spending less time on ideation and more time on materials generation. How do they know that? Because there's an administratively required audit log that as a scientist, you log this and he was able did you see this, Adam?
• 21:43
  Yes. And he's able to consume all of these audit logs from the thousands of scientists to determine how they spent their day. I mean, it's like, have you like what company on earth can take a thousand knowledge workers and get them to fill out what is the equivalent of a TPS report with any kind of fidelity?
• 22:02
  Adam Leventhal: You're like these thousand PhDs, experienced researchers, we asked to like fill out a time card like do productivity Like, yeah, oh, no. A %.
• 22:15
  Bryan Cantrill: A %. And I definitely will not use this free form text field to say that I stuck bananas up my nose. I mean, it just feels like, I mean, I don't know. But Ben feel free to, like, condemn us as software engineers, but, like, certainly could not get software engineers to do anything. I think that it you know, half of the people respond to the survey and these survey questions are just like, I I Ben, is that as absurd as it sounds?
• 22:40
  I mean, that's it to you.
• 22:41
  Ben Shindel: I guess in the in the company that he's created in his, you know, in in this fictional paper, they have, like, they must have tremendous, like, worker discipline. They have, like, hundreds of teams, each of a size of five. They have 100 compliance with randomized trial. I don't know. This
• 23:02
  Bryan Cantrill: fictional company needs to be talking a lot more than about merely their material science. It is the it is the managerial alchemy that this this company has discovered. It is I mean, there's the the this fictional company has really done extraordinary things. Yeah. Yeah.
• 23:17
  But that I I just thought was just and and then uses the survey results to get to kind of this happiness. And it feels like Ben, it feels like it it is all in service of kind of feeding this narrative about about what AI is gonna do with respect to our jobs. You know? It feels like and then that's, like, certainly how it was inferred by the journal and and the Atlantic and some of the other folks that published on this. Did you listen by the way to I mean, I I and I know I'm I apologize for having DM'd you last night when I was falling down the podcast rabbit hole.
• 23:51
  Ben Shindel: I so I read it in full, and I listened I listened to a part of it to, like, hear what his voice sounded like during the podcast. But, yeah, I read the whole thing. The Internet
• 23:59
  Bryan Cantrill: It's crazy. It is crazy. It is crazy. Is crazy. And you're like, okay, so this gets us to like, I've got so many questions.
• 24:07
  First of all, where do you think this thing goes off? Like, what do you and obviously, I'll be inviting you to wildly speculate here because we certainly don't know. Where is the ground truth in this do you think? Where did he go off the road?
• 24:28
  Ben Shindel: Well, I mean, if the data is fabricated from the beginning, I guess he must have like come in, you know. I mean, there's no data to work with from the start, so I guess it was just the entire thing was in his in his mind. I don't know.
• 24:41
  Bryan Cantrill: I mean, that is gonna be nuts. I actually don't see a middle ground for this. I got to tell you, like I don't see how this isn't because there's so much here that, don't know if you bet if you saw, actually was like, all right, what does ChatcheapT think of this thing? And so I uploaded the paper and I'm like, hey, I'm just like trying to understand this thing's claims. ChatGPT is like super tight, looks great, methodology looks great.
• 25:05
  Like no complaints, can't imagine anything wrong with this thing. And I'm like, hey, what do you think about like the thousand researchers? That's like a lot of people at one company. And ChatGPT, of course, its I mean, it's such a sick effect. I mean, just acts like, oh, you've just like, wow, what a discovery.
• 25:21
  This is you're right. You're really onto something. And of course it immediately goes from there too. And you kind of start bringing up some of the issues in this thing. And it's like, oh my god.
• 25:30
  This thing is like obviously a complete sham. I just don't know like what part of this is real. Do you think he ever talked to a materials company at all?
• 25:41
  Ben Shindel: I I doubt it, but I mean, I maybe.
• 25:46
  Bryan Cantrill: Maybe he talked to, like, a
• 25:48
  Ben Shindel: friend who was working for one to, like, kind of get a vibe of what it was like. But Yeah. I don't know.
• 25:54
  Bryan Cantrill: Because you mean and and you talk about this as well because you get into like another problem here is like, okay, let's mythical company, know as you say, enormously disciplined, these incredibly small teams and so on blah blah blah blah blah. Why do you give this data to a, like, a second year PhD? I mean, Ben, you're coming out of your PhD. You know how like, oh, yeah. We just all companies just give PhD students this incredibly high quality.
• 26:17
  That's the way Not
• 26:19
  Adam Leventhal: only that, but but his PI. Right? His his lead professor apparently, you know, maybe either didn't broker this conversation, or if they did, also then backed away and said, you know what? You take full credit. Like, don't put my name on this paper.
• 26:33
  This seminal work. Like, you take all the credit. I'll just be happy to to step back into the background.
• 26:40
  Bryan Cantrill: Yeah. Well, it's the difference.
• 26:41
  Ben Shindel: Timeline of the paper, it implies that he had started it before he'd even come to MIT. So I guess they were like, yeah, I mean, it's your work from before. But then other people thought that, oh, maybe it's real because his advisors are really big names, so it doesn't make sense that they give him the data. Maybe they would give you know, one of these very famous people saying next to him in the Wall Street Journal cover, right? H.
• 27:04
  M. Lau is like the most cited economist in the world. So okay, guess they gave him the data and then he passed it on to the student, but that doesn't make a lot of sense either. So, yeah, I mean, I don't I think people were just like speculating as to why he had this data in particular.
• 27:18
  Bryan Cantrill: Well, and and there's definitely a halo effect there, right, in terms of I mean, had he not been associated with a Nobel Prize winning economist, you know, maybe this would not have had the legs edited. But people I mean, it reminds me, Adam, very much of Theranos and kind of like the first people that kind of Liz Holmes endeared herself to allowed her to get all this other credibility. Like, and that credibility kind of starts with like a con job in a single professor's office. You know what I mean? And I Yeah.
• 27:45
  I it's hard not to read that the same way where he persuaded the Nobel Prize winning economist somehow that this was legit, and they clearly did not I I I mean, Ben, you got to think that these two What do you think their disposition is to this? And how much fallout do you think it's going to be for these two?
• 28:07
  Ben Shindel: They're probably really torn up about this. I would guess they feel really badly about it. Also like, it's not necessarily their fault. I mean, these are some of the busiest people in the world. Why would they assume that this student at one of the best econ programs in the world would have falsified this?
• 28:25
  There's a good chance he's just the next one of them, next really good genius economist. So, and they're not material scientists,
• 28:34
  Bryan Cantrill: so they might not pick up on some of the
• 28:35
  Ben Shindel: things, and maybe they assume that he had more details, maybe he was misleading them with, so like that's the thing that I don't know if I'm going ahead of myself, but there was the thing with the intellectual property complaint filed by Corning.
• 28:48
  Adam Leventhal: That is amazing. You gotta talk about that. This is amazing.
• 28:51
  Bryan Cantrill: Yeah. This is amazing. Yeah.
• 28:53
  Ben Shindel: So someone someone, I think, Will Wang, Will Wang twenty one on Twitter found found it somehow, like searched his name and and Corning, maybe searched a few companies, and found an IP complaint with the the World IP Organization, WIPO, for a URL, corningresearch.com, and they were saying, hey, you can't do that. That's our IP. You cannot make that URL. We're requesting that it be transferred to us. And I don't know if Corning even I don't know if they were like, oh, this guy is the guy that put out that paper reporting to be something like us.
• 29:30
  Bryan Cantrill: Or if they're just I I would I say no on that. Adam, do you think? I think this is, like, literally this is a totally other division of Corning. It's like, no, this is just like what we do. And this is like, we've got a standing thing that we do this kind of search on.
• 29:45
  We always look at the new new domain names, and this has come up before. I I think these, I think it is like almost likely that Corning still doesn't know by the way.
• 29:54
  Adam Leventhal: I'm with you. I bet this is some like, AI, this is like some grep based lawyer that just is looking, I mean, you know what I mean? Just like looking for a Corning
• 30:06
  Bryan Cantrill: thing. Slayer.
• 30:07
  Adam Leventhal: And, like, knocking knocking down, you know, fraud and stuff like that. I'm sure I'm sure this is just like first line of defense.
• 30:14
  Ben Shindel: Interesting. Yeah. They actually got a
• 30:15
  Bryan Cantrill: paper about how AI has made their division much more effective, by the way. I think has. It's actually difference I think where they've actually, no. We've eliminated all of our TCs. So I think I mean, time will tell.
• 30:28
  But I think that this is, like, totally unrelated. They just saw it and, like, no. We just, like, filed this thing. And they won on it. Right?
• 30:37
  Then they I think that they prevailed.
• 30:40
  Ben Shindel: I well, the the URL's, like, no longer up. So I would assume I mean, you know, it's certainly their case seems strong, but I don't know a lot about IT complaints. They are corning.
• 30:53
  Bryan Cantrill: So but but
• 30:54
  Ben Shindel: what's interesting is that that basically implies that he's using that. And I I would guess he was using that. Someone said, like, that perhaps it was to send fake emails from at corningresearch dot or maybe to like make PDFs with convincing URLs. I don't really know what his goals were
• 31:10
  Bryan Cantrill: with that, but like that could have been why people believed him.
• 31:13
  Ben Shindel: Because maybe he had correspondence with some reporting. So like that, that's pretty indisputable if he did that. I
• 31:21
  Bryan Cantrill: mean, again, I definitely And a part of the reason, MIT definitely launched this guy into the surface of
• 31:27
  Ben Shindel: the
• 31:27
  Bryan Cantrill: sun. I I think even their language is, like, no longer affiliated with MIT. And you're like, okay. But is he alive? Because you're actually not answering that.
• 31:37
  Like, that's not like
• 31:38
  Adam Leventhal: Right. Have you let him live?
• 31:40
  Bryan Cantrill: I longer affiliated with MIT. I get stuff the press release is exactly what we said. No longer affiliated with MIT.
• 31:46
  Adam Leventhal: Sleep, sleep, suspicious.
• 31:47
  Bryan Cantrill: You do wonder. Well, you do wonder because if one of his advisors is like, hey, you know, you've been working with this guy. I haven't like heard anything about it. Like, could you CC me on the next email? He registers his domain name and starts cc ing the the don't you think?
• 32:02
  I don't I'm just feeling like it it Totally. I mean,
• 32:04
  Adam Leventhal: I think I think MIT, you know, would have it both ways. Right? I mean, like most major institutions, You know, when he was riding high, you know, I bet that they were kind of riding along with him. And and it would very obviously, I mean, there's not like, there's another scapegoat, but easy to separate themselves from a second year PhD student.
• 32:25
  Bryan Cantrill: And when was the journal story, Ben? Do you know right off when they because the journal was the journal the kind of the first one to break this?
• 32:33
  Ben Shindel: I I think well, I think MIT broke wait. Wait. You mean the, like, original paper?
• 32:38
  Bryan Cantrill: The original the the the story that kind of this became a new story. And Yeah. Because I think it's so the other thing that that is I mean, so in the complaint, they say the domain name was registered on 01/12/2025. It is conceivable that it's part of fact checking at the Wall Street Journal. And they're like, hey.
• 32:57
  I I need to see something from this. Like No.
• 32:59
  Adam Leventhal: No. So the journal
• 33:00
  Bryan Cantrill: story Two years.
• 33:01
  Adam Leventhal: The journal story is late December December '20 ninth '20 '20 '4. But MIT does reference an internal review. So possible that, like, there's some internal review that's kicked off in fact checking, and that that website was in service of that, let's say.
• 33:20
  Bryan Cantrill: It feels very likely. Just feels it's a I mean, it's insane, I think, that the So then the what is your kind of because then you've just been through a PhD program. You've a PhD material science from a a friend of mine from college, Rachel Bishop had been in college. But she did her PhD in material science at Northwestern. Nate ultra material science program.
• 33:53
  She by the way told a very funny story of, and I think it was Rachel herself, but it was not Rachel, it was a friend of hers in the program that was working on her PhD because she got her PhD very quickly if I'm remembering this correctly. Rachel, I'm sorry if I'm remembering the details incorrectly. But she got her PhD in, I believe four years. And because her advisor lost track of the number of years she'd been there and it's like, hey, like we got to like get you out of here. Like you've been here for a long time.
• 34:22
  And she's like, okay, yeah, sure. Like, average time is like six or seven years for the PhD. Be like, no, we gotta like get you out of here. It's like time's marching on. And so when she actually got her PhD, everyone at this like this, you've got your I bet as
• 34:38
  Adam Leventhal: I'm sure you did, you've
• 34:39
  Bryan Cantrill: got like a celebration. Everyone is after you've successfully completed your defense, and everyone's like, man, you've set like the new land speed record. And her advisor's like, are you talking about? Like, she no. He's like, no.
• 34:50
  She got her PhD in like four and a half years. Advisor's like, nope. She's been here for six years. And she was like, no. I've been here for four and a half.
• 34:59
  Apparently, there was the advisor was like, oh my god. I lost a year of free work. What have I have I done?
• 35:06
  Adam Leventhal: Very telling. Yeah.
• 35:08
  Bryan Cantrill: Anyway but, Ben, I mean, you've just gone through, and I presume it was not four years, but, like, I mean, this is really hard intellectual labor. What is your kind of emotional reaction to someone who is, as you say, like kind of like a doppelganger in some regards, like similar age and so on? I mean, you just like, man, this is like, this undermines all the hard work that all of the rest of us are doing? I mean, on one hand, yes. On the other hand, it seems like what
• 35:36
  Ben Shindel: he did was very hard. Like, it must have taken a lot of work to create something. Even if it's possible.
• 35:45
  Adam Leventhal: You're like, you you if you contemplate this sophistication of fraud, and that was writing impressive.
• 35:50
  Ben Shindel: You know, it's quite impressive, the extent and, like, the imagination. You know? But but yeah. I mean, obviously, you know, fraud is pretty bad.
• 35:58
  Adam Leventhal: That that I mean, that back to Elizabeth Holmes. I mean, you gotta you gotta hand it to her too. I mean, that that level of fraud did not come easy.
• 36:05
  Bryan Cantrill: That is a lot of work. A lot a lot of sweat on the brow. Yeah. Exactly. And if if he could have pulled off a fraud
• 36:11
  Ben Shindel: of this magnitude, like, he could have just done the work and you he wouldn't have gotten a paper in his first year of this magnitude, but like I feel like he would have been fine.
• 36:23
  Bryan Cantrill: Well, and this gets you to kind of that larger question of like what goes on? Because I mean, this is a person who's got a ton of things going for them. Right? Like, that has got a sterling education, you know, as you say, was a was a play college hoops. I mean, is like a like, what a polymath.
• 36:43
  And, like, in kind of the progression of things did it enter your mind to, like, take I mean and also, like, did you play this out? Like, is this gonna work? Is this and I definitely wondered, but part of the reason I was listening to that podcast closely is he at some point like, oh, shit. This is getting way too much attention. Like, I'm supposed to use this like to get my PhD.
• 37:08
  I'm not supposed to be a media darling with this and this is gonna cause way too much attention to cause I mean but he doesn't seem to be thinking that. I don't know. What was your take?
• 37:19
  Ben Shindel: Yeah. I can't I don't understand how he thought he would get away with it. I mean, like, it would only take one material scientist to, like, the right you know, to, like, read this paper after it came out and just be like, what the hell? Like, that's not that's not real. But I I don't know.
• 37:35
  Yeah. I don't I don't know what he was thinking.
• 37:38
  Bryan Cantrill: And newsletter, I mean, got there are a bunch of comments on there that I thought were interesting. And you had a bunch of people who were like, damn it. I read this paper and I thought to myself, wow, that's too many researchers. And I like the guy that said, like, I was really envious that, like, oh, man, MIT gets all the great research. And I, you know, I just would not I wouldn't have an opportunity to be, like, have that kind of access to that kind of data.
• 38:02
  And as he said, listen,
• 38:05
  Adam Leventhal: But no, Brian, you found an amazing comment on hacker news.
• 38:09
  Bryan Cantrill: Oh, I did find an amazing comment on hacker news. Yeah.
• 38:12
  Adam Leventhal: I've got that up if you want me to read I'm
• 38:14
  Ben Shindel: I'm Yeah.
• 38:15
  Bryan Cantrill: Yeah. Go ahead. Yeah. Read it. So I bet you follow Hacker News as well.
• 38:19
  This is kind of our this is like I I feel like I don't wanna, like, introduce you to Hacker News if you haven't seen it because I this is isn't this, like, teaching teaching someone to smoke, Adam? I feel like this is not A little bit.
• 38:30
  Ben Shindel: The y it's the y combinator, like There
• 38:32
  Bryan Cantrill: you go. Yeah. Yeah.
• 38:33
  Ben Shindel: Yeah. Okay.
• 38:34
  Bryan Cantrill: Thought he was already able to smoke. Never mind.
• 38:36
  Adam Leventhal: There you go. Right. Right. Already pack a day habit. So this is interesting.
• 38:40
  A large US company with over a thousand material scientists, there can only be a handful of those, introduced a cutting edge AI tool and decided to make a study out of it, randomize it, and give all the credentials to some econ PhD student. Would love to know more about how this came to be. Also, why his PhD supervisor didn't get a coauthor. Never seen that. I'm always slightly suspicious of these very strong results without any public data or way to reproduce it.
• 39:07
  We essentially have to believe one guy's word. If you dug that out and you're like, yeah, damn. Like spot on guy.
• 39:15
  Bryan Cantrill: That's a pressured comment. Yeah. This is when the original paper was published in the journal.
• 39:19
  Adam Leventhal: Six months ago. Yeah.
• 39:21
  Bryan Cantrill: Yeah. This is Leander who I actually then emailed Adam. I was like, hey, I don't know if you saw the epilogue on this, but like, were you right on this? Leander has heard of the podcast, go check out the episode, but that was amazing and saw that like, this doesn't quite add up. And it was funny because, you know, someone else had kind of pointed out that comment on Hacker News.
• 39:41
  I discovered the comment. So I saw a comment that referred to the comment and someone's like, oh yeah, but there's always someone on Hacker News saying like, you can't do something. And it's like, which is true. Like admit, it is like true. But like, no, this person was not saying was actually not it was not a disparaging comment, I felt.
• 39:58
  I felt it was more like a, this is kinda that's
• 40:01
  Adam Leventhal: super weird. And Much more specific than the usual Hacker News jerks. Right? Real concrete, oh, this is curious. Not like, this would never work, or this is fraudulent just on its face, but like real specific critique.
• 40:20
  Bryan Cantrill: Totally. And so, Ben, another question I've got for you is because because this did go first of all, do do you know how journalists got wind of this, or do we know how they got wind of this?
• 40:32
  Ben Shindel: I think MIT put out a thing on their website, so I mean it probably they reached out to the Wall Street Journal, you know, journalists about it, because
• 40:39
  Bryan Cantrill: Wall Street Journal ran with MIT. You know, MIT rescinds you know, they say no longer a faith in this paper kind of thing.
• 40:46
  Ben Shindel: That morning I think, or maybe early that afternoon. So presumably they communicated
• 40:53
  Adam Leventhal: with
• 40:53
  Ben Shindel: the Wall Street Journal, but I think it apparently leaked, and then I heard that they sent out an internal email to the econ department at MIT, so presumably it's soon be hit an hour or two earlier.
• 41:08
  Bryan Cantrill: I think, I mean, it was your line, right, where someone who had that internal email was like, when I read the first two sentences, thought someone had died.
• 41:15
  Adam Leventhal: Yeah. It was this you had a great line, Ben. He said, so direly worded on the on the matter that on first glance students reading the email had assumed someone had died.
• 41:27
  Ben Shindel: Yeah. So I I heard this second heard this second hand.
• 41:29
  Bryan Cantrill: I heard this from a person
• 41:30
  Ben Shindel: at MIT who heard this from someone else in the econ department. So, you know, that's my level of source on this matter.
• 41:38
  Bryan Cantrill: Well, in in in a way, someone did. Right? Because this and, Adam, is this a good segue to the experiment that you were doing
• 41:46
  Ben Shindel: with John GPT?
• 41:48
  Adam Leventhal: Yeah. I I got to pull that up. So we we were you did a great experiment to evaluate, you know, was this, like, was this real? A little back and forth. And we were joking about how, you know, this would be a quick way to fifteen minutes of fame and then ending your career.
• 42:05
  So I I said to Chad GPT, let
• 42:08
  Bryan Cantrill: me pull
• 42:09
  Adam Leventhal: up this. I I said, I'm writing a novel, I don't know, about a PhD student in economics. Can you give me ways in which the student could achieve brief glory but ruin their professional lives irrevocably? And note result number one says data manipulation for breakthrough result. Let's say they have a blockbuster paper claiming major finding, and then later it turns out to be the dataset was selectively cleaned or or fabricated, and the result is a retracted paper destroyed reputation.
• 42:36
  Just like, I mean
• 42:39
  Bryan Cantrill: It's chilling, as you said. Yes. I mean, our view is like, he's been here. Like, he must have like I think Tony Rogers has asked this exact question. I mean, I was observation was I feel like I'm playing, like, where in the LLM is Carmen San diego?
• 42:55
  I mean, I feel like this is the
• 42:56
  Adam Leventhal: A very extra reference, to be clear, but a a a video game where you would go around to different cities on your, what, like, Apple two GS. And 5% Apple two gs. What a great reference. And you would you would like and this is the one Apple g two gs in your school or in your classroom or whatever, by the way. Like, not every kid had one on their desk, like a Chromebook.
• 43:19
  And you'd ask some question. They say, oh, you know, someone was just here asking the exact same thing, but but your person is in a different city.
• 43:27
  Bryan Cantrill: They mentioned the Statue Of Liberty.
• 43:29
  Adam Leventhal: That's right. That's right.
• 43:30
  Bryan Cantrill: You think, okay. Where do you wanna go next? Or you wanna go to, like, Cleveland? Do you wanna go, you know, do you wanna go to to Seattle? Do you wanna go to New York?
• 43:37
  And you're
• 43:37
  Adam Leventhal: like, the
• 43:40
  Bryan Cantrill: the trust us. This is a great currency bet. I know this is like this. And we're so we're gonna be we'll use this. It's fine.
• 43:46
  Adam Leventhal: Yeah. That's fine. But yeah. Chad GPT nailed it. Like, ruining your career
• 43:52
  Bryan Cantrill: in one move. Yeah. And I feel like it is it's dire. And, you know, Adam, I looked up Steven Glass. And Ben, know Steven Glass is by any chance?
• 44:01
  That is that's actually turning into a generational reference, although not of the emergency broadcast system, Carmen San diego variety. This was a do you know who that is? This is this is a guy who wrote for the New Republic, I think
• 44:15
  Adam Leventhal: it was the That's right.
• 44:16
  Bryan Cantrill: Yeah, right. It was the Republic, right? And wrote these amazing stories and would get these amazing stories and was kind of turning them out. And Adam, I knew folks at the time because I mean, Stephen Glass and I are of the same vintage and definitely knew people who were kind of writing for those kinds of publications at the same time and just like green with envy over this guy. Like, how is this guy getting, like, all these amazing stories?
• 44:44
  As it turns out, like, he was making them up.
• 44:47
  Adam Leventhal: Yeah.
• 44:47
  Bryan Cantrill: And, like, it it truly ended his career. I mean, he he tried to go to law school. Sort of so. And in his career refused to bar him.
• 44:59
  Adam Leventhal: Right.
• 45:00
  Bryan Cantrill: Then he went to California. California also refused refused to I hate because he passed the bar, but they refused to to to seat him there and, like, fought that and, they went through an appeal, and they ultimately, like, denied it based on character. It's like this this stuff is gonna you you can't put this genie back in the bar. It's very hard to dismiss this stuff as a youthful mistake.
• 45:23
  Adam Leventhal: Okay. But you read did you read his novel? I mean, not his his his memoir. So so, yeah, I read the fabulous. So he I mean Did you say it?
• 45:32
  Oh, wow. Yeah. So ended his career, but sort of. Right? Because then he wrote a book about exactly what he was doing.
• 45:39
  Pardon me. Not a novel, but but more memoir. And I I felt fair I mean, whatever. Right? Who knows if it's made up as well.
• 45:47
  But a meta novel novel perhaps. But I felt pretty conflicted about, like, buying this guy's book about, like, how he had lied, cheated, and steeled. But, you know, maybe That's I mean, pardon me?
• 46:00
  Bryan Cantrill: Did you steal a copy?
• 46:02
  Adam Leventhal: That's right. So then I stole the copy. No. No. No.
• 46:05
  I'm I I it's around here somewhere, I'm sure. But I just mean, like, maybe this is maybe that's the next step in this in the the Stephen Glass fabulous plan. Right? Like, you achieved notoriety and then and then write about it.
• 46:19
  Bryan Cantrill: Well and, Ben, you referred to another I looking at previous instances of fraud, you referred to something I'd not heard of this when contact changes minds piece, which I then fell down that rabbit hole. Do you want to explain that? Because that was a more academic fraud that definitely had far reaching consequences.
• 46:38
  Ben Shindel: Wait. Are you sure this is me? What what fraud?
• 46:42
  Bryan Cantrill: You linked to it's like sorry. You linked to a piece when contact changes minds.
• 46:48
  Ben Shindel: When contact changes minds.
• 46:50
  Bryan Cantrill: Did you?
• 46:51
  Ben Shindel: Maybe I did. Let me see. What was this about? This is about liqueur? I think someone else might have done this, because I am familiar with liqueur, but, like, not very well.
• 47:01
  Bryan Cantrill: So if you follow the link entirely fabricated to begin with, that is a link to this when contact changes minds.
• 47:09
  Ben Shindel: Oh, linked it in the article. Yeah. Yeah. Yeah. Sorry.
• 47:12
  Think it meant, like Article. Yeah. So my friend who proofread the piece told me to add this link in there. Oh, interesting. I I have heard of Lacour.
• 47:25
  I think I linked to like the wiki page about Lacour or about Yes. The fraudulent article by him. Yeah. I don't know a ton about Lacour, but he was also, like, I think a social scientist who falsified data and then, you know, had to deal with the consequences.
• 47:41
  Bryan Cantrill: What what a better friend you have to be like, hey. I've got some I've got some other fraud links you need to throw in there. That's like you I like the circles you run-in, Ben.
• 47:49
  Ben Shindel: Right. Thank you.
• 47:50
  Adam Leventhal: Ben Ben, there's something sort of chilling in in your blog post where I I think you were you were saying that you weren't sure that some of these publications, you know, it's this this article was in preprint, but you weren't sure if some of these publications would have ever sniffed it out. Right? Like that they might have just gone to press with it, but and these were some pretty sophisticated, like exclusive publications. Can you talk more about that? Because I kind of wonder what else is out there.
• 48:17
  Bryan Cantrill: Yeah. I mean, I don't I I find it hard to believe
• 48:19
  Ben Shindel: it would have actually gotten all the way through peer review, but maybe. I don't know. I mean, like, it's not hard to get a revise and resubmit. I mean, like, also they they may have asked in the R and R. They may have been like, oh, well, you know, can you show us the raw data or, like, explain how you actually do these methodologies that, like, don't make a lot of sense in your brief explanation.
• 48:40
  Just use
• 48:40
  Bryan Cantrill: the register domain and I'll be right back then I can get you all the raw data. Right.
• 48:45
  Ben Shindel: And so it's possible this would never have actually gotten published or it's possible three economists reviewed it and they were all like, yeah, the econ stuff seems reasonable and it's like stellar work, so sure. Also reviewers don't like read over everything with a fine tooth comb assuming that it's going to be fraudulent. Mean, they generally operate in
• 49:04
  Bryan Cantrill: the assumption that the work is real. Is there also a peril of the fact this is cross disciplinary work, right? And so you're relying on someone who is an economist to have enough material science knowledge or be able to phone a friend for it. Is that part of the kind of the crack that's able to squeeze through, the fact that it's cross disciplinary?
• 49:29
  Ben Shindel: Maybe. It's possible they would have reached out in the editorial process. You know, I don't know how how it works. Those journals, I mean, it's a really good journal. Maybe they would have asked the material scientists to check it.
• 49:40
  I know that MIT, I think, solicited comments from material scientists. That might be how they originally uncovered it. I think I don't remember which article I think was the Wall Street Journal. They said that like a material scientist who it didn't name, but I think people on Twitter were naming it, but I don't remember who it was, had like personally messaged that, you know, I think a general and an auteur or something like that, or maybe the department and and, like, raised concerns with the paper, which which makes sense. I mean, it's, you know, it's suspicious.
• 50:10
  It's suspicious.
• 50:12
  Adam Leventhal: I'm listening right now to this very mediocre book that that every once in a while has just enough wisdom in it to to keep me going. And one of the it's it's about it's from this broadcast journalist. And one of the things he talks about is a story being too good to check. And I just feel like this paper suffered from that in spades in that it it's just in the right time and showing you know, Brian, you were saying there's sort of this, like, bummer downside of like, oh, these these pretend scientists no longer enjoying their pretend research. But I think that that is part of what makes it juicy.
• 50:48
  Right? If it was just like everyone's happy, more productive, and better, I think like the thing that makes this a headline and makes it exciting to think about and write about is the the downside. Right? Like this is the this warning.
• 51:03
  Bryan Cantrill: The thing we're losing. Exactly. Yeah. We're gaining these novel materials, but we're losing our craft.
• 51:10
  Adam Leventhal: Kind of. And I I think I think this loss of humanity is the thing that makes it too good to check and may and and sets people's disposition to say, I want this to be real. This this verifies
• 51:20
  Bryan Cantrill: Yes.
• 51:21
  Adam Leventhal: Like Yeah. Interesting. Either what I see or I wish I what I wish I could see. And and and it causes people to to not be as as thoughtful that they would given the exposure that they're giving it.
• 51:35
  Bryan Cantrill: Yeah. Yeah. Ben, what do you make of that? I I definitely agree with that. What do you what do you think?
• 51:39
  Ben Shindel: Well, I have I have a little anecdote. There's so there's there's a person, Robert Paulgrave, is a materials chemist, a professor, who I think I linked him at a couple of his tweet threads at the end, but he right when the paper came out, he had a thread on it, and he obviously didn't didn't assume it was fraud right then, but he kind of also he pointed out like, oh, this is interesting. It's this weird unnamed large firm. And then he was very skeptical of the, like that the materials they were discovered would actually be novel and original because in the past he had uncovered, and it wasn't fraud this time, it was actually a really good paper, an outstanding paper from a lab, I think somewhere in California, either Stanford or Berkeley, on automated chemistry and also AI enabled materials discovery. And that was actually, it was a great paper.
• 52:31
  It just so happened that the characterization tools they used to determine the novelty of the materials, it didn't quite pan out and the claims were a little more optimistic than they could support. And this was battled out in the journal itself. Think he wrote a response or something formally, or maybe he just tweeted about it a lot. But I think he kind of gained a lot of popularity for that because it a very good sober analysis. And so we had kind of the same critiques with this paper.
• 53:00
  And then once the fraud was announced, you know, he kind of then, you know, sort of like cited back that original thread and talked about it a lot. And I think it was at the time that the other paper had come out, it was also kind of the thing where everyone kind of wanted to believe it was true that like the automated chemistry lab, it could just come up with new materials and without a human in the loop, it was kind of doing these really impressive things. And it didn't pan out, but I think honestly, what they did in that paper back then, now if they did that again, I think it would pan out. Like the AI tools have gotten better, the categorization tools have gotten better. I think it probably works now.
• 53:34
  It's just a little bit ahead of its time and they're making a little bit too ambitious claims. But with this, I think again, it might be just a little ahead of its time. Probably in a couple years, if you actually did some sort of 1,000 person trial, you might find that it does really dramatically improve their ability to find new materials. But just because it's, like, what people want to believe doesn't mean that you can write a paper on it as if that's the data, certainly. Yeah.
• 54:04
  Bryan Cantrill: And that's what I mean, I I gotta say, listen to listening to to Tony Rogers on the podcast, I'm like, this guy really wants this to be true. And he's got a disposition about like, well, I believe in the future of this technology in terms of making things more productive, but I also got like concerns. It's like, oh, things that like exactly mirror the kind of the findings that you may have created, which to me was kind of already suspicious. And I wonder like to what degree, and I'm eager to get the full story here, did he want this to be the story? And then maybe there was some reality somewhere that Ben to your point just like didn't quite support it.
• 54:47
  It's like, yeah, they're using the AI tools and it's like, they're kind of in the noise like, or, you know, they and we're doing it in a much smaller level or like, yeah, they're discovering more materials, but they actually they wasted more time because they actually had materials that were that that they explored that they wouldn't have otherwise done that mean, mean, you could just imagine lots of reasons why it just is not it's not world changing. It's like, that's good enough.
• 55:11
  Adam Leventhal: Lack of job satisfaction is definitely the most credible part.
• 55:15
  Bryan Cantrill: Well, certainly in a in a in an organization that is able to maintain that kind of survey compliance, no shit people don't like their job. It's like, I mean, no, we all turn into survey because we're afraid. Layoffs. Can we talk about the layoffs?
• 55:33
  Adam Leventhal: That was great.
• 55:35
  Bryan Cantrill: Yeah. Ben, can you talk about the layoffs? This is such a cherry on top of this whole fecal sundae.
• 55:40
  Ben Shindel: Did I even mention I can't remember if I mentioned it in the article. Oh, yeah. Yeah. To to cap it off, here's how Toner Rogers describes the fortuitous round of layoffs at the firm. Yeah.
• 55:48
  So like, it's just funny that like, he doesn't mention it all in the paper, oh yeah, like 6% of the people, you know, like left the job or transition jobs at the firm during the time. So we had to like take them out of the sample. None of that happens, But like right at the end, after collecting data, three percent of the researchers were fired and that helped and that like, and the ones who were fired were the ones that had like worst judgment or whatever, which means that they like weren't able to adapt to the AI tools as well or something, which is just perfectly fits with his story. So it's a
• 56:19
  Adam Leventhal: little too perfect. Especially of of perfect organizational harmony where, like, they have stacked ranked everyone exactly accurate. I mean Exactly accurate.
• 56:30
  Bryan Cantrill: No one everyone agrees that it is this 3% that needs to be fired. And and then I think it was in the podcast or maybe in the paper where it's like, oh, but don't worry. They also continue to hire more top producing scientists. It's like, oh yeah, of course. Yeah, these things always, I mean, they fired, they put 30 PhDs on the street.
• 56:51
  And I'm sure like, yeah, I'm sure like no one ever reported about that. And then they hired 30 back that were much more productive. It's like, oh yeah, all this makes This is like no questions. Yeah. What is going through the mind of someone who's like, I don't know.
• 57:04
  This paper needs a little extra something. I just like it doesn't it's like, you know, I what about a layoff? We'll do a layoff at the end of my but don't worry. It doesn't actually it it it doesn't affect my results. That's the other thing I think it's like
• 57:20
  Adam Leventhal: If any if anything, it confirms them. Right?
• 57:23
  Bryan Cantrill: It confirms them. It's like, what a fortuitous layoff. It's like, do they consult you before or not? I just I mean, absolutely. It is so organize I mean, it's like forget the science.
• 57:35
  It is so organizationally deeply implausible. I mean, and obviously Ben, let me ask you like from a from your perspective as someone who understands like the the how these tools have evolved and as a material scientist, like what do you see as the kind of the promise for these tools to actually revolutionize material science? I mean, obviously a lot of things have revolutionized material science. I also love the idea that like when these guys talk about it, like no one else has ever thought about the material science. It's like, oh my god, like no one thinks about materials, but it's like actually plenty of people actually think a lot about materials actually.
• 58:11
  This is not Yeah, so for
• 58:13
  Ben Shindel: instance at Northwestern, the materials department, it's the largest graduate department in the engineering school. People are doing, it's getting attention, it's not like materials are like the unsung hero or something like that, tons of companies definitely respect materials research. So I thought it was funny that you kind of portrayed as, like, this, like, underdog feel.
• 58:34
  Bryan Cantrill: Dude, total total underdog. It's like like, no one's thought about it, and it's like, oh, it's given us, like, purified silicon for our integrated circuits. Like, it's giving us a hell of a lot more than that, by the way. Like, if you're anyway.
• 58:47
  Ben Shindel: Yeah. What was what was the original question? It was No.
• 58:51
  Bryan Cantrill: No. It just was like, how how much actual potential do you see there for these I e is this a domain that's gonna be revolutionized by And I've gotta be more specific. I mean, it gonna be revolutionized by LLMs? Is that what we're talking about? Is it gonna be revolutionized by GNNs?
• 59:06
  I mean, do you see this kind of this possibility?
• 59:09
  Ben Shindel: I think before the launch of IGBT, like people sort of thought that the most promising field for AI would be kind of like expanding what DeepFold did, which is just like going through the hard science fields and having it do this kind of work to generate new novel things or whatever. But I think LLMs have kind of changed the paradigm a lot, where now, I mean, they're making such faster strides in automating software engineers and that kind of stuff, that I think that this will be later and less important than that. But, you know, it's hard to say, and I don't really know. Can I ask
• 59:48
  Bryan Cantrill: a mundane question? Because like the thing that can I ask a mundane question? Because the things that LLMs are indisputably good at is understanding a lot of text really quickly. And as a material scientist, as any scientist, you're spending a lot of your time reading papers. Surely, I mean, it just feels like and I know this is not what AI proponents wanna hear, but, like, it feels to me like the most likely way that these disciplines are gonna be revolutionized is the same way we're all gonna be revolutionized.
• 1:00:17
  It's gonna allow you to consume a lot more information a lot better, a lot more precisely, ask questions of the research much quicker, be a better tool in your hands.
• 1:00:27
  Ben Shindel: Ironically, the like ease at which he made this, you know, super comprehensive, like beautifully polished paper with like nice figures, That's a testament to like the time saving, you know, the productivity advance of AI,
• 1:00:44
  Bryan Cantrill: like better than the findings of the paper itself in some ways. Okay. So are we gonna get a is Aiden Toner Rogers gonna write an if I did it paper that describes the the mechanics of but I'm sure I can make it make an note. Js reference. Maybe not.
• 1:01:02
  Adam Leventhal: Bring the bell at another extra reference. There we go.
• 1:01:06
  Ben Shindel: No. I know that.
• 1:01:08
  Bryan Cantrill: Well, we're down here at the home. We've got no idea who knows what. But it's like, yeah, I mean, it would be because you do kind of want a like, maybe this is his second career. This is where Adam buys the fabulous just because Aidan Turner Rogers brings us behind the workshop. Just kind of like a Mark Rober YouTube video for writing a fraudulent paper.
• 1:01:27
  Ben Shindel: Yeah. I mean, well, I feel like PhD students can have a lot to learn about how to use AI tools effectively in non fraudulent ways. Like for instance, this like, towards the end of my PhD, it's so much easier than writing like 100 lines of MATLAB code to like just to make a single plot. I can have it generate a program to plot my data for me, And that's like a massive time saving bonus. And obviously I think what he might have done is used, I think some, I don't know if these are matplotlib, but like, if you have the AI generate the data too, as well as the plot, just wholesale one chunk, then that'll maybe get you there.
• 1:02:07
  Bryan Cantrill: I don't
• 1:02:07
  Ben Shindel: know if he, like, did the data by hand and then had the AI help him plot it or if it was, you know, he had the AI generate the data and plot it. I don't really know how he did it, but
• 1:02:16
  Bryan Cantrill: I mean to something. Because, oh, let me tell you, Steven Glass did not have plots. Nobody was interested in how Steven Glass had done the mechanics of this. But you're right. It's like this guy could go on, like, a speaking tour.
• 1:02:28
  He was kinda like you have the guy you got, like, the school assembly where they got the guy coming in from prison or whatever trying to scare you straight. And he can come in and, like, scare you straight on the fraud and then, like, show you how to use an LLM to generate your your
• 1:02:40
  Adam Leventhal: You guys your your math plots. You're so right. This is such a topic for every university, you know, where where they worry about LLMs and the authenticity of students' work, and this guy can come in and make a killing on the speaker tour. Man,
• 1:02:55
  Bryan Cantrill: We've Hey, Dayton, Dayton, Rogers. If you're listening. Yeah. You've got We will book you. Will book you, but you've got to come clean.
• 1:03:04
  To like describe everything you did. Like we know you made all this shit up. So you're going to have to come completely clean. And okay, so when he goes on his university tour to lecture on, and I love the fact that Ben, I love it, it's a doubleheader. It's like a lecture on the dangers of fraud followed by a, let me help you on your MATLAB plotting.
• 1:03:26
  So it's a little community service. You walk him in in like leg irons and a jumpsuit? I think you got to, I think you've got to have like, he needs like the tier what what's the teardrop tattoo equivalent of a fraudulent PhD student? He needs I I mean, is he gonna have like the the different LLM models that he used tattooed on his knuckles? I mean, what's the what walk me through what this looks like.
• 1:03:52
  Adam Leventhal: No. I think you nailed it. I
• 1:03:54
  Bryan Cantrill: I think this is, like, I think this is his path. In fact, think this may be his only path. What does ChatGPT think about this? Telling you what to write.
• 1:04:01
  Adam Leventhal: Listen. Need to Second volume of my novel?
• 1:04:04
  Bryan Cantrill: Second volume. The novel's done very well. And people are I thought I had a lot of closure in the novel. People are really demanding previous escapades of this guy whose life I thought was ruined, but now he actually may may act He's got life.
• 1:04:17
  Adam Leventhal: Act. Yeah.
• 1:04:19
  Bryan Cantrill: A second act. And then what's the good thing I would just say is that the the thing that I'm also worried about it is because and, you know, you you kinda mentioned this mentioned them when contact changes minds or links to it because of the the advice that you got. The one thing I am a little bit worried about, Adam, is that the inventor of your take on because this thing has, like, escaped in to the the world of the media, there were people, like, may have seen the story but never see the retraction. You wonder that because this is like MMR and autism. Right?
• 1:04:51
  The MMR vaccine and autism starts with a fraudulent paper that has been retracted. And but that that's enough of a seed to kinda and I think that that when the contact changes minds, that piece also, like, people believe it cause it felt plausible. Ben, do you think this is something that you're gonna be dealing with like people will kind of assume this is fact because they didn't get the update?
• 1:05:14
  Ben Shindel: Maybe, although I feel like this fraud is like so interesting enough that and the paper was never even fully published, so it might be that the fraud was like more famous than the actual paper was when it came out. But like, I guess it honestly doesn't even matter because there's so much other results showing like huge productivity gains from AI that like, it might just, you know, the findings of the paper might be end up being kind of validated.
• 1:05:39
  Bryan Cantrill: Like Sam Bankman Fried's crypto holdings where like actually if you let me have As it turns out, like I wasn't insolvent after all. It's like actually I was right on all those things that this was it was entirely fraudulent. So I guess that's an important point is that from your perspective, the data is far too clean, but the results in the abstract that AI related tools would make material scientists more productive is not in and of itself earth shattering.
• 1:06:09
  Ben Shindel: Yeah, although I don't think it would like improve the patent filings by like 40%. That just seems like a lot. Those kinds of gains are massive, but also there are other claims in the paper are probably not going to be borne out, because those were the ones that were like more surprising to the other economists, which were things like the worst performers, sorry the best performers, or like the people who already were discovering the most materials, they had the most to gain from it, which is I think that one's contra in the literature already. But I mean,
• 1:06:43
  Bryan Cantrill: That was a surprising result. That was a surprising result that like the the best people did the best and the like these shitheads that shouldn't even be on this highly disciplined team of a thousand people, they're unable to make these tools do something for themselves. That is what you're saying is a surprising result.
• 1:07:04
  Ben Shindel: Yeah, seems, I guess it seems surprising to the economists. And know, I guess if I had to like predict what the outcome would be if this trial was actually carried out, this randomized trial at some company like Corning, I guess I don't know, like, if I would think, you know, what hypothesis I would have as to whether the worst performing or best performing researchers would have the most to gain from the AI. Like, it doesn't seem necessarily like it would be incorrect, but again, you'd have to, like, actually do the study to find that out.
• 1:07:33
  Adam Leventhal: Interesting idea. Yeah.
• 1:07:38
  Bryan Cantrill: Mechanically hard to do all that, right? Mechanically hard to figure out who are the best researchers and the worst researchers. Mean, Adam, you and I both know that like looking at patent filings is definitely reductive. Like you're not getting Because Ben, to your point about like that's only looking at one dimension of material science. It's not looking at many, all patents are equal.
• 1:08:03
  So it it like even just answering that question is really complicated. So yeah, it's very hard to do this kind of research and maybe they just like, he got sick of it and had the LLM do it, which you can read in his forthcoming book if I fabricated it. Well, Ben, this has been great. People should so describe the BS detector a little bit because I I'm I'm this is a a newsletter you kicked off. The I'm sure you got a lot of new subscribers coming in over this.
• 1:08:34
  Ben Shindel: Yeah, I did not expect this would go that viral, but a lot of people like the story. So I mean, yeah, thanks for having me on for sure. But the newsletter is just where I post things that I write.
• 1:08:46
  Bryan Cantrill: It's a more or less
• 1:08:47
  Ben Shindel: generalist blog, so it's not about any one topic in particular, but it's definitely about science and forecasting. I do a lot of forecasting as a hobby.
• 1:08:57
  Bryan Cantrill: That's awesome. Well, definitely got two new readers in us for sure, And it was a terrific piece. Adam, when I was going back and forth with Ben, I'm like, hey, so do you have like a Blue Sky? I don't really see you on Blue Sky. It's like, I haven't really logged into Blue Sky in a long time.
• 1:09:12
  Like you may want to go over there because it's kinda gone viral over there. Ben, were you surprised at the the the degree that people were talking about it on Blue Sky?
• 1:09:22
  Ben Shindel: Yeah. Well, because I know that, like, there's less there's just, like, numerically less users on Blue Sky than on Twitter, but, like, it was generating a lot of numbers on Blue Sky. Maybe someone famous had, you know, posted or something, but I was getting a lot of subscribers from Blue Sky. I noticed that after you brought it to
• 1:09:38
  Bryan Cantrill: my attention. That's great. Again, love reading it. Thank you so much for being willing to join us and walk us through this. We'll have to have you back on the next broadcast.
• 1:09:53
  This is not our last broadcast, Adam. We've got we've got just feels like we've got this is never great. We're we're we're gonna come back here. This is like, it doesn't matter what technology we invent. There will always be fraudsters and hucksters and everything else.
• 1:10:09
  There are certain ideas that are older than time. Yes. All right, well Ben thanks again. Adam, I'll be less reverby next week. So this was my last time down in Reverb Central.
• 1:10:23
  Exciting. Yeah, exactly. Looking forward to slightly better audio, but I still welcome all the unsolicited comments about bad audio. I love them all. Sure.
• 1:10:30
  All right. Thanks everyone. We'll talk to you next time.