Latest / Oxide and Friends / The Hardest Kind of Unsafe Rust

Transcript

The Hardest Kind of Unsafe Rust · Oxide and Friends

• 0:00
  Bryan Cantrill: Is it better?
• 0:01
  Adam Leventhal: Yeah. I think so.
• 0:04
  Bryan Cantrill: Usually, what you say is say more when I say it's better. Like, you you you don't give me you know what it reminds me of? Like, when you do you know anyone who's Danish? Have you ever tried to pronounce anything in Danish? So Danish is very hard to pronounce.
• 0:17
  Yes. Okay. I know. Alright. Fine.
• 0:19
  Alright. So Danish is very hard to pronounce. And Just like
• 0:22
  Adam Leventhal: furiously scribbling editing notes. Okay. Please continue.
• 0:24
  Bryan Cantrill: Yeah. Exactly. It's like, Am I cutting this? Cutting this again? No.
• 0:27
  And I'm it's like we just got done apologizing to and now we're all All of Germany. All of Germany. No. The no. Any Dane will tell you this.
• 0:37
  Danish is brutally hard to pronounce. And I don't know that I've ever pronounced anything correctly because my interaction with every Dane about pronouncing anything that I wanna pronounce in Danish is they say it, I say it, and then they say it again. And then I say it again, and then they say it again. And then I confused say it again and they say it again. I'm like, I'm getting no feet.
• 1:00
  Okay. So I'm doing it wrong, I guess. Like, I definitely like, I I I don't think I'm doing it correctly. I don't think you'd be doing this if I were doing it correctly. I think I must be doing it wrong.
• 1:09
  So I you know, I'm and I feel like it's like whenever I'm like, how do I sound now? It's always from you. It's always say more. And then I say more and then like we move on. And like, I know, okay.
• 1:18
  The audio is not good. I I guess I get it. I'm giving up. I guess.
• 1:21
  Adam Leventhal: You're reminding me this is this is how I felt like I went to pitch Ericsson years ago. You know, we're meeting with someone, and I said the name of the person. And the the receptionist just looked at me completely blankly, and I showed the person the name. They said the name, and then I said the name, and they said the name. And they're like, whatever.
• 1:37
  Fine. I I understand now who you wanna see. But I was like, I thought I was very close.
• 1:42
  Bryan Cantrill: But now okay. We are now gonna be apologizing to, like, the I'd like all Oh, okay. Maybe. Yeah. Exactly.
• 1:48
  I just wanna clarify. That was in Sweden, not Denmark. Do we know if there's separate countries?
• 1:52
  Adam Leventhal: I understand.
• 1:52
  Bryan Cantrill: Okay. Okay. I I mean I I was so it was it was another story about an unrelated separate sovereign country. I just wanna clarify that.
• 2:01
  Adam Leventhal: I'm not, like, saying let's take over Greenland. Okay? Like
• 2:04
  Rain Paharia: Right. Right.
• 2:05
  Bryan Cantrill: It's it's still you know, I was that's still it's it's an area of understandably heightened sensitivity. We would like to apologize to all of our listeners abroad. Yeah. We we should get going before we have to apologize for yet more countries. We're we're really just kind of like racking up the list of of folks we apologize to.
• 2:24
  Rain, welcome. It's great to have you back. I feel it's like I mean, is it fair to call you an Oxide and Friends regular? Do you do you take that as is that pejorative? Do you you
• 2:34
  Rain Paharia: I was, you know, I feel like I show up maybe two or three times a year. I don't know. Is that regular?
• 2:39
  Bryan Cantrill: Yeah. I think you're a bit of it like a cast member, I guess. I don't know. What are we strangely? Recurring character.
• 2:46
  Rain Paharia: Talk about yeah. Recurring character, guest star, whatever.
• 2:50
  Bryan Cantrill: Yeah. Yeah. That's right. That's right. The in the the extended Oxide cinematic universe.
• 2:55
  Yeah. And so I did Ray, you were joining us to talk about this blog entry that you were last week on IDD QD. Yep. And we've talked about IDD QD before.
• 3:06
  Rain Paharia: Yes.
• 3:07
  Bryan Cantrill: I did as I was just be and I don't know if this is a generation Mark or not. The Doom cheat code, Adam, do you like, you knew this as as the I know we talked about this last time.
• 3:17
  Adam Leventhal: Yes.
• 3:18
  Bryan Cantrill: Yeah. And like, the the this is bullseye for you. And and I'm like, what was it? A Castle Wolfenstein cheat code? I mean, otherwise, you know, that that would that would hit me.
• 3:27
  Otherwise, I and so I just wanted to Google I wanted to know the origins of of this as a cheat code. Oh, actually, as an aside and tie into the battle first episode, do you know what the battle first cheat code was?
• 3:37
  Adam Leventhal: Do you know what There's cheat code?
• 3:39
  Bryan Cantrill: No. There was. Yeah. There was. And I I you know, I may I may mention this last time so quickly that you didn't I was actually a little surprised that you didn't really latch on to this one.
• 3:48
  So you must have been You said must have
• 3:50
  Adam Leventhal: keys and that you tried to use them in our Yes. Okay. Battle. I didn't realize there was like there was It's
• 3:56
  Bryan Cantrill: a cheat code. Yeah. It's a cheat code. So I I just was like, I, you know, I feel like the the Wonder Woman truth lasso is upon me and I must tell you the truth that so the the keys are JK and l and it was m semicolon m in that order.
• 4:10
  Adam Leventhal: Okay.
• 4:11
  Bryan Cantrill: Which you the idea being that you could cheat with someone staring at you believing you were cheating.
• 4:17
  Adam Leventhal: Totally.
• 4:18
  Bryan Cantrill: So it made it very easy to cheat. And but we all kind of agreed that this is only to be used in extraordinary circumstances. And I felt that someone five years younger than you beating you in your own game was like, I'm like, if you're not gonna break the glass for this, like, when when are you using it? If not, I just acknowledge that you're never gonna use it if you're not gonna use it right now. And then of course, it didn't do anything because we disabled it.
• 4:41
  Damn it. So I wanted to know the or just the what the history of this is. And I guess the history of this is like, it's a kinda like the name of Rust itself. It's got like kinda multiple origins. And I was trolling myself because like the Google AI search I know it's bad.
• 4:58
  I know it's bad. It but it the it's so bad though. Doesn't it bother you how bad it is? Am I the only that gets bothered by how bad it is? Or is everyone else like, no.
• 5:08
  Of course, it's bad. Like, stop.
• 5:09
  Adam Leventhal: You still go to Google? I go directly to chat GPT.
• 5:12
  Rain Paharia: I'm just
• 5:12
  Bryan Cantrill: I I know. I know. I know. Exactly. Well, so I'm getting the AI so and I I noticed that, like, it, like, sometimes defaults to to, like, it's AI assistance.
• 5:21
  Although, actually, it's AI. Now that I go there, know. It does not seem maybe it's overheard me. And it's but I'm literally clicking reload on this thing, and it's giving me a different answer every time. And, like and actually, it's giving me a different answer every time right now too.
• 5:35
  So, yeah, this is like, well and I go like, what are you expecting? I don't know. Something
• 5:39
  Rain Paharia: You you click reload and chat your video or clutter and it gives you this different answer every time. I mean, you know, that's how these things work. And whether you wanna put it front and center in in the Google search box is I think, you know, a product question, but
• 5:55
  Bryan Cantrill: Right. So I'm kind of like getting frustrated with this. Rain's like, what are you doing? This is of course yes. This is how like, what are you doing?
• 6:01
  Just yes. Anyway. But so Rain, we talked about this crate this crate last time. And this is a so a really clutch crate for us. I want you to explain again what it does because it's a it it is really, really important for us.
• 6:15
  Yeah. And then in particular, this as a segue to to the work that you needed to do. And I assume by the way that the forcing function to this was the bug that you all that that was found in this and kind of fixing that. But I I wanna get to that in a second. Before we do, can we just just describe again for folks who didn't tune in or may have missed the last one about what this crate does and why it's so important for us?
• 6:37
  Rain Paharia: Yeah. So IDTKD is a Rust crate. It's on crates.io. You know, it's like an open source Rust crate, like so much of our open source work, and it maintains maps where keys get to borrow from values. And so, you know, if if you're like me, that might just glaze over your head.
• 6:54
  But, the the context for this is that, we, you know, we we build a large distributed system called Omicron. Right? And Omicron uses, CockroachDB as our storage layer and, you know, we often will, like, read these massive records from Omicron. Right? Like, so so we we read these, records, like, you know, for example, a sled inventory or a disk or something, And, what we wanna do is we want to index those, those records by, whatever primary key that there exists in the database or maybe some other some other key that makes sense.
• 7:33
  Right? So, for example, we may have, for sleds, we assign every sled a UUID, and so, you know, we we let you index by UUID. Now you can do the same thing with Rust's default maps. Right? So you can do something like, if you have, say, a sled inventory struct, then you can do, say, a b tree map of sled UUID to sled inventory.
• 7:59
  Now and and so that's how we did things for a very long time at Oxide. But one of the recurring issues we kept running into over and over again, and and we had a few bugs around this, was that the we you end up having to store two copies of the UID. So so you end up having to store the UID in the key and the UID in the value and that's how Rust's maps are structured where, you know, if you look at a hash map or a b tree map in the standard library, there is a key, you know, k comma v, that's how it works. And so there's a few different ways to approach that with if you just use the Rust standard library, one way to approach that is to duplicate the key where, you know, you you store the key twice, and so once you store it in the key, once you store it in the value, that is okay but, you know, it has downsides, the big one just being that those two can fall out of sync. Another approach is you kind of don't store the value don't store the key in the value and then you pass around the tuple everywhere.
• 8:56
  That also is pretty inconvenient, because you're you're passing on a tuple or maybe some wrapper around it or whatever. So we, so, you know, on the update team, which is which is the team that was kind of doing a lot of this work, we we end up having conversation and we were like, okay, what we really want is a pattern where you store rather than storing keys and values separately, you store records. And the user gets to define a way to extract a key from a record. And so my coworker John Gallagher actually, who's also been on the podcast a few times, he he wrote an initial version of that and that was really cool, but but there was an issue that we ran into there which was that the we required that the key we we required that the key implement clone. So so clone is a Rust trait which basically means that you have to create a copy.
• 9:54
  So so the way it worked is kind of you, you know, you have a key and so you duplicate the key and and so what what John's implementation was doing was that it was duplicating the key twice. But it did prove its worth. Right? Like, we we ended up using this pattern in a bunch of places and it made a lot of sense. It did have this limitation.
• 10:11
  So then kind of John
• 10:12
  Bryan Cantrill: And Rain, fair to say that that kind of that stage is kind of is doing what we were doing manually, but now doing it in a way that that you got more guaranteed robustness effectively because you're assuring that these things either you're you're taking some of this latitude out of the way the programmer would programmers don't want this latitude anyway. Programmer wants these two things to be in sync.
• 10:31
  Rain Paharia: Right.
• 10:32
  Bryan Cantrill: Right. But it it it has the the minus that it's relying on cloning it.
• 10:36
  Rain Paharia: A cloning
• 10:36
  Bryan Cantrill: Exactly. It into the into the cave.
• 10:39
  Rain Paharia: Right. Right. And so, you know, in in some cases, that was fine. Right? Like, if it's a UID, UID 16 bytes, whatever.
• 10:45
  Right? Like, you can you can copy it. But then there were some other cases where, I guess, in, like, databases, you might call it, like, a natural key you you have a key that's formed out of like the three or four fields, you're not using a UID, but rather you're kind of using some combination of of a few fields to do that. And so so John kinda gave me this challenge basically, and he was like, hey, can we can we do this in a way that we borrow, we rather than needing this clone bound, can we borrow the the key from the value? And so sometime last year, kind of, I started exploring a few designs while, you mostly just while waiting on reviews and stuff from people.
• 11:24
  And and so after a few weeks of work, ended up finding something that actually worked. And then, you know, the last 90% is kind of what what, turned into IDDQT. So that's the that's the context of how it came about and why it's really useful. It's just, like, such a common pattern, especially for, like, the kinds of fat records that you end up storing. So, you know, we we still use like, it's not that we've switched every single map in Omicron over to IDDKD.
• 11:52
  Right? There are a lot of maps where, you know, it's like some, like, u three two or whatever, and it's fine. But it it like, there are surprising number of cases where it just makes a lot of sense to express things in this manner. So, yeah, that's kind of the context for IDDKT and and how it came about.
• 12:09
  Bryan Cantrill: Okay. So then why does and then this has to be unsafe though, or rather the the the need we we you can't do this strictly in safe rust. It's Do wanna ex did explain why that this can't be done, at least can't be done in a way that performs well in in strictly safe rust?
• 12:27
  Rain Paharia: Yeah. So the, the version that has the clone bound, that can actually be done in SafeRust. And and the reason that can be done in SafeRust is because, it effectively delegates all the unsafety to the Rust standard library maps. So there's like a a couple of fine points here, but but like what happens is that when you are kind of not using the standard library mechanisms and rather you were you were so, okay. So I should mention that folks, if they're interested, they should have a look at how the standard library, hash map and Btree map work and especially Btree map because there is a huge amount of unsafe for us within standard library beaching map.
• 13:12
  And I think, Bryan, you've talked a bunch about how how important it is to have that kind of primitive as as in available in your in the standard library for performance.
• 13:22
  Bryan Cantrill: Oh. It was and I guess how you rate I mean, know you and I talked about this in the past, but it was to me, it was I was like, how is this working? Like, when I was for like, a b tree is really gnarly to implement in c. Like, so gnarly that people basically don't implement it because it requires the movement of a of data structures. It requires, like, anybody that has a pointer to the old thing.
• 13:45
  It's like, yep. You you like, that that's no longer a valid pointer because that's the whole point. And I'm like, how am I able to do this safely? And I'm like, this is and then I had no idea, of course, like, the rocket the rocket science in B Tree is I don't know you ever looked at the implementation. It is it is amazing.
• 14:05
  I and it was to me, what it was like a real watershed moment for me in Rossins like in 2018, where you realize that like, wow, there's this amazing level of sophistication in the implementation of the beach remap. And because of all that sophistication, everybody else can use it safely. Like only the that implementation, like, hasn't changed that much right now. I mean, at least when I you know, it's kind of like that code kind of is correct. Well, is
• 14:38
  Rain Paharia: have been a few issues, but, yeah, for the most part. Yeah. And it's interesting because a lot of the issues that have happened have happened along the same lines as some of the unsafe issues that I talk about.
• 14:48
  Bryan Cantrill: Oh, interesting.
• 14:48
  Rain Paharia: So Yeah. So, you know, this is kind of a shared pain that anyone dealing with unsafe rust or or what I call the hardest kind of unsafe rust has to deal with because the rust beachy map also is the hardest kind of unsafe rust in exactly the same way.
• 15:01
  Adam Leventhal: Before we get too far, I I just I I thought your blog post had a great discussion of unsafe rust because I think sometimes when people hear unsafe rust, they think, wait a minute. Rust is supposed to be safe, and now you're just disabling safeties and yellowing it. But that's not at all what unsafe rust is.
• 15:18
  Rain Paharia: Correct. So I think this is this is a common and and what I think is a is a pretty frustrating understand misunderstanding of of the the point of unsafe rust. So and I I talk about this in the blog post, but, you know, for the most part, like, Rust is kind of amazing because the Rust compiler, like, upholds all of these invariants for you where it makes sure that, you know, you're not aliasing like, you don't have mutable aliases to the same data at the same time and and all of these things. But there are, like and so and and this is like I mentioned this as a fundamental undecidability result called Rice's theorem. There it is impossible for any for for the Rust compiler or or really any algorithm that terminates, right, to accept all programs that are, without undefined behavior which is defined by the Rust language and reject all programs with it.
• 16:18
  So it's kind of impossible to have a perfect compiler that perfectly answers whether a program is, like, has undefined behavior or not.
• 16:30
  Bryan Cantrill: And this is so and I actually had not heard of Rice's theorem because I'm like, isn't this this is just the halting
• 16:36
  Rain Paharia: it is a halting problem, isn't it? Yes. Yes. It is. The Rice's theorem, it's a it's a it's a fairly small proof to show that Rice's theorem and halting problem are basically the same thing.
• 16:45
  Yeah.
• 16:46
  Bryan Cantrill: Yeah. Yeah.
• 16:47
  Rain Paharia: I mean,
• 16:47
  Bryan Cantrill: you gotta think, isn't Turing clearing his throat a little bit around Yes.
• 16:50
  Rain Paharia: I mean, I'll You'll
• 16:50
  Bryan Cantrill: be like, yo, dude. I mean, it's like and I guess it's like, it's the it's the fifties and, like, Dijkstra's getting an algorithm for something. It's like, come on. Is that really wait. That's innate.
• 16:59
  Alright. You know? Alright.
• 17:00
  Rain Paharia: It's the, you know, the infancy of computer science. Right?
• 17:03
  Bryan Cantrill: It is. Yeah. It is. It was it was easy days. Yeah.
• 17:06
  Rain Paharia: Yeah. Exactly. Right? So, you know, back in back in the those days, you could just, like, make a small, like, extension of the halting model. You'd have a theorem named after you.
• 17:14
  Bryan Cantrill: That's right.
• 17:15
  Rain Paharia: Yeah. But but, yeah, it is the halting problem. And and and so the the thing is that, like, a lot of people kind of you know, they look at the halting problem or they look at Rice's theorem, and they're like, oh, this this whole, like, line of work is useless. But, actually, halting problem that writes this theorem don't say anything about approximations. Right?
• 17:34
  And so Interesting. And so what Rust shows is that it's going to, its its compiler is going to be stricter than perfect. And so what that means is that, the compiler within the context of SafeRust is going to accept, is going to accept sorry. It's gonna reject all programs that have undefined behavior and it is also gonna reject some programs that don't have undefined behavior. And so, Unsafe is an escape hatch for kind of if you're in that middle area where you know your program doesn't have undefined behavior, but you can but the Rust compiler cannot prove it, so it is then up to the human to establish that.
• 18:19
  And so so Unsafe is meant to be the escape hatch for those kinds of situations. And now, you know, I talk about, like, Unsafe Rust, is just pure Rust, but it also applies to things like CFFI. Right? Like, the Rust compiler doesn't have visibility into c code, so if you're doing CFFI, then you kind of, you know, you're like, okay, I'm upholding the invariance here. The Rust compiler show that.
• 18:42
  It also applies to, you know, when you're doing hardware access or things like that, right, things that are completely outside the domain of what the Rust compiler models, it just assumes that, you know, it'll reject that by default but it has the unsafe escape hatch to let you accept to let you kind of, you know, play around with that.
• 19:01
  Bryan Cantrill: And, Reed, do you happen to know the history of unsafe? Well, got two questions. One, is there a precedent in I mean, I feel like so much of Rust draws inspiration in the best way from other languages, but I don't know of Unsafe. I mean, is is there a precedent elsewhere
• 19:17
  Rain Paharia: Yeah. Yeah. I think
• 19:18
  Bryan Cantrill: for Unsafe?
• 19:19
  Rain Paharia: Yeah. So I think I I wanna say the first major version of Unsafe comes from Haskell. So Haskell has gonna say, this is
• 19:27
  Bryan Cantrill: gonna be Haskell Haskell, Simula, or clean. It's gonna be like I feel yeah. I'm sorry. Yeah. Haskell has this.
• 19:34
  Rain Paharia: Yeah. I don't know I don't know enough about the other two, but I do know that Haskell has this thing called Unsafe Perform IO. So if, I can give a little background on that. So, you know, if if you've heard of Haskell, you're like, you know, it's this pure functional language. Right?
• 19:47
  Like, the idea is that everything is just an input and the output is exactly this you you run it multiple times, you get the same output each time and all those things. Right? And and Haskell has a lot of infrastructure around that. There's like monads and and all of the other infrastructure to let you have side effects in your program. But sometimes, you want to just kind of, you know, poke at memory from inside while not having users have to deal with the monad or whatever outside.
• 20:15
  And so you may want to add some like logging or something to to just do some debugging thing or something like that. And so Haskell has this concept called Unsafe Perform IO and there's a few different versions. Unsafe Perform IO is most relaxed version and then there's like some that have maybe a little bit less power. But what all of them basically let you do is that they let you break the type system in a controlled fashion in this exact way. And so Haskell has that, I believe C also has some version of this, though I haven't looked too deeply into it.
• 20:48
  I guess I guess that I I do have kind of a little bit of a larger view, is that I feel like often the things that in in memory safe languages, if you wanna do unsafe stuff, you end up writing it in a different language. So for example, for Java, you know, like, you you have your Java is a purely safe language, but, if you wanna do, like, native calls, then you have, like, this JNI extension that you write in c or c plus plus or or whatever other language. Right? And so I think Rust is kind of I wanna say it is the first systems language to bring that Haskell idea of unsafety and let you write it in the same language. And and so let you write unsafe and safe in the same language.
• 21:27
  And so I think that's what makes Rust distinct.
• 21:30
  Bryan Cantrill: Wait. Which is huge because, I mean, I think and Adam, you you did you you were able to you were just a conscientious objector to all of, like, my node phase. My my
• 21:39
  Adam Leventhal: Yeah. Pretty much.
• 21:40
  Bryan Cantrill: More or less, like that was just yeah. You I I thank you. It was so it was so generous of you.
• 21:45
  Adam Leventhal: I knew it was
• 21:46
  Bryan Cantrill: a phase. Yeah. Was a good you grew out of it. You thank you. You you had so much confidence.
• 21:51
  The the unmerited, but I'm I'm glad you had confidence and grow out of it, which obviously did. But the so in in it was v eight was all in c plus plus. And if you wanted to do anything like that, like, were dropping down into this like, into the c you're basically augmenting the run time effectively. And I mean, you can just talk about going from rail to rail. Like, unsafe is not the word for this.
• 22:12
  This is I mean, this is so actively dangerous. And it's so easy to screw up in so many different ways. And so and like, broken from version to version. It's just like it's not a there it is not a middle ground at all. And I I kinda love that.
• 22:27
  Mean, I do love that about Rust that you've got this.
• 22:29
  Rain Paharia: You know, I think there is kind of another, like, subtle point here, which is, like, a competency point. Right? Like, you, it's hard to expect someone who writes Java or JavaScript most of the time to kind of make that leap and write, you know, this really deep unsafe c code all of a sudden. Right? And, you know, maybe maybe have to, like, go through extensive training, maybe have to go reviews, all of these things.
• 22:52
  Right? And I feel like the in Rust, the on ramp is a little bit smoother. Right? It is still very hard, and I talk about why it is very difficult, but you are still writing the same language, and, you know, you you can try and do all this encapsulation work and so on. So it is I feel like it is a skill that for a for a developer who is comfortable writing SafeRust, getting to write on SafeRust is, I would say, a little bit easier than it is for, say, a Java or JavaScript developer to, you know, jump and write C if they've never written C before.
• 23:22
  Bryan Cantrill: You know, you say that, but having been very few times that I've really had to write Unsafe Rust other than dealing with memory mapped IO, like I'm just discounting that, where it's like, okay, that's in in the embedded context hubris, we have to do this all the time. Yeah. And you you do end up with these little unsafe blocks that are effectively reading from a magical address, but like that's fine. I mean, those are very, very bounded. You're not doing anything like really sophisticated.
• 23:47
  Yeah. When when I actually had to write UnsafeRust to deal with a very, very bad library interface. It it with a native binding. And I was like, was having shortness of breath as I was writing it because it's like, I feel like the the when you I mean, when especially coming from c where you know that all of c is effectively unsafe.
• 24:09
  Rain Paharia: Right.
• 24:10
  Bryan Cantrill: And now, like, Rust gives you so much. And now Rust is like, okay. Yeah. Like, you got I'm I'm fine. Yep.
• 24:16
  Go ahead. Just like I I don't just don't call me for help, basically. And you're just I found that, like, I am very nervous writing. I think appropriately so. I mean, Adam, do you have the same have you heard much on SaveRust?
• 24:29
  Or you
• 24:30
  Adam Leventhal: Not much, but some. And I definitely got it wrong. And I think that it is I and when you say this in the blog post, I quite agree that it is harder than, like, writing c. Like Yes. In that Yes.
• 24:41
  Because more is going to there there are there is sort of more expected of you. Like, the the compiler says, you know, you're you're welcome to give it a shot, but I really expect you to uphold all of these principles, not just the ones that you're aware of. So for example, Bryan,
• 24:58
  Rain Paharia: we know in the in the
• 25:00
  Adam Leventhal: in the USTT crate that that Ben and I wrote to expose you DTrace USTT probes through Rust. We had gotten something wrong where basically we we we did some stuff out of order where the Rust compiler quite rightly decided to, you know, freeze some memory that we thought was still hanging around and was no longer hanging around. Yeah. Which so, you know, anyway, easy easy to do.
• 25:28
  Bryan Cantrill: Yeah. Hard to debug, though. How did you debug that?
• 25:32
  Adam Leventhal: I don't remember. I mean, it was I don't remember whether it was, like, some miry thing that someone found or whether, you know, on inspection you know, what once you have the actually, you know, I think it was basically from inspection. The the result was effectively, we had two different you know, one of the weird things that The US t two provider does is it will serialize objects into JSON, which is very handy, but a little weird.
• 26:00
  Rain Paharia: I it. I love it.
• 26:01
  Adam Leventhal: No. No. I love it, and it's but it is weird. And it only does it with the probes enabled. And we were we had two different objects, but we were getting the same data or, like, the same data, but offset a little for both of them.
• 26:14
  It So was a pretty big clue that something was getting overwritten.
• 26:19
  Bryan Cantrill: Oh, interesting. Yeah. Yeah.
• 26:21
  Adam Leventhal: Yeah. Anyway, yes, a little bit. And and and as I said, I got it wrong the first time.
• 26:26
  Bryan Cantrill: Got it wrong. And boy, because as you say, mean, for the reason because I think this is what makes me so nervous about when I again, very few times I've had to do it. It is harder than writing c because it's not like your responsibility is more than just don't crash, which I feel is like your responsibility when writing c. It's like, okay. I got it.
• 26:46
  Like, can manage my own memory. I cannot crash. Yeah. But with Russ, it's like, well, no. You you no.
• 26:50
  You've said that like you don't you wanna be unsafe here. So you you need to adhere to all of my rules as don't know the way you phrase it. Like, all of my rules, including the ones you don't know about.
• 27:01
  Adam Leventhal: Like, especially those rules.
• 27:03
  Bryan Cantrill: Especially those rules. Yeah. And obviously, like, I mean, I you know, the first rule of the unwritten rules is like, I can't write down. I mean, that obviously. But but I mean, I am gonna and I'm I'm being unfair to Rust because there are actually rules.
• 27:14
  Rain, you wanna describe, like, what are the the rules of of writing Unsafe Rust?
• 27:21
  Rain Paharia: Yeah. So so this is a little bit tricky because the exact rules of writing UnsafeRust have not actually been formalized yet, and this is an ongoing discussion about exactly what the rules are. But there are some things that I think everyone agrees on. Right? So so, you know, the exact boundaries of of what's safe and not are are being worked on actively, and and Ralph Young and he has a a big project, and they they published a bunch of papers at, like, POPL and PLDI and so on.
• 27:52
  So so, this is, a big research grade project, but there are a few things that everyone agrees on, and, like, you know, they they are clearly, like, if you do these things, you've clearly screwed up. Right? So I think, you know, the the most obvious one is that there are no data races, And so, you know, you cannot have two bits of memory or two different threads accessing the same data at at the same time. That's just not allowed. Another one is that you you should not have ever any reads of uninitialized or freed memory.
• 28:22
  I feel like this is every like, in c, you know, this is also not correct. Right? Like, you know, you should not read past the end of an array and so on, but c helps you a little bit less with that than Rust does. It with Rust unsafe, you can potentially read past the end of an array, or you can read memory that's been freed as as Adam, I guess, discovered. And and so, you know, that that is another rule of unsafe Rust that everyone agrees on that this is not a rule that you should be like, you should not do this.
• 28:48
  Right? I think the one that bites people the most and and especially coming from c, think, is that in in Rust, you should never be able to have mutable references to the same region of memory at the same time, like, even exist, even if they are not actively accessed, and and that is something that's really important. If you're in if you've if you've written c, you may be familiar with the restrict keyword. And and and what the restrict keyword does is basically the the, developer is promising to the compiler that, this particular pointer has no other aliases to it. Right?
• 29:32
  And so what the compiler can do then is that, okay, it can know that based on this restrict that, you know, if you're if you're fetching, like, something dereferencing that variable over and over again, then you don't have to, like, load that memory over and over again. Instead, you can load it once and you can cache that. Right?
• 29:48
  Bryan Cantrill: Now importantly, Arin, this is only an optimization. This is like a hint for the compiler to generate faster code. This is not about correct us. Right?
• 29:55
  Rain Paharia: Yes. Yes. And this is this is a hint. But if you get it wrong, then the compiler will screw up. Right?
• 30:00
  And so Right. Right. Right. Right. And and so what that means is that no one actually uses restrict instead.
• 30:05
  Bryan Cantrill: Right. Right.
• 30:06
  Rain Paharia: Because it is Because it
• 30:07
  Bryan Cantrill: is Because no one felt like actually debugging those compiler bugs and or the compilers had supported and or it wasn't a win and or, you know, the etcetera etcetera etcetera. Yeah. And the the consequences of getting it wrong are just too grave.
• 30:17
  Rain Paharia: Right. Right. And and, you know, yeah, you'd have, like, stale reads and and all of the issues that you can imagine with that. Now in Rust, what the Rust compiler does is that it tags every like, when it compiles it down to LLDM's what it it tags every ampersand mute reference with the equivalent to restrict. Right?
• 30:37
  And so what the Rust compiler is telling LLVM is that everything that's a a percent mute is actually restrict so that you can you don't you can assume at at the LLVM stage that there are no other references to this memory, so it can do all of these caching things and so on. And and getting that enabled is its own journey, which I'm not gonna get into, but, like, there were bugs in LLVM that were found by the Rust compiler doing this. My god. Because I
• 31:00
  Bryan Cantrill: can imagine.
• 31:01
  Rain Paharia: Because, like, no one else did it. Right? So this scope
• 31:03
  Adam Leventhal: was really
• 31:04
  Rain Paharia: under tested, so it took years to enable this feature. So so but, like, in Unsafe Rust, in in Safe Rust alone, you cannot do this. You cannot create multiple ampersand mute accesses or or you cannot have mute references. But in Unsafe Rust, you can do that. And and so the bug I walked through in the blog post is actually one where I did exactly that.
• 31:25
  And so and so, you know, that's that's I feel like I have found reasoning about that to be so difficult because it is so easy to screw this up and the failure modes are so bad because they'll, you know, happen under highly optimized circumstances in release mode or whatever. And so then you're looking at assembly and you're like, okay, why is this not reading? You kind of have to figure it out, right?
• 31:50
  Bryan Cantrill: The so how did this bug that you that you found with this, was that the kind of origin of the actual blog post where you found the bug and then fixed that and realized that, like, okay. I gotta actually explain how we've done some of this. But
• 32:03
  Rain Paharia: Yeah. Well, the the so there are a few things that happened at the same time. So so one of them was that, you know, I I found this bug, and and I can go in a little bit deeper into some of the context where, you know, I I have to talk about this adversarial code and so on. But and and so so there's kind of one of these things was that I found this bug by looking at IDKD. Another thing is that there's this, like, rewrite of this famous Rust of the famous Zig project to Rust, and that rewrite strolled the hell out of me, and I was like, just like in despair for days about that.
• 32:39
  And and so I was like, okay, you know, I feel like it's kind of important to communicate a message of, okay, here is how to think about these things, like, rigorously and correctly. So just all of these things happen, there's a confluence of events and both of these things happen at the same time. I was like, okay, sure, I think it's worth writing this up. Also because I think it's an interesting narrative and I think it's just really interesting material.
• 33:03
  Bryan Cantrill: Okay. So just because we're here, do feel like we we gotta talk a little bit about the the the bun rewrite. And I'm sorry to troll you yet again. And I I don't know. I feel like we need we need another chime when we are we are trolling people with, like, Internet weather, you know, where they kinda because I I think I you go back to this, like, you go if we go back and relisten to an episode from, like, two years ago, and we are, like, very worked up about and, of course, I mean, like, you know, read the tweet, guys.
• 33:28
  They they were very worked up about something, but, like, it takes a while to figure out, like, what is oh, write this. Right. Right. Right. A cat.
• 33:33
  Right. The date silver piece or whatever. This is not bad. Yeah. So the just for our future selves, the Bunn, which was famously I think famously, right, Rain written in Zig?
• 33:45
  I feel like Bunn was written
• 33:46
  Rain Paharia: in a project for Zig. Yeah.
• 33:48
  Bryan Cantrill: A headline project for Zig. Yeah. And Bun was rewritten in Rust and very much passive voice. A lot of were they using Meetos for that? Or are they using the the they using advanced models at Anthropic to
• 34:06
  Adam Leventhal: I important to note acquired and then rewritten.
• 34:09
  Bryan Cantrill: Right? Acquired.
• 34:09
  Adam Leventhal: That seems very important.
• 34:11
  Bryan Cantrill: It is very important. Yes. Acquired in the, like, in in the m and a sense, the mergers and acquisitions sense Yeah. That the company behind Bunn was acquired. And then the there was a PR floated to rewrite it in Rust Yes.
• 34:29
  Which like, okay. And I mean, I just feel like I mean, is there a bigger possible fuck you to your community than they'll have a PR that changes the language? It just feels like, okay. Okay. I guess my outs okay.
• 34:45
  So I guess I have merge conflicts, I guess, with this with this PR. Because I guess my work is no I have to figure out how to write it in a different language, I suppose. But the and then there was like, well, don't know. Like, this is we're not actually doing this. This is just like what it might look like.
• 35:01
  This is like, this is like OJ Simpson if I rewrote it. Yeah. The and then like a week later, it's like, actually, we're smudging that. You're like, okay. So the and then part of the I mean, aside from all of the kind of antisocial behavior of that, the one of the things that and, Rain, I'm sure this is like trolling you in many different dimensions, but one of the dimensions was people were going into the the the actual rust that was generated by this thing.
• 35:28
  Yeah. And there there was a lot of unsafe rust in here. In part because it was kind of like I mean, transliterating it is probably being overly pejorative. But it's taking it from Zig, a language that has got different constraints
• 35:39
  Rain Paharia: Right.
• 35:39
  Bryan Cantrill: And recasting it as in Rust without, like, exactly rethinking it. Is that a fair
• 35:46
  Rain Paharia: Yeah. And and, you know, in to Steelman, their kind of side of things. Right? Usually, like, I think it is a fair way to do a rewrite. Right?
• 35:54
  Like, I think that, you know, doing kind of a a transliteration before you, like, do that as one phase and then do, you know, the conversion to more idiomatic constructs in, like, another phase. I think that is a perfectly defensible way to do a rewrite. I think the the thing that trolled me was that there were, like, several things that trolled me. I mean, one of them was that, again, it was done in nine days. It's a million lines of code.
• 36:19
  No one's looked at this code. To to your point from earlier, apparently, they stopped taking PRs a while ago. So, you know, they were already kind of doing the sequel like model Right. A little bit. So
• 36:30
  Bryan Cantrill: Wait. Wait. Your community must be outraged. What community? We do they yeah.
• 36:34
  Adam Leventhal: Oh, okay. The one they burned down.
• 36:35
  Bryan Cantrill: The one we burned down. Oh, okay. Yeah. Guess that's right. There we Good point.
• 36:39
  Fair.
• 36:40
  Rain Paharia: But I think also, it was kind of I I feel like it was, I think, you know, part of kind of the rust and and unsafe rust is that there's, like, the social contract, right, where people who are writing Unsafe Rust have this tremendous responsibility Yeah. To make sure that people who are doing safe Rust are you know, have an easy life, and and not have to debug all these issues. And I feel like it was kind of, at least to my standards, it wasn't quite at that level of responsibility and so that was a big part of what was strolling me and so this was, yeah, so I felt like I had never written, like, what I think it means to do Unsafe Rust responsibly. So Yeah. See.
• 37:25
  This was the jumping board for that.
• 37:27
  Bryan Cantrill: It is so fair to say, like, the thing that's upsetting there is the kind of casualness of the Unsafe Rust. There's just that that that really whenever you write UnsafeRust
• 37:37
  Rain Paharia: Yeah.
• 37:38
  Bryan Cantrill: You that you really because you are now you're discarding so many assurances of the compiler. You've got this really, as you say, very serious responsibility to those that are that that are are going to be writing SafeRust.
• 37:50
  Rain Paharia: Right.
• 37:50
  Bryan Cantrill: And the consequences of getting it wrong are really very grave. And this and you're probably and you're coming off this recent example that sounds like, I mean, if you found it by Gadank and experiment or or code inspection, great. But boy, if you hadn't found it that way, it'd be a brutal problem to find because we don't expect to have those kinds of problems with rust. Right. Right.
• 38:11
  Exactly. Right? Interesting.
• 38:13
  Rain Paharia: Yeah. And and I do wanna say here that, you know, IDDKT, like, even the first few versions of that, they were fine as long as, the user code that you wrote was correct. Right? And so so a lot of the bugs that I found recently have been bugs where the user code was buggy in some way. Right?
• 38:33
  And that and that bugginess got amplified into, the rules of Rust being violated. And so bar and and so this is where I talk about some of the hardest kind of unsafe Rust is that when you have unsafe Rust that depends on user code for correctness, it is extremely important that if the user writes a bug that that doesn't translate into the rules of Rust being broken. It is okay if if you end up being buggy as a result. That is fine. But it should not end up in the rules of rust being broken.
• 39:06
  And so so that's kind of, you know, that's the context for a lot of that.
• 39:10
  Bryan Cantrill: Yeah. Okay. So let's talk about that because and this is I think this is and this is why it I mean, as you say, this is why it's the hardest kind of unsafe rust. The hardest kind of unsafe rust is where your are ramifications for user authored code and in you need to be able to accommodate in an incorrect program from SafeRust still has bounds about what the actual manifestation of that incorrectness is allowed to be. Correct.
• 39:43
  So so what are some of those those those bounds? Because I think this is what makes this really, really gnarly.
• 39:51
  Rain Paharia: So so looking at the map types. Right? And so, you know, the map types implement hash and or or or, you know, these traits, right, which let you, hash a value, compare two values. These are all, like, traits that are provided by the Rust language. And the Rust language says that, okay, you know, you have to, you can implement these traits and the Rust language has contracts for here's how you should implement these traits.
• 40:14
  But in you don't have to obey those contracts because the user can just write code. So the user can do things like flip around the sign of a comparator randomly. Right? Like, the user can just, like, call into the system RNG and do that. The user can hash the same key to different values, you know, again, like flip a coin and and return one value sometime and return, like, a a zero value some other time.
• 40:37
  And so so unsafe rust can or or safe user code, which calls into unsafe rust, can behave in these arbitrarily pathological ways. And, and yeah. And so if you're if you're dealing with Unsafe Rust, you actually and and the way I think of it is, like, you're kind of in in more of, like, an infosec mindset more than a software engineering mindset where you're like, okay, how can an attacker get in here? How can how can an attacker kind of screw up my assumptions on things? And so that's that's what leads to that's what leads to all of these challenges.
• 41:15
  Bryan Cantrill: When I love the fact that you you linked to the Rustnomicon here Yeah. On on the fundamental principle. I I I love the Rustnomicon for its like, it's gotta be one of the best prefaces of all time. I mean, Adam, did you ever have you gone into the Rust Omicron where it where it's basically, like, warning you to put the book down? Like, this is actually, like, literally Pandora's box that you can't that that the the document itself is incomplete, that the the we're gonna take you into the dark arts like you're gonna you are gonna I mean, your metaphor.
• 41:49
  You're gonna eat the the the fruit of the tree of knowledge of good and evil like you are you basically and no one
• 41:56
  Adam Leventhal: To be forever changed.
• 41:57
  Bryan Cantrill: You'll be forever changed and and only you will can accept responsibility for the for the dark things you learn from I just I I just love it. You've got the I mean, it again. It's like there's so many different folkways that have the idea of, know, the monkey's paw or Pandora's box or the golden apple of Discord or the the you know, it's great. Anyway, I think it's I I I love that you link to it because I mean, the rest of the Omnicon has been around for a long I mean, as I I remember because I think you go I've and there's a great series of blog entries on so you really wanna write a doubly linked list.
• 42:30
  Rain Paharia: Yeah. Yeah.
• 42:31
  Bryan Cantrill: Which I went through what I feel is a very age appropriate phase with respect to Rust, where I thought I really wanted to write a doubly linked list. And the blog entry that is trying to talk you out of it at every step, and you're in the Rust Omicron trying to figure out. And then you finally realize that, like, I do not wanna write a doubly linked list. I anyway, I was glad to see you linked to that.
• 42:52
  Rain Paharia: I think it's I think it is the main resource for Unsafe Rust. And if you're if you are writing Unsafe Rust and if you're embarking on the journey that I did with don't do it. Right? But if you'd want if you decide to do it, this is like, you know, you don't don't smoke weed, but if you decide to, then then I think it is worth really worth reading that book top to bottom and really internalizing a lot of a lot of the ideas there. Like, there there is so much here in in the Rustenomicron that I didn't touch on or, like, I didn't directly interact with, but I am aware of, and, like, you know, there are, like, things that I know will, like, raise warning bells for me if I look at it.
• 43:30
  And and I and I'm like, okay. Well, are you thinking about variants? Thinking about a drop checker? There are so many things that that you if as as a safe rust user, you probably never encounter, but it's like there is a lot of machinery in the background making sure that your your safe rust is reasonably done.
• 43:49
  Bryan Cantrill: It it actually, the as an aside, do you recommend that folks dive in the Rust omicron just to appreciate what Rust is already doing for them? It feels like you can learn a lot about even as a as a strictly safe Rust user
• 44:02
  Rain Paharia: Yeah.
• 44:02
  Bryan Cantrill: You can learn a lot about what these things actually are. And then when you actually see I mean, the first time I saw a transmute, I was like, oh my god. You can just do that? I mean, it will holy god. I mean, this is like unsafe open brace.
• 44:16
  Let's party. We're gonna if you're like, holy smokers, we're gonna actually just we can just do that. It was a it was a real surprise.
• 44:25
  Rain Paharia: Yeah. There's a couple of things I would recommend reading. I think I would recommend learning about subtyping and variants. I think that is part of Rust that shows up in SafeRust occasionally, but it's, like, always surprising when it does. And I would I would recommend learning a bit about that.
• 44:41
  I'm just looking at the the Rustonomicon right now, and that stands out. I would also recommend reading about zero size types and how that is especially tricky. So so, you know, in this is actually interesting because it also tells you so in Rust, if you declare an empty struct, right, then that struct has no runtime representation in in memory. So it is it is a zero it is what is called a zero size type. Now c doesn't have zero size types, and so if you don't have an empty struct or whatever, then then in c, you know, that will always be, I think, at least one byte.
• 45:18
  I I don't remember the exact details here. But with Rust, because you have these zero size types, Unsafe Rust that is generic over the kinds of structures that have to store zero size types versus not has to be written very carefully to account for the existence of zero size types because there are certain other assumptions that break down around zero size types, like malloc on zero is it's own can of worms around, I know Windows has its own behavior around that stuff. There are lots of other subtleties. So like if you look at the standard vec code in the standard library, then that is very carefully written to deal with both zero sized and non zero sized types because you can store a vec of like some empty, like a like a unit type. Right?
• 46:08
  You can do that. And in memory that basically what happens is that that doesn't call malloc at all internally, that's just stored on the stack. So that's just three three numbers basically. But all of that and and but the internals have to be very carefully written to account for that case. And so that's I think another thing where, okay, all of this convenience where it's like, oh, this type vanishes at run time, well, there's a lot.
• 46:31
  Like if you're writing this stuff, you have to pay a lot of attention to it. So I would recommend reading about that for sure.
• 46:38
  Bryan Cantrill: Yeah. That's interesting. And then so as you and do you have to I mean, naive question I think, but is is are zero size types something that you need to consider for IDQD? I mean, if
• 46:50
  Rain Paharia: Yeah. I was lucky enough that I was able to delegate to the standard library for for worrying about zero size types. So IDQD doesn't have to have special handling for zero size types because the standard library does the hard work here.
• 47:02
  Bryan Cantrill: Oh, that's interesting. Can you be a little more specific? What do you what do you mean specifically?
• 47:05
  Rain Paharia: So so let's see. So I think this now I'll start going to some of the internals of IDD QD. Yeah. Yeah. So IDDQD uses a fairly straightforward, like, linear storage where you just have, like, a flat list, a flat back of things, and then you have indexes into that.
• 47:27
  So so you end up having, like, a a hash map which has pointers, which has integer pointers into that flat list. And and so it's a fairly straightforward way of doing things. I think it's it's pretty standard. But the flat list of things is just a VAC. Right?
• 47:41
  And so there, I kind of outsourced the WorryingWise To write this to VAC. Right? So you can store a flat like, a ID hash map of zero size types, but, like, it'll end up getting delegated to the VAC, and the VEC will and and the VEC will, like, you know, take
• 47:59
  Bryan Cantrill: care properly. Yeah. Yeah. Yeah. Yeah.
• 48:02
  We and this is actually a really I mean, there are many very concrete embodiments of this, but just the point you said at the top about the power of unsafe rust where you can have, like, you're still writing rust. Yes. And so you can still just because you're writing unsafe rust doesn't mean you have to reinvent everything yourself and you get to reuse existing data structures and leverage the fact that they've been written correctly, albeit we yeah. And you need to use unsafe. Right.
• 48:27
  And leverage their own correctness. Yeah. Yeah. Interesting. Yeah.
• 48:30
  Your example of the the kind of elementary example of like splitting it splitting it mute and which which takes a slice and and splits it. But highlights, I think, some of the peril that you get into when you are there's a lot that even in that something that is so basic, there is still a lot that can go wrong if you're called pathological. So, like, particular, it's like, what if I give you a bound that's, like, totally outside of this thing? Like, you've gotta you've gotta deal with that a little more concretely. You can't just, like, whack it.
• 49:08
  Rain Paharia: Yep. Yep. And and I think I think splitter mute is pretty illustrative of of, like, you know, the thing we talked about at the beginning about Rice's theorem. The Rust compiler doesn't have a native way to split a, like, a slice into two. Right?
• 49:24
  And so you have to write that in Unsafe Rust. And so if you open up the slice splitter mute, then that that implementation is written in Unsafe Rust where, you know, there is, like, this little line. But even for something that small, there are there are all these different analyses you have to do. And so so the first analysis is, of course, that the slice that you are passed in is an ampersand mute rather than something else like an ampersand t. Right?
• 49:50
  And so so it is a mutable reference and not a shared reference. In Unsafe Rust, you can accidentally convert a shared reference to a mutable reference, and that would be extremely bad. It would be bad.
• 50:00
  Bryan Cantrill: Yes.
• 50:00
  Rain Paharia: Right? And so, I I think, and so so, you know, that so so that is an example of a case where you cannot actually like, that that is just, like, that is a core invariant that the safe rust has to uphold. And so if you're analyzing the fun safe rust, you have to look at the input parameter and be like, okay. Yeah. This parameter is ampersand mute.
• 50:19
  The the next one is that the, the the mid that is passed in has to be inbounds. If the mid is not in bounds, then you then you will end up reading past the end of the end of the buffer, and that's also not great. Right? And and and so, you know, there are all these little things that you have to think about where, like, even the simplest kind of Unsafe Rust where it's like, you know, you have this big slice and you're just splitting it into two, even that has all of these invariants that that you have to maintain. And I feel like if you are if you are, like, a seasoned Rust developer, then some of this might be obvious that, yeah, you know, you can't read past the end of the array.
• 50:59
  But I think it is something you have to reason about, and I I do think that if you're stepping into Unsafe Rust, it is worth building up those muscles to start reasoning about these things so that when you get into the more complex kinds of UnsafeRust, there is less that surprises you.
• 51:15
  Bryan Cantrill: Okay. So then let's talk about the more complex, Russ, because that that's what IDQD is actually quite a bit more or has to be more sophisticated Even though it's it's able to use a standard library for its own internal data structures, which is a relief. Makes it easy or makes it easier. Yes. But you still have got this this problem of having to deal with really path and in particular, so I mean, you talk about like implementing the the the inner mute is really really gnarly.
• 51:44
  Yes. So you wanna talk about that example a bit?
• 51:47
  Rain Paharia: Yeah. So Itermute is, I think, very, very interesting. Mutable iteration is a surprising challenge because of a certain a certain set of decisions that were made in the Rust ecosystem around Rust one point o or maybe a little bit before that. So the big thing about iterators in Rust is that the values that they return cannot borrow from the iterator. And so what I mean by that is that, in most cases, you know, you just have like say like a for loop.
• 52:20
  Right? And so the for loop, you know, you you iterate over like a list or a a map or something, and the for loop you're just kind of chain you're you're accessing the things that you iterate on inside inside the the for loop. Right? But what Rust lets you do is that it lets the the things that the iterator return, it lets it escape the context of the for loop. And and and that is and that was driven by and and there's like these long discussions that happened before Rust one point o about this, but what that means is that there are certain so so what it means is that one of the things you can do with Rust's iterators is that you can collect them all into say a vec or or a map or something like that.
• 53:04
  And and so you you do like this, Yeah. So so the example I gave in the blog post is, you know, you have this items map and then you do like an item mute and collect. And now this collect has this vec has a vec of mutable references and you and so you have all of these different mutable references to all of the individual items off the map exist at the same time, and now it is your responsibility to make sure that you don't return the same item twice. And doing that requires you to write unsafe Rust because the Rust compiler does not know that the successive items you're returning are actually all distinct from each other. And so if you are writing an item mute kind thing, then you have to promise to the Rust compiler that the values you're returning are actually all distinct from each other.
• 53:55
  And so that is what led to several finding several different unsafe, bugs in Rust around pathological implementations.
• 54:04
  Bryan Cantrill: Yeah. And this is where you also get into the you're kinda relying on I mean, the pathological user code makes makes this particular challenge more difficult.
• 54:13
  Rain Paharia: Yes. Yes. It makes this challenge extremely difficult, it turns out, because, when you have pathological user code, then it, at least with the way IDTTD is structured, and and this is also true for, like, if you're writing your own map type from scratch or something, then you have to worry about, okay, has my map been tricked into storing the same item twice somehow? Right? And and and this is and and this is just, you know, something that you have to pay, like, really close attention to.
• 54:42
  Right? And I think if you were a seasoned Unsafe Ross developer, you're like, okay, this is something that might sound familiar to you, but, honestly, I learned about some of the pathologies that user code can have while working on this. Right? So I didn't come into this knowing everything. I I think very few people do, but the the forcing function of actually having to reckon with, okay, user code can misbehave in all of these ways forced me to learn about some of those ways and and about how to guard against some of those ways.
• 55:13
  Bryan Cantrill: Okay. Because can you talk about your fixes here? Because I think that the fixes to, in particular, to we we you you need to not be tricked into into to is and some of your fixes are I've been in particular, I I I love a trade off that you made in one of your fixes. You wanna describe the the the the the how you kinda fixed this?
• 55:34
  Rain Paharia: Yeah. So what happened is that while, you know, while doing this investigation, I ended up finding a case where if you could trick the IDQD to store the same index twice and return the same index twice and lead to mutable references to the same, to two mutable aliases to the same memory. And the way to trick IDDQD was that you have an OR implementation that is almost always correct except for there is one point at which you trick you you say that, okay, you return an incorrect result and if you do if you return an incorrect result at that exact point, then the map could be tricked into storing duplicate indexes. Now and and so that that was, know, that was one issue, but I felt like this is kind of emblematic of a class of issues because, you know, I found one case and I'm sure there are other cases around this. So I ended up doing two different things.
• 56:31
  So so I ended up kind of were cases where we would do, like, this false match against and and we would have a false positive match on the wrong entry. And so I used, I I basically used a tiebreaker to ensure that the we would not never have a false match. We might have some matches that, you know, we we don't end up matching, but at least we won't ever have a spurious match for the wrong entry. But then the other one that I think is really interesting, which is that one of the things I realized is that the bugs happen when you call into user code. Right?
• 57:07
  Like, IDGKD itself is perfect. Right? Like or at least that's the assumption. Right? But but I think it's it's and and so the the trade that I made was that if we have encountered a situation where we have established that user code is buggy, then rather than doing an efficient remove, we end up doing a linear scan.
• 57:27
  So we we end up doing a scan that throughout the whole, throughout the whole map that, does not call into user code at all. Now this is less efficient, obviously. Right? It's linear rather than logarithmic or o one for a hash map, But, at least this doesn't call into user code, and so we are kind of guaranteed correctness at the cost of some inefficiency when it comes to buggy code. And so so I felt the trade off was okay because because of the fact that you could only encounter it in the buggy code situation.
• 57:58
  Like, incorrect implementation would never encounter it.
• 58:01
  Bryan Cantrill: I love this. I I have to say I absolutely love this because I don't know. We're gonna be correct, but now the performance is gonna be poor. Like, you're gonna be like and I and I welcome you, you programmer who have written faulty code to investigate why is the performance pathological. And it'd be I I mean, I guess the the it would be truly pathological if you were using this because you allow for an org mapping.
• 58:30
  You had a true you know, a very large number of these things and because it is not inconceivable that you would be able to actually see this. Although, I mean, be a you're in a truly very pathological code because and maybe this is possible with it it it'll be interesting to see if it occurs in nature. But I just love the idea of someone investigating this. And I mean, love is a bit strong. That that sounds a bit sadistic.
• 58:55
  But I think Roberto Ledru, how how frequently have we mentioned Ledru on the on on the podcast?
• 59:03
  Adam Leventhal: Infrequently.
• 59:04
  Bryan Cantrill: Infrequently. Okay. So this is the is the French detective that is investigating a murder and this is in the nineteenth century. And this is from the I first learned about this in a terrific book that is a a total snapshot of a very narrow range of the eighties, Strange But True. And the did you have Strange But True?
• 59:28
  Do you know this book, Adam?
• 59:29
  Adam Leventhal: I I'm I'm familiar with it, but I don't I don't know that.
• 59:32
  Bryan Cantrill: I know. I think this is like one of these like Scholastic books. This is like a life changing Scholastic book for So Strange But True, it actually is not googling well. This is a bad sign. This is real I'm realizing okay.
• 59:44
  Yeah. Okay. So this is in in the book Strange But True, and I I actually got I think it's Donald Sobel. Is that the I think it's Donald Sobel as the author, but, like, now okay. Good.
• 59:54
  Thank god this book exists. I didn't I didn't I didn't hallucinate this whole thing. So in the this has a story of Robert Ledru who Roberto Ledru, this French detective investigating a murder and he figures out that the that that the murderer has he finds a footprint for the murderer on the beach. And when he is I feel like laboriously scanning this beach looking for the footprint. And when he finds the footprint, he realizes that he himself is the murderer because he it has a crooked foot in the some way.
• 1:00:30
  There's a foot deformity. And this guy had murdered someone in his sleep. He was like an extremely active sleepwalker. And the you know, I know I know I know what you're thinking. And I'm thinking the same thing.
• 1:00:43
  This feels like a hallucination that is even beneath like GPT two, but here we are. I look, it's in the book strange but true. It's gotta be true. But the the and and David and Dave Chico and I particularly would would joke over the years about like getting Ledrood where you you are investigating something and like you're getting closer and closer and closer to the murderer and you find the murderer. You are the murderer.
• 1:01:07
  You are the murderer? Yeah. And I feel like someone who who looks for pathological performance is gonna be like, uh-huh. I oh, I damn it. I screwed up the because you'd have to screw up in a very particular way the ORD trade, the word trade.
• 1:01:23
  Right? Right? In order to see this. Is that the right way
• 1:01:25
  Rain Paharia: to end? Yeah. I mean, the the cases that I came up with are very much you, like, poking around. You know, at the the way I wrote this is that you have a thread local, and the thread local, you set it to a value, and at the end, the org implementation checks for that value and and kind of returns the wrong result. Right?
• 1:01:40
  And that's how I that's how I relate the bug. Right? So you have to do something on that order. But all of that is purely safe rust, importantly. Right?
• 1:01:50
  And so and so, you know, Unsafe Rust has to guard against people abusing threat locals to, you know, return the wrong order implementation sometimes. And at some level, I hate it. Right? This is like a a thing that we have bound ourselves to as as Unsafe Rust people, as people writing grassroots, like, people people can do thread local Odd stuff. Like, I mean, you know, the the Haskell, like, for example, Haskell also has the Odd thing, but if you are doing thread local stuff, you have to use some version of Unsafe in order to do thread local stuff to abuse your org implementation.
• 1:02:23
  Right? And so Haskell has it slightly easier, but Rust lets you do so much in safe Rust, which is, I think, really cool, but there are also some lines. Yeah. Right? So yeah.
• 1:02:34
  I don't know. I mean,
• 1:02:34
  Bryan Cantrill: I think it's the I think it's the power of it though. I mean, honestly, the the the it is very powerful that you can go do this. And but it okay. So and then you how did you find that? This particular issue, did you find because you you talk about, like, the various techniques that one uses or that you were using when running on Safe R Us.
• 1:02:52
  Was this just, like, pure reasoning about the code that you found? How did you find this?
• 1:02:56
  Rain Paharia: It was a combination of reasoning and using so so this was a combination of reasoning. I was talking to people, and I especially wanna call out Ixie who's in the audience here, and Dan Cross. These are the two people who did a lot of the Unsafe reviews here. We so so, you know, thank you to both of them. Seriously, like, it was so helpful to have multiple sets of eyes look at this.
• 1:03:21
  But I was talking to them, and then I realized that, okay, you know, this this duplicate index thing is something is is a place where I was worried that there was some give. Right? Like, there's some like, I wasn't entirely sure that this duplicate index thing was always upheld. And so I started investigating and yeah. So then, you know, I I realized that, oh, at this exact moment, just while while looking at the code at this exact moment, You can kind of screw up and and introduce a duplicate index.
• 1:03:49
  Bryan Cantrill: And this is where I mean, and you talk about using property based testing with the the naive map is extremely helpful, I assume.
• 1:03:57
  Rain Paharia: Yeah. Yeah. So there there are several layers of validation that have gone into IDTQD to make sure that it is actually sound and and, correct. And and one of the one of the the most basic ones, which which I consider to be table stakes, I I don't know if that is a general opinion, but I personally consider just to be, like, absolutely essential, is that you write a second implementation of what you wanna do, and the second implementation is very inefficient, but it is obviously correct. Then you generate random sequences of operations, and you make sure that, okay, with this random sequence of operations, the the two implementations agree on everything.
• 1:04:34
  And so I I personally consider that to be, like, if you are writing a data structure, you should do this. Like, I I I would I I would be suspicious of a data structure that didn't have some version of that.
• 1:04:44
  Bryan Cantrill: Wait. I think this is So Rain, this is a very important point because a lot of these kind of novel I mean, even like a b tree. Right? I mean, ultimately, like a b tree is just more efficient. Like, you can actually have you can have a a a a dumb dumb b tree that is definitely correct, does not use unsafe rust, and is not something you would I wouldn't call it accidentally quadratic.
• 1:05:06
  It's more like deliberately quadratic. I mean, in terms of so like you like, for a lot of these things, there is often a way of doing a naive implementation that is is gonna eat it in performance. I think it's a it's a really important point.
• 1:05:20
  Rain Paharia: Yes. And I think you should write that second implementation, and and then you should essentially fuzz your implementation and make sure that those two implementations agree. But I will say that the original naive map that I wrote, and I think what most people do is that they will only write a correct implementation of user code and and call naive map against that correct implementation. So having tests for correct implementations of user code is very important. And then I think there's another layer on top which is, okay, can we have tests for incorrect implementations of user code?
• 1:05:53
  And so both of these, I think, are essential for, you know, this kind of high assurance on SafeRust. And so you you kinda have to approach this from both of those angles, I think.
• 1:06:05
  Bryan Cantrill: So how do you do that? How do you, like, I mean, obviously, it it I mean, incorrect programs are unbounded. So Yeah.
• 1:06:12
  Rain Paharia: Yeah. And so so they're actually so so this is where I unfortunately have a little bit of a less satisfying answer because there is no way to kind of express the space of all incorrect programs in as a test. And so and so what I do is I have a variety of different kinds of tests. And and so these are not property based tests, but rather these are example based tests that there is a battery of tests, there's a file called pathological dot r s, and there's a battery of tests that are do things like, you know, return the value at the wrong time. Many of them are regression tests where many of them are like, okay, you know, I fixed a bug and then here is the test that that it shows that it fixes the bug.
• 1:06:52
  There is one particular place where I was able to get a really good, set of, of, like, randomized tests and that was for panic safety. And so do you wanna dive into panic safety?
• 1:07:06
  Bryan Cantrill: Yeah. Definitely. Yeah. Yeah. Absolutely.
• 1:07:07
  Rain Paharia: So oh god.
• 1:07:11
  Bryan Cantrill: Oh, was I supposed say no to that? I can say no to that if you want me to
• 1:07:13
  Rain Paharia: say
• 1:07:13
  Bryan Cantrill: no.
• 1:07:14
  Rain Paharia: Mean, No. It's This this is good. This is good. Because I think so user code can panic at any time. And and so, you know, you are allowed to just say panic in your org implementation as an example.
• 1:07:27
  Right? Now panic in user code can actually be caught by, by surrounding code. So so Rust has, like, cache unwind, and and Rust has a bunch of ways to do that. And, and and this actually touches into some of the lock poisoning stuff as well, if if folks are familiar with that. But panic unwinding is a recurring source of issues in Unsafe Rust because what happens is that you would you will often, like, set up something.
• 1:07:56
  Right? And if you make sure that that code runs to completion, if if user code doesn't panic, then it is fine. But if the user code panics, then you're left in this kind of torn or indeterminate state, and maybe down the road that can lead to things like duplicate indexes being stored. Right? For example, you you know, you allocate an index, you store it, and then you call into user code and that panics, and suddenly you have like this orphaned index, and maybe that can get reused because, you know, you screwed up somehow.
• 1:08:21
  Right? And so, Unsafe Rust has to pay a lot of attention to, panics in user code, and and that's where, I think adversarial user code does include this kind of code as well. And so what I did was I I wrote a set of property based tests that were basically what they would do is that they would create a bunch of panicky operations and at every point that, the maps called into user code, each one of them was instrumented with, okay, I can panic here. Then what I do is I generate a sequence of those operations and then for each operation, I also generate a panic countdown. And a panic countdown is like I've empirically chosen these values to essentially cover like a a space of, you know, panics happening early in the process, panics happening late in the processes, carefully empirically chosen as a distribution.
• 1:09:13
  But the the net result of all that is that we end up in a situation where we we have all these and and so we generate a random sequence of operations that panic after the countdown hits zero. And we just run them and we make sure that the map's internal invariants don't get invalidated. And so that shook out like, I wanna say three or four different bugs that were all related to panic safety in in this context where the maps internal invariants were getting were getting violated. Not all of them were soundness issues, so not all of them led to memory on safety, but we can actually provide a stronger guarantee now. There's actually a lovely user who who read the blog post and then submitted a fix, which is really really kind of them.
• 1:09:58
  Bryan Cantrill: Oh, yeah. That's kind.
• 1:09:59
  Rain Paharia: Which is really really really nice. Their name was Sebastian. And it was really cool because, like, now we can actually say that if user code panics, we don't end up mutating the map at all. And so Oh. And so we we actually have a fairly strong guarantee around that.
• 1:10:15
  And and that is similar to the guarantees that the Rust standard library install provide, but those, if you look at their implementations, those have been very carefully written to be panic safe in this context where, you know, you essentially have like this this period where you call user code and at the very end, whenever you're doing mutations, you do it you always do it after all user code has been called. So so it's a it's a careful thing that you have to write, but, but it is definitely something that, is fairly easy to do property based testing with. So I think that's super cool.
• 1:10:45
  Bryan Cantrill: Yeah. And we generally abort on panic. I mean, I know this is maybe Yeah.
• 1:10:49
  Rain Paharia: Yeah. Yeah. Yeah.
• 1:10:49
  Bryan Cantrill: This may be maybe a divisive issue, maybe not. But, like, we we do not I mean, for a bunch of reasons, we want a panic to immediately abort and not to and and in part because like we don't wanna have to reason about the system that's in this state. Yeah. But it but that's not the only behavior. You don't need to do that.
• 1:11:06
  You can actually and there are, I guess, reasonable arguments. I'm sure reasonable arguments.
• 1:11:12
  Rain Paharia: There's some there's some arguments. I think the big argument is, like, you know, if you have a request handler then Yes. For HTTP requests or something, then you want those to be isolated. But yeah. I mean, there's definitely one world in which Rust just always aborted on Panic, and that would have been an interesting world.
• 1:11:29
  That is not the world we live in, and so unsafe Rust people have to, you know, do all this extensive work on property based testing and and and that kind of thing.
• 1:11:38
  Bryan Cantrill: Right. I mean, it's something that that you've gotta be able to go deal with. I mean, the that you also then did the LM based adversarial review, which I thought was very interesting. And did that did that yield things? Because I thought that was a Yeah.
• 1:11:50
  Rain Paharia: That that was that was a very interesting thing. So what I did was I kind of wrote a bunch of pathological tests myself, and then I essentially used that as a seed for an LLM to be like, okay, can you can you find ways to break the map invariance? And both Claude Opus 4.7 is what I use. I know 4.8 is the newest one, but but I use 4.7 for this investigation and GPD 5.5 each individually found, like, one or two different ways to break the the maps in in a way that I I thought was really surprising.
• 1:12:21
  Bryan Cantrill: That's really cool. And so Yeah.
• 1:12:22
  Rain Paharia: And and I thought that's really cool, right, where you kind of given it the structure where it's like, here is the here is the constraints, here are the invariants that, you know, I wrote out, and can you find ways to break these invariants? And I think it was g p d 5.5 that just chewed on it for, like, twenty minutes, and then just, like, presented a thing. And it's like, here's here's a way to break the invariant. And that was a fascinating experience, and and I was just very surprised at that. And and I feel like, you know, this is kind of the whole like LLMs for rigor thesis.
• 1:12:50
  I I felt like this is a very productive way to use LLMs as this kind of the person doing the adversarial doing, you know, trying to break stuff. Right? And so A
• 1:12:59
  Bryan Cantrill: 100%. Yeah. And I mean, obviously, bring the chime for our our episode on that, Adam, because I think that this is a almost like a canonical example of where, you know, using an LLM to do this thing that would be kind of unbounded. It's very brute for and and and also anything that the LLM finds is very readily verifiable. Right?
• 1:13:18
  This this is not like you're having an LLM vibe code something. It's like, I found that if this is like using LMs to find security vulnerabilities. And it's very easy to validate whether this is an actual like, we found something real or not. And that's that's pretty great. And so I found found real stuff, which is which is terrific.
• 1:13:36
  Rain Paharia: Yeah. I I do wanna give a special shout out to the verifier, which is Miri. So Miri, folks are not aware of, is a so it stands for the so so Rust's compiler has a few different intermediate representations. And so Miri targets the mid level intermediate representation, so that's m I r and it is an interpreter which is where the last I comes from for the mid level representation and what it does is that Miri is the vehicle by which essentially people are figuring out, okay, here are the things that are allowed and unsafe for us and here are the things that are not allowed. So it interprets the code.
• 1:14:12
  Right? It's it's just an interpreter and it's not very fast, but the important thing is that it will do things like, you know, make sure that there are never mutable, references to the same memory as an example. And so, query is a really, really good for an LLM because an LLM can, like, propose a thing, can write a test which tries to show the unsoundness, and then Mary will be like, okay. This is clearly unsound or or Mary will not detect it. I do wanna mention here that Mary is maybe not perfect, so, you know, you cannot be sure that if Mary says that it isn't the case, that it is unsound.
• 1:14:50
  But but you can be reasonably sure that if Mary says that, okay, this is this is not this is unsound, then, yeah, it really is unsound.
• 1:14:59
  Bryan Cantrill: So so Mary does not have false positives? When Mary finds unsoundness
• 1:15:04
  Rain Paharia: So it's a little bit more complicated than that because there so this is where you kind of get into some of the boundaries of what is understood about Unsafe Rust or not. Mary implements two different models for what it means for, like, this kind of mutable aliasing not to happen. And so, the two models are called stacked borrows and tree borrows. And stacked borrows is kind of the older original model. It is widely understood now that stacked borrows is too strict, and it accepts or it rejects too many programs that are otherwise fine.
• 1:15:39
  So tree borrows is a way to kind of thread the needle a little bit more. IDDQD passes both stacked and tree borrows, and and I have tests in CI to make sure of that. I I will say though that there are I had to put in specific workarounds to make to make IDQD pass with stacked borrowers in particular. So IDQD actually is at exactly that boundary where, you know, there's some things that are allowed in stacked borrowers but not in tree borrowers and so on. So so with IDGTD, there are there are some workarounds to make things pass in stacked borrowers that that are do already pass in tree borrowers.
• 1:16:15
  Bryan Cantrill: That's very interesting. I do remember Adam, do you remember we had that that that one of the predictions episodes, someone predicted something around Miri and Treboros. And all I remember just like, I'm gonna it's gonna take me hours to even understand this prediction, let alone so whoever made that prediction is gonna have to let us know if that prediction came true or not. All I know is that, like, tree borrows and muri were two of the nouns that were involved in my general level of, like, total I'm like, I so is this a good thing? I don't even know what's going on.
• 1:16:41
  And meanwhile, like, we're making dumb predictions about the pope or whatever. Well,
• 1:16:45
  Rain Paharia: I know. I'm sure. Yeah.
• 1:16:47
  Adam Leventhal: No. I remember that. And and if they tell us they got it right or wrong, then it'll take us hours to verify
• 1:16:52
  Bryan Cantrill: them. Exactly. That's right. You'd either way. So Yeah.
• 1:16:59
  But the so, yeah, that that and and that is really interesting, right, in terms of being able to actually use so using Miri to to to kinda run these pathological tests effectively. Yeah. Yeah. Interesting.
• 1:17:14
  Rain Paharia: Yeah. I think it's so there's a couple of different things that are interesting here. One of them is the fact that you can, like, directly use Myriad against the pathological tests. Right? Yeah.
• 1:17:23
  Another thing that I think is even more interesting. Right? I feel like use Miri is kind of the default advice that people give you. By the way, the BUN rewrite did not use Miri at first either. I know they added Miri later, but but, you know, they didn't follow that.
• 1:17:36
  So so that was part of what trolled me. But I think there's kind of a a little bit of a more subtle point here, which is that you can, like, kind of prove things analytically by saying that if this invariant doesn't hold, then then the code is unsound. Right? So so for example, we established that if you have a duplicate indexes stored in in one of the indexes, then the code is unsound, and and we've kind of established that. And so and then you you don't need Miri to show those invariants.
• 1:18:05
  Right? Because you have kind of analytically bridged the gap between showing that those invariants don't hold and unsoundness, and so you can work and and you don't have to do all the slow interpreter stuff. You can actually throw a property based test. You can throw all of this fast testing at making sure that the invariants are upheld. And so I think a big part of Unsafe Rust is also finding, you know, the the things that you can kind of analytically reason about and then use empirical verification to get to that point before and and then let analytical reasoning carry the rest forward.
• 1:18:38
  Bryan Cantrill: Well, and I I I mean, I loved your breakdown of, like, all of these different kinds of techniques because you're gonna and you're gonna need at some level, like, you should be using certainly, if you're writing a data structure and using Unsafe Rust, you should be using all of them to because you are not gonna be able to make this provably correct for the reasons that you outlined. You had to, kinda what about formal models, which I guess was it was that an attempt to, like, front run any Hacker News comments?
• 1:19:03
  Rain Paharia: Just I mean, a little bit. So it was actually a a question on lobsters that came up about the the the formal methods thing. I actually gave formal methods a shot, and I think that, with sufficient work, I think you will be able to use formal methods. I think that there is a world in which you can formally prove that no matter how adversarial the user code is, you end up, like like, the invariant store get violated. Right?
• 1:19:28
  And so that would actually would bridge that gap there. Right? And and so if you could if you could mathematically prove that no matter how misbehaving user code is, everything is still fine, that would be amazing. That
• 1:19:43
  Bryan Cantrill: Feels hard.
• 1:19:43
  Rain Paharia: Tooling doesn't quite exist to that level yet. But I think that that is tooling gap, not a theoretical gap. And so I I am very, very interested in exploring how we can bridge that gap, and and if if, like, you know, if you're if you're listening to this and and you wanna kinda work on some formal method stuff, it would be amazing to formally prove that IDDQD is resilient to arbitrarily pathological user code. That would be kind of an amazing outcome of this blog post, I think.
• 1:20:09
  Bryan Cantrill: Yeah. So let's just talk briefly about what the kind of the the the reception of the blog post has been. So we you put the blog post out there. Yeah. And what has been some of the reaction to it?
• 1:20:20
  Rain Paharia: So it's been it's been really interesting. I think so a bunch of folks, and and I know someone in the chat here, and and a bunch of folks commented and said that, okay. They thought that there was a really good primer on, some of the ways in which, you can verify UnsafeRust and and, like, a like, a just a guide on here's how you can try and make sure that Unsafe Rust is, is correct and and, you know, doesn't have undefined behavior. I think, there's some folks who work at, I know Angela Lee Brinker works at MITRE. He he had a nice comment about, like, you know, this is how you do high assurance trust and this is how you go beyond, you know, things like simpler statements like use Mary or whatever.
• 1:21:04
  Right? You know, really tackling the problem from from every possible access that you can. I think there's a lot of appreciation for the diagrams, so so I do wanna special shout out to Ben Leonard.
• 1:21:16
  Bryan Cantrill: To Ben Leonard. Exactly. Diagrams are beautiful, I gotta tell you. They're really, really nice.
• 1:21:21
  Rain Paharia: Yeah. So the the context there was that I kind of made some sketches in Mermaid, and then and then Ben and then I was like, hey, can we put Mermaid on the blog? And Ben was like, oh, Mermaid, my nemesis. And then and then he was like, okay, well, I'm gonna just like do these diagrams myself. And so so Ben was very very kind to lend his time to do these diagrams.
• 1:21:42
  They're I think they're beautiful.
• 1:21:44
  Bryan Cantrill: They're gorgeous. Yeah.
• 1:21:45
  Rain Paharia: And so so, yeah, that's the context behind that. I think
• 1:21:49
  Bryan Cantrill: You always love that. You were people were people were like, oh, yeah. The blog entry. Yeah. Yeah.
• 1:21:53
  I don't know. You said some stuff. The diagrams. How'd you make the diagrams? You're like, okay.
• 1:21:57
  I mean, I love the diagrams too, George. Yeah. But I get yeah. This is this is a a dense technical topic you can see. Yeah.
• 1:22:03
  Some go to the diagrams.
• 1:22:05
  Rain Paharia: I think there was a lot of really good reception about kind of the the ladder of thinking about Unsafe Rust because I think a lot of the simpler introductions to Unsafe Rust will talk about, you know, the where I talk about the ladder where there are three different cases increasing levels of difficulty, a lot of the simpler ones will often talk about the first or maybe the first two. There is a little bit less written about the last one, and so actually taking on the hardest version of the problem and and trying to explain that was was really interesting. And so so a lot of folks really really seem to enjoy that. So overall, it's been it's been it's actually been a really, really good reception. I think a lot of people liked it.
• 1:22:47
  I I do wanna say that it is kind of amazing that I get to post like this on the Oxide blog. Right? This is I think this is really cool, and and I think that this really calls out to our values of transparency and rigor and responsibility that, you know, we we we we're not like, there isn't like someone breathing down the next thing like, oh, you said that there is a bug in our code. But, like, you know, we all understand that bugs happen in code, and being able to honestly talk about it, I think, is is really powerful.
• 1:23:18
  Bryan Cantrill: And, Rain, I really appreciate that. I do think it's a sad indictment on the industry, though, that this is like that this is just that you're I mean, I think it is great. I think it is great, but also, like, this feels like an extremely low bar that you should be able I mean, the such a terrific dense technical content. Like, no one no. Every company should be elated to have one of their technologists, one of their engineers write something like this because it is so extraordinary.
• 1:23:42
  It's extraordinary detailed. I think it's a great, like, it's a great primer for understanding, you know, how like, if you need to write unsafe unsafe rust because I think the other thing that was, like, very eye opening for me is just, like, the the wait for this kind of rust, as you say, the hardest kind of unsafe rust, it's not just the your code that you are writing. It's the fact that you you need to an arbitrarily incorrect user of this that's writing safe rust
• 1:24:11
  Rain Paharia: Yes.
• 1:24:11
  Bryan Cantrill: Needs to actually like, you're bounded in terms of the you have to be bounded in terms of the of of of the pathology there. And I think that that is like I I I think getting people beyond just like abstinence on this in terms of like, okay. Like, abstinence is important, but like, okay. So if you you need to do unsafe rust, let's talk about how to do unsafe rust as safely as you can. It is obviously tautologically unsafe.
• 1:24:39
  But, you know, what are the the how to kind of proceed about thinking about it? And again, I love like the approach of like, you're gonna need to bring all these different things to bear to go do the best you can to validate that you've got I mean, you're not gonna and it's gonna be very difficult. And maybe we'll get to the point where you can use formal methods to really prove smaller stuff correct. But it's like, you're basically gonna be relying on doing a bunch of different things, and you've got a great responsibility to anyone using your software to get this exactly right.
• 1:25:12
  Rain Paharia: I I I really wanna agree like, I agree about the responsibility, but because I think that, you know, what what do we owe to each other as technologists, right? And I think we owe the sense of who here is the responsibility, here is where it lies, here is who is responsible for concentrating, you know, the the reasoning in one spot so that everyone else has an easier time. Right? And I think that safe and unsafe rust especially this kind of generic unsafe rust is really shows you where the responsibility lies and I think that as technologists if we are choosing to embark on this kind of hard thing that we are we go into it with eyes open, understanding that we we have people that, you know, work with us or people in the open source world or wherever that we need to really look out for. Right?
• 1:26:04
  Bryan Cantrill: Yep. Yeah. Absolutely. Well, the great stuff. Thank you so much.
• 1:26:07
  I I really extraordinary blog entry. Ben, excellent diagrams. Diagrams are beautiful, but I thought it was just a, you know, I think part of what I I love about it is just the pedagogical nature of it in terms of like really taking people along a path and explaining to someone who's never dabbled or never really understood Unsafe Frost about why this problem is hard and especially why it's important. So
• 1:26:31
  Rain Paharia: Thank you. I'm Thank just glad that Oxide is a way in which I can do this, right?
• 1:26:36
  Bryan Cantrill: Is Oh, yes. What I'm Yes. These are the kind of problems that get you excited. We've got a lot like this. So we're really again, thank you again for this terrific blog entry.
• 1:26:50
  Thanks for joining us as a regular on a quasi regular, pseudo regular cast member, whatever we we call it. But I really appreciate you joining us on the podcast as well. Terrific stuff.
• 1:27:04
  Rain Paharia: Thank you for having me.
• 1:27:05
  Bryan Cantrill: Alright. You bet. And I think hey, Adam. We may be going I I think we I I I sense we may have a a European friendly episode coming up somewhat soon because well, speaking of Ben Leonard, he's done some visualizations of the rack that I think we wanna talk about. So I know I'm gonna tease a future episode a little bit.
• 1:27:25
  I may need to get Ben on here to talk about some of the the visualizations he's done recently on the website. So Cool. European friendly coming near you, which I know is news you can use because I know you got you got dinner to run to. You you mentioned you got something. You you have done something that I think is like borderline work workplace harassment.
• 1:27:45
  You've told me what is what is for dinner and that you can smell it. And now I feel like I'm having like a I'm having a visceral reaction to what is for your dinner, not mine. My I knew that was
• 1:27:55
  Adam Leventhal: a little mean. I knew that was
• 1:27:56
  Rain Paharia: a little
• 1:27:56
  Bryan Cantrill: It felt it felt unnecessarily sharp and cruel.
• 1:27:58
  Adam Leventhal: Yeah. Yeah.
• 1:27:59
  Bryan Cantrill: Yeah. But the go to your delicious pulled pork. The rest of us I will scavenge here among the crackers in the office and, you know, don't worry about me crackers and diet coca's calling you
• 1:28:14
  Adam Leventhal: you did. Right?
• 1:28:15
  Bryan Cantrill: Exactly. Alright. Go to it. Rain, thank you so much. Thanks for joining us.
• 1:28:19
  This is amazing stuff. Alright. Thanks, everyone. Talk to you next time.