Latest / Oxide and Friends / from /proc to proc_macro

Transcript

from /proc to proc_macro · Oxide and Friends

• 0:00
  Speaker 1: Adam, can you hear me?
• 0:04
  Speaker 2: Yes. I can hear you.
• 0:05
  Speaker 1: Alrighty. Very good.
• 0:10
  Speaker 2: And I'm gonna step away from the mic just for a second to make sure that the recording is going properly.
• 0:14
  Speaker 1: Yeah. That sounds good.
• 0:20
  Speaker 2: Oh, if you could talk for a sec while
• 0:21
  Speaker 1: we work out a
• 0:22
  Speaker 3: couple of hours.
• 0:23
  Speaker 1: I've got a a very rare request. I'm actually being asked to speak. I don't think I'm being asked to shut up. And with do you want me to do the the standard thing and tell you what I had for breakfast? Alright.
• 0:35
  And, so folks that are are just joining us, definitely, we we is that a what? Brian Cantrell, identify yourself. That is normally
• 0:52
  Speaker 3: Do you
• 0:52
  Speaker 2: have your super fan on?
• 0:53
  Speaker 1: I I do no. That is normal. Yeah. That's my 13 year old. My 13 year old, that is not my 13 year old.
• 0:57
  But my 13 year old, it in order to troll me, invented an account, b cantle number 1 fan. Synopsis, I'm just b cantle's number 1 fan. And then he likes the tweets that he thinks are lame, which is, like, the it's kinda so pernicious.
• 1:12
  Speaker 2: It's good. Sounds like somebody's son.
• 1:15
  Speaker 4: I hate
• 1:15
  Speaker 1: now. You know, that is I have to say it is the it is the worst when you see some attribute in your brood and you realize, like, they're getting that from watching me.
• 1:28
  Speaker 2: It's it's even worse when you get blamed for it.
• 1:31
  Speaker 1: I get blamed by yes.
• 1:33
  Speaker 2: Exact By my spouse.
• 1:34
  Speaker 1: By your spouse. Exactly. That's right. I know. It's like or or like, yeah, he said something that you would say.
• 1:39
  I'm like, wait a minute, but I didn't but I didn't say it. I had the I had the prefrontal cortex that told me not to say
• 1:45
  Speaker 2: it. Yeah. Still your genetic fault. Yeah.
• 1:48
  Speaker 1: Alright. Brian Cantrell, I've asked you to be a speaker, and I'm sure that you and I have gone through, I did he leave? He left. He left? Did it happen?
• 1:57
  No. No. He's there. The this Twitter Spaces does not make this part easy. But I'm sure this person has gone through life well, you know, we've gone through life being confused with one another.
• 2:07
  This is still how my daughter spells her last name, by the way. So
• 2:10
  Speaker 3: I I have received mentions for you in the past, but, haven't engaged.
• 2:17
  Speaker 1: Wow. I I'm sorry. I think I'm sorry. This is really You don't need
• 2:23
  Speaker 4: to be sorry.
• 2:24
  Speaker 3: Okay.
• 2:28
  Speaker 1: Yeah. I I'm I'm sorry. My name, obviously, I mean, I feel that I have spent my life correcting the spelling of my first and last name from your name. I feel everyone wants to spell my name as your name.
• 2:41
  Speaker 3: I I guess I I win then?
• 2:43
  Speaker 1: You do it. You do it. No. That's exactly the right attitude. You actually do it.
• 2:49
  Speaker 2: Brian, with an eye, I I admire your restraint because I get confused both for the Adam Levinthal, who's a British sportscaster, and I readily reply to, his bank, which has the wrong email address from for him. And to the American Hockey League, whenever people are very cranky with officiators, I always like their tweets or weigh in and explain that our officiator our our referees are as good as we pay them and things like that.
• 3:15
  Speaker 1: And it should be said that that you are actually holding up an American Hockey League jersey in your profile photo.
• 3:21
  Speaker 3: Yeah. I mean, it's it is is a reasonable
• 3:24
  Speaker 2: I mean, it's a reasonable confusion to think, I guess, that, like, some dude wearing a jersey who speaks for the league. But, yeah,
• 3:32
  Speaker 1: they they You relish this confusion.
• 3:36
  Speaker 2: I mean, only during Calder Cup season.
• 3:40
  Speaker 1: Is is the Calder Cup? Is that the obviously, that's the American Hockey League Cup?
• 3:44
  Speaker 2: Yeah. It took it took me a while to learn this, but yes. And and and the checkers are my favorite team.
• 3:50
  Speaker 1: Where are the Checkers from?
• 3:51
  Speaker 2: I think I knew that was given the I think they're from Nashville, but I might be confused.
• 3:55
  Speaker 1: You know, the the last I don't know if it was a the last minor league hockey oh, not the last. The a a a recent of the, like, 3 minor league hockey games I get the other one I went to was at Roger Faulkner's wedding.
• 4:08
  Speaker 2: Oh, my goodness.
• 4:09
  Speaker 1: Were you at that wedding?
• 4:10
  Speaker 2: No. I wasn't. In it was in Michigan. Right?
• 4:13
  Speaker 1: It was in Michigan. It was in Grand Rapids, Michigan, which is if you are ever in Grand Rapids, first of all, I definitely would recommend catching your Grand Rapids Griffins as they take the ice. The, it was like in October of 2001. So it was very, like, going to a minor league hockey game felt like a very patriotic act. So it was very, like, very post 911 kind of a thing.
• 4:35
  And the and then we also went to the Gerald Ford's presidential museum is in Grand Rapids. I don't know if has anyone been to the the I what? Go ahead. Oh, no. Go ahead.
• 4:48
  I'm just No. Please. You go ahead. No. No.
• 4:50
  No. You wanna make some denigrating remark about Gerald Ford, and I and I'm here for this. That's okay. I because I wouldn't need the same denigrating remark.
• 4:57
  Speaker 2: Mark. Sounds like your dream come true is what I was gonna say.
• 5:00
  Speaker 3: Well, oh, you're
• 5:02
  Speaker 1: right. I should've known that that was not a dedicated your archangel part. It's gonna work about me. That actually makes even more sense.
• 5:08
  Speaker 3: No. Not at all. I think I think just you you live
• 5:10
  Speaker 2: for those moments. I I think I remember you trying to rally the crew to go to the to the Gerald Ford Museum.
• 5:15
  Speaker 1: Well, and I was being very gen x and snarky about it in that, like, I I it's like, I wonder how how prominently they're gonna feature the fact that this is our only non elected president. Right? Gerald Ford never won an election. Gerald Ford was the the he was speaker of the house. He became the vice president when Spiro Agnew was rightfully new from orbit, and then became president when when Nixon resigned.
• 5:37
  And I'm like, I wonder how prominently you're gonna feature that. And you walk into the Gerald r Ford Presidential Museum, and you get about a foot in and inscribed in stone in letters that are like a foot high are the excerpt from his inaugural address where he says, I recognize
• 5:59
  Speaker 3: that I'm the only
• 5:59
  Speaker 1: person to not be elected to this office and the responsibility that he felt to the American people to represent the whole American it's it's it's amazing. That so I came out, like, I went in a Gerald Ford cynic, and I came out a Gerald Ford Super fan.
• 6:14
  Speaker 2: Really leaning into it. And just just for context, Roger Faulkner, his wedding in Michigan, our late colleague who invented the proc file system among other things.
• 6:23
  Speaker 1: Yes. The inventor of the proc file system. When Roger's goal was to be the oldest living hacker, and he may have succeeded. He was definitely Roger has passed, but definitely died with his boots on. And he would always say that I am at I don't know.
• 6:42
  I'm not sure if Tom Tom Linenow is here. I'm not sure if Tom had any I don't think Tom may have just intersected with Roger. Roger was coming out of that AT and T heritage and was at AT and T working on And as he said, I met Sun because Sun is UNIX headquarters. That's what, Tom, did you have any did you intersect with with with Roger at all?
• 7:03
  Speaker 5: I I don't think so. I don't think I ever met him.
• 7:06
  Speaker 1: I think you'd remember. So a story that kinda this is the I mean, Roger has many there are many, many, many Roger Faulkner stories. And when Roger died, I wrote a, an obituary for him for Usenix. And I convinced Usenix to make the paper that he wrote with Ron Gomes on the /proc file system freely available. And that paper, I think have you read that paper anytime recently, Adam?
• 7:37
  Speaker 2: Not recently. No. But it I I mean, I remember years ago.
• 7:40
  Speaker 1: It's a good paper. I mean and it the Roger viewed it. He's as he said, I am on a mission from God to make programs debuggable. That was that was pretty
• 7:51
  Speaker 2: great. Yeah. That was definitely the zeitgeist of of, you know, what that I fell into when I started at Sun and and was was sort of, like, fortunate and unfortunate enough to be in a bunch of Rogers code. You know, a 22 year old wondering why some jackass had used, a linked list, you know, instead of an AVL tree and kind of being slapped down time and again, be like, well, we didn't have that, you know, back in the day And, you know, 5 watches were not for anyone.
• 8:20
  Speaker 1: You were wondering why he was using a link list and not an AV Autry.
• 8:23
  Speaker 3: Yes. Yes. Yes.
• 8:24
  Speaker 1: So I was gonna call bullshit on that story if
• 8:26
  Speaker 2: Oh, no. He never he never used anything except for a link list.
• 8:29
  Speaker 1: They he never sent anything except for a link list. The the preferably nested link list. I mean, the order of n squared was the way he
• 8:36
  Speaker 2: liked the role. Yeah. At
• 8:37
  Speaker 1: least. Right. And, I mean, performance is the root of all evil, is a is a Roger Faulknerism. And he he viewed performance optimization as I mean, he did he was not trying to make his code, but he also viewed it much more important that the code be correct than the code be fast, if you have to choose. As we pointed out, like, you actually don't have to choose, and this code could be correct and fast, but
• 8:59
  Speaker 3: you've chosen to make it.
• 9:02
  Speaker 1: So so he so you had the temerity to to code review in the new Navy Ultra.
• 9:08
  Speaker 2: Oh, not so, you know, I wandered into, like, code that Roger had abandoned years, you know, years ago. For for example, with the with the, Solaris watchpoints, I remember in particular.
• 9:18
  Speaker 1: Oh, God.
• 9:19
  Speaker 2: He had he had done some kind of very cursory thing where there were a bunch of sharp edges, for example. So watch points, you know, regions of memory where you wanted to take a trap, when the user program, you know, modified or read or whatever. And it was and it was really fun stuff because it's you know, all, all this, virtual memory stuff. But for example, you couldn't have overlapping regions and the system wouldn't really tell you that. You would just sort of find out the of watch points.
• 9:51
  So it's fine when you're sitting at the debugger entering them by hand. But when you started having, automation insert watch points for you, for example, when you wanted a watchpoint at the end of every allocated region, it it didn't work so good.
• 10:06
  Speaker 1: It did not work. That was libwatchmalloc. Right?
• 10:09
  Speaker 2: Yeah. Exactly. Exactly. So libmatchmalloc watchmalloc
• 10:12
  Speaker 1: just,
• 10:13
  Speaker 2: I mean, just was a dog after your 10th malloc.
• 10:16
  Speaker 1: Right. It's called what live watch malloc because you'll be watching every malloc as it unfolds in in front of you.
• 10:22
  Speaker 2: That's that's right.
• 10:22
  Speaker 1: Belacial face.
• 10:25
  Speaker 2: So yeah. So, But it
• 10:27
  Speaker 1: I mean, watch points are are magical. Right? When they when they work, it's magical.
• 10:33
  Speaker 3: Yeah. It's like No.
• 10:34
  Speaker 2: It it's very very cool, and and very useful and very slow even with an AVL tree, because you're you're you're trapping and and remapping memory and single stepping and trapping again. But when you need them, man, that it the it's it's very hard to imagine alternate mechanism.
• 10:52
  Speaker 1: To be able to say, when is this memory modified? When this memory is modified, I wanna stop the program or I wanna take a core dump or I wanna I mean, it's just it it feels like a soup it felt like a superpower when it works.
• 11:03
  Speaker 3: Absolutely. And I think that the
• 11:04
  Speaker 2: the thing that Roger instilled, certainly, in me and I think in in, you know, all of us, you you know, you and Mike and the other folks interested, you know, really interested in debugging, was how how this kind of debugging stacks, where once you make these layers reliable, then you start building automation on top of it, like libWatchmalloc. That then exposes some of the shortcomings of these mechanisms you initially thought were just be manual. But you you to Roger's point, then then there's plenty of time to optimize once you've once you've proven that these mechanisms are are really useful and valuable.
• 11:36
  Speaker 1: Yeah. You know, that's actually a good point. And I I had not really thought of it quite that concisely that Roger made this incredible contribution about debugging infrastructure being an an attribute of a production system.
• 11:48
  Speaker 3: Yeah. Yeah. Yeah.
• 11:50
  Speaker 1: And the so for those of you who have if you've used, s trace, which traces, system calls on Linux. S trace is, I'm pretty sure that that it was following in Truss's footsteps. Although, Tom, maybe you have to so 4.x had I'm not sure what 4.x did for this, but trust, which was to, it it it was to trace it was originally t r s s. Right? To trace system calls and signals.
• 12:21
  What did 4.9x use to trace system calls, Tom?
• 12:26
  Speaker 5: I don't remember. I remember implementing an s trace thing back in Amdahl.
• 12:31
  Speaker 1: Oh my god. But it it this is on UNIX on Dara 370.
• 12:37
  Speaker 3: Right.
• 12:37
  Speaker 1: And is that p was that ptrace based?
• 12:42
  Speaker 5: Yeah.
• 12:42
  Speaker 1: Oh, ptrace. Such a Talk is the historic debugging interface in UNIX that ultimately /prock replaced. It's still what you'd use in Linux to debug. You use /proc for system information, not for debugging, and you use ptrace for debugging. Flaws.
• 13:14
  Have you ever, like, dealt with ptrace, Adam?
• 13:17
  Speaker 2: Yeah. Bunch on Mac OS. Yeah. And it's it's a huge pain in the ass. It's awful.
• 13:25
  Speaker 1: Ptrace in particular had this idea that you should be using the wait system call to wait for ptrace if that's
• 13:30
  Speaker 2: Oh, yeah. And just, like, overloading it.
• 13:32
  Speaker 1: Overloading it. Strange. Yeah.
• 13:34
  Speaker 3: It it
• 13:34
  Speaker 1: feels like doesn't it feel like there should be, like, one word for this in German? Of do you know
• 13:38
  Speaker 6: what I mean?
• 13:40
  Speaker 2: Yeah. The, like, the sort of misappropriation of Right. Mechanism in in like a seemingly clever way Yes. But one that is ultimately a disaster.
• 13:50
  Speaker 3: Yes. Yes. That's exactly it.
• 13:52
  Speaker 1: What is the one word for that?
• 13:53
  Speaker 2: What's that German word
• 13:54
  Speaker 1: for it? For those of you who are German software engineers, please hop in and tell us the one word for this in in German that explains, because, no, that's exactly it. And I thought
• 14:03
  Speaker 5: But but but in defense, there there was, like, 0 no form at all of inter process communication back then.
• 14:11
  Speaker 1: Okay. Wait a minute, Tom. Are you actually gonna defend PTRS's
• 14:14
  Speaker 5: use of
• 14:14
  Speaker 1: way? No. You may wanna think carefully here. You gotta you gotta legacy to protect. The
• 14:19
  Speaker 3: No. But the but there
• 14:20
  Speaker 5: was some mechanism in the kernel
• 14:21
  Speaker 3: that they they hijacked. Right? So Okay. So that so that actually is
• 14:25
  Speaker 1: interesting, Tom, because that is so in that era, your argument might be that the system call space was so sacred that you couldn't you you and you couldn't also use I Octal, I guess. You you had to use because the problem with using wait, wait is to wait for child processes, which one can argue whether they should have named it more concretely or more specifically.
• 14:50
  Speaker 3: Yeah.
• 14:51
  Speaker 1: But that is just death to overload that with ptrace.
• 14:55
  Speaker 5: Yeah. Just basically wait for any of that from some
• 14:59
  Speaker 1: Josh, would you like to say something about ptrace?
• 15:03
  Speaker 3: I'm I'm trying not to.
• 15:04
  Speaker 1: There you go. Good luck. So
• 15:07
  Speaker 3: It has it has it's the the manual page. The interface from what I recall is documented as, like, specific integers that you pass to the thing.
• 15:18
  Speaker 1: Oh, yeah.
• 15:20
  Speaker 3: I feel like it it predated the use of, like, define.
• 15:25
  Speaker 1: Well and it's an interesting object lesson too because there Torvalds talks about ptrace in, like, I wanna say, like, 2,001, where people a bunch of people are complaining about ptrace in Linux. And he's like, look. It's just we're too far into this point. Like, we can't replace ptrace. We're just gonna have it forever.
• 15:42
  You know, like, man, that was, like, 20 years ago. Like, I bet you could've replaced it 20 years ago. Doesn't that feel like it it it's just an interesting reminder in terms of, like, this stuff doesn't need to be beautiful.
• 15:52
  Speaker 3: I mean, over the course of 8 years at Giant, there were definitely times where it's like, we could fix this, but it'll take years before we can actually use it because everyone's gonna have to update and move past the the flag day and a bunch of other stuff. And then but then looking back, it's, like, been 8 years now. Could have probably changed things back then. At some point, it's been 8 years, then it's been 20 years, and it's been 30 years. Yeah.
• 16:15
  You can actually add a parallel improved path, even if it takes forever to get off the old thing.
• 16:22
  Speaker 5: P traces the x86 of system calls.
• 16:25
  Speaker 1: It is.
• 16:26
  Speaker 3: Right. The thing you have to remember is that computers are growing exponentially. So the best time to fix anything is always right now.
• 16:33
  Speaker 1: Oh, that's it. That that that'd be the best time to plant an abstraction is 20 years ago, but the second best time is today. Right?
• 16:38
  Speaker 3: But but it will always right. It's always it's always worse to wait another minute. Yeah. You know, Brent, I I
• 16:45
  Speaker 2: I feel like this is a long coming apology because, you know, I I was realizing the the other thing that I did, the other, like, kind of a super fun site I left for for at Sun that you guys carried into Joynt was this emulation of ptrace in branded zones. And so, you know, just just because there are a bunch of, like, keywords there.
• 17:06
  Speaker 1: Are you are you trying to make Josh scream or cry? Is this I mean, is this something some sort of Stanford Prison experiment that we're doing
• 17:11
  Speaker 2: with Josh? I feel like if I apologize in public, then he he can only be so loud.
• 17:18
  Speaker 3: Are you are you saying are you doing this in public so I wanna make a scene?
• 17:21
  Speaker 1: That's right. That's that's right. Well, I just wanna Look, Josh. Don't make a scene. Okay?
• 17:26
  Like, accept the apology. Like, he's trying to Is
• 17:27
  Speaker 3: that the calculus?
• 17:28
  Speaker 2: That's right. Yes. Yes. That's exactly the that's exactly the calculus. So so, you know, Solaris had zones, and we created the in in at Sun, these these branded zones that would pretend to be Linux.
• 17:41
  And the way we did it was, I think that my my joint friends will tell you, wrong. And the No. The really I don't think
• 17:50
  Speaker 1: so. It well.
• 17:51
  Speaker 4: The real Oh, well.
• 17:51
  Speaker 3: The real Oh, well. The real Oh, well. The real
• 17:52
  Speaker 2: Oh, well. The Right. Right. And the way that we emulated ptrace was with a shadow process that would use slash proc and then do this sort of half baked emulation of ptrace. And then, Josh, the thing that you'll really hate me for is that
• 18:08
  Speaker 1: Already does.
• 18:09
  Speaker 2: I think I think I think yeah. More for is I think we were granted a patent for this, but
• 18:15
  Speaker 1: I didn't okay. So I didn't see that one coming. Josh, I didn't know anything about that. I don't know. He's on his own on that one.
• 18:21
  But if you're not
• 18:21
  Speaker 3: Did you get a hat did you get a hat that said patented on the hat?
• 18:25
  Speaker 2: I think I I think I have a plaque even. Oh oh,
• 18:27
  Speaker 5: oh, good.
• 18:28
  Speaker 1: San San Francisco man, I guess you should know, Alameda man found belugined with plaque.
• 18:32
  Speaker 2: That's right.
• 18:33
  Speaker 3: Yeah. It's the same. We've we've we've determined the murder weapon. That's
• 18:35
  Speaker 1: Exactly. He
• 18:36
  Speaker 2: was actually by flag.
• 18:37
  Speaker 1: By his own patent plaque. Wow. Yeah. Someone must have been half crazed.
• 18:43
  Speaker 2: And and and, actually, to bring it back, Brian, I think it's the application of this German word that we're looking for, which was exactly that. Like, this clever application of this existing mechanisms that ultimately I don't think immediately was disaster.
• 18:56
  Speaker 1: So I don't think I would I because with ptrace, you're trying to do something very hard, and this is just in general very hard. We found over and over again when you're emulating 1 system on another. And, Tom, I know you've got a lot of experiences too on this too. You are having to emulate a system's bugs. I mean, I feel like as the as the one emulating the system, you feel like you are thinking about the semantics of the system more than the original implementers thought about it.
• 19:23
  And I I I think it, like, it whether you're implement emulating x86 or you're emulating a different kind of operating system. I don't know, Tom, what do you think about that?
• 19:31
  Speaker 5: Oh, yeah. You gotta be bug for bug compatible. And for each thing
• 19:36
  Speaker 1: And I I feel that, like, for me personally, it was the v fork, that we when we learned that Go was very dependent on the v fork semantics with respect to signal disposition, which is like, v fork ins I mean, v fork unsafe at any speed, toxic at any quantity, signals unsafe at most speeds, toxic in in in in many quantities. And, like, to take the confluence of them is unconscionable. And yet Go somehow manages to depend on it, to this day and deliberately. And it's like, so you're as the system emulating this, like, yeah, you gotta find a way to make that work. So I don't know, Adam, that I don't I don't think I I maybe I'm just trying to service this, your therapist here, but I don't think that I mean, Josh, maybe you can use an I feel statement when you talk to Adam about how you felt about inheriting a a half thought out ptrace mechanism.
• 20:32
  Speaker 5: Use your words. That's right.
• 20:35
  Speaker 3: I think that I feel. That's it. That's
• 20:39
  Speaker 1: better. That it doesn't feel better.
• 20:40
  Speaker 3: The it but the, the part that I remember being most frustrating was so the reason we even cared about it was not for s trace, if you recall. It was for Ubuntu's, short term attempt to be to be system d. It was for upstart.
• 21:00
  Speaker 1: Oh, it was for upstart.
• 21:01
  Speaker 3: And they were using it to track or attempt to track correctly, process life cycle across double fork boundaries by using ptrace to intercept all of the forks. Right?
• 21:14
  Speaker 1: You were already making Adam feel so much better because Adam is already like, dude, I was the least of your problems. Like, that is
• 21:20
  Speaker 2: Seriously. That's a mess.
• 21:22
  Speaker 3: And but so, like, the emulation that we had was pretty thin, and I think may have been dependent on signal delivery or something that just stopped happening in the context. And so, like, it it was just, we wouldn't do the thing that it was looking, and soft start just didn't work at all. It wasn't like it crashed. It just never got any of the information it thought it was gonna get. But it's kinda work.
• 21:51
  That turned that turned into a lot of surgery to implement it, with sufficient fidelity that processes couldn't escape.
• 21:59
  Speaker 1: Right.
• 21:59
  Speaker 3: And it had to work it had to work just like the real one. And in the end, we did get on the the back of that emulation, we did get STrace to work basically correctly, I think.
• 22:10
  Speaker 1: It was it worked so remarkably well. It worked so great. Because I I we're gonna get the in the show notes. We'll put the tweet of us. So a bunch of, like, basically non drinkers all hoisting a beer when Josh got this working, because it required us to to to put all that in the car.
• 22:26
  It was just it was insane.
• 22:28
  Speaker 3: Well, we were trying to lift if you recall, we were trying to do stack switching to begin with.
• 22:32
  Speaker 1: That's right.
• 22:34
  Speaker 3: And the emulation because our original model for LX was that the internal component was basically a trampoline. It would save some registers on the user stack a bit like signal handling, and it would just always return you back into this, shared library that we had secretly and mapped into the Linux process memory space every time for every system call. And so things like ptrace emulation could happen in the context of that user library, and it could make any any native system call that it wanted and go figure out a bunch of stuff. When we moved to stack switching, we also moved to to some system calls would be emulated entirely inside the kernel, and the kernel would be involved in knowing which emulation stack to drop things onto. And it just it it abridged very heavily the emulation path, which made things go a lot faster, but it also meant that the the whole of the existing ptrace emulation, which itself made, opened a bunch of proc files into things just wasn't gonna work out.
• 23:38
  Speaker 1: And, Josh, you may wanna explain why we had to do stack switching because that I mean, that itself is kind of an interesting story. That's, of course, Go.
• 23:45
  Speaker 3: The yeah. It was they they were using the red zone. Right?
• 23:48
  Speaker 1: Right. They're using segmented stacks. And we were Mhmm.
• 23:52
  Speaker 6: I mean, this is why
• 23:53
  Speaker 1: it's so hard to live as an emulator
• 23:55
  Speaker 3: because Linux system call API is passed by register with no stack requirement, and our emulation was not like that.
• 24:04
  Speaker 1: It was you you basically cannot you you can't live in in the usable stack at all. Like, you just cannot Right.
• 24:12
  Speaker 3: Because we it we we would use whatever the stack was. Whatever your existing stack, we would put things on your stack, which is normally okay because c processes usually have abundant stacks. And the fact that we use it, you can't tell if you don't know to look because we put it all back.
• 24:32
  Speaker 1: We put
• 24:32
  Speaker 3: it all back. Exactly. Return control.
• 24:34
  Speaker 1: As far
• 24:34
  Speaker 3: as you know.
• 24:34
  Speaker 2: Yeah. Yeah. Right.
• 24:35
  Speaker 3: Right.
• 24:37
  Speaker 1: Yeah. The and I I remember we also had and so we would have these bugs where it's like, hey. This application works on a Vodou and it doesn't work on LX or it worked or or even worse. Like, this application used to work and now no longer works. And I remember the one of those that was also, the real object lesson for me was where, application core dumped, but, was regression.
• 24:59
  We upgraded the operating system, the application core And, of course, I well, that's a query bug. What's going on? Well, it's just that it wasn't as query bug because the application was implicitly relying on phishing in its own stack and relying on a effectively, it was relying on data corruption. It was relying on the fact that this value would be 0. And on it's like we needed to push in additional value on the stack in order to work around one of the intel bugs.
• 25:25
  And in order to do that, it's like, yeah, the value you're fishing is no longer 0. Now it's non zero. And now with your your application is port up. Like, you're just like, oh, god. It's so hard when
• 25:33
  Speaker 3: you have Do you remember do you remember what closure common list did with the context? The m context, m e context?
• 25:41
  Speaker 1: Yeah. Right. We would
• 25:43
  Speaker 3: They would copy it somewhere else. Right.
• 25:47
  Speaker 2: No. No. No. No. No.
• 25:49
  Speaker 3: Yeah. Which is technically legitimate, as it turns out. It's just that we had stashed in the stack all of this other ancillary information, the real information. So we would give it the emulated version of this information, which was about the Linux emulated process stage, and our our second, like, real native stack stuff that would happen on a signal or whatever would not come along for the ride because it didn't know that it existed. It didn't know the ride happened.
• 26:14
  Speaker 1: It didn't know there is a ride.
• 26:15
  Speaker 2: Some some of these kinds of lies just don't nest. So the u context or end context are collecting register information. So if you've got some shadow copy somewhere else
• 26:25
  Speaker 3: Yeah. I've we collected really good information. It's just that it's just that we also collected, you know, additional secret information
• 26:33
  Speaker 1: that we we would
• 26:34
  Speaker 6: Yeah. And this is one
• 26:35
  Speaker 1: of the lessons I do. We we learned
• 26:36
  Speaker 3: With the with the emulation stack, the the the bad thing was, so like, we had moved the collection of that information to, a more secure location. But you still you still need to unfortunately, the the part that we would put on the emulated process stack, which by this point was separate from the emulation stack, which was somewhere else, you still have to chain them all together. Like, you have to pop the same number of contexts off the the emulation secret stack as you do off the native emulated thingy stack. Anyway, just it was a mess. And we did it we think I think we did it with magic values and fishing expeditions
• 27:17
  Speaker 1: It was bad.
• 27:17
  Speaker 3: As a as a fallback.
• 27:19
  Speaker 7: Why isn't that a performance problem?
• 27:22
  Speaker 3: For Yeah. Yeah. You don't wanna take signals, at an audible frequency, I think, in in general.
• 27:31
  Speaker 1: You don't have signals at all. Signals are such a Yeah.
• 27:34
  Speaker 3: If you could avoid it, but like CCL at least took a lot of signals because it was from an era, I think, where programs are single threaded and when stuff happened and you wanted to handle it, it would be because of the signal, possibly a nested
• 27:51
  Speaker 1: signal. But programs and I I I get the abstractions were lean, but programs that use these abstractions in a load bearing fashion, and I I think it it highlights one of the things that we've seen a couple of times I feel in my career that magic does not layer well. When you got someone in the stack being magical, they're kind of relying on people elsewhere in the stack, like, not being as magical.
• 28:14
  Speaker 2: Yep. Playing by the rules.
• 28:15
  Speaker 3: Playing by the rules.
• 28:16
  Speaker 1: And it's like, well, why should I play
• 28:17
  Speaker 3: by the rules?
• 28:17
  Speaker 1: Like, you're not playing by the rules. It's like, yeah. But one of us has to play by the rules.
• 28:20
  Speaker 3: And if we had had unlimited resources and time, right, I think that the the final end result of LX would have been to move all of the things into the kernel. Into
• 28:31
  Speaker 1: the kernel.
• 28:32
  Speaker 3: So that there was no user space visible thing that had to happen. And signal handling, our signal handling occurs about half in the kernel and about half in libc, you know, like a Lumos libc. So, like, getting to the point where we can do emulated Linux signal handling entirely in the kernel was gonna be a bunch of work and we just never pulled on those threads. Well, so
• 28:56
  Speaker 1: I'm always curious because the I I talked to the you know, the Microsoft folks were were very inspired by our work. And so they, like, you know, the WSL, the WSL one. WSL one was directly inspired. Like, you know, you guys showed that, like, this is possible. And I felt like, man, do you regret following us down this path?
• 29:18
  Like, I'm sorry if we let you down this.
• 29:20
  Speaker 3: Well, they they did because they stopped.
• 29:22
  Speaker 1: They stopped. I know. They they re
• 29:23
  Speaker 2: What what's what's WSL? Sorry.
• 29:26
  Speaker 3: They they The Windows Subsystem Yeah. The Windows subsystem for Linux. Oh. Oh. Oh.
• 29:30
  Oh. Which the name is on backwards, emulation thing that lets you run unmodified Linux binaries on a Windows system in a kind of context of some sort. And I think what Folks,
• 29:43
  Speaker 4: can I ask a question?
• 29:45
  Speaker 1: Yeah. Go for it.
• 29:46
  Speaker 4: Is it, what is the state of, running Illumus on m 1?
• 29:53
  Speaker 3: The on the like, on an ARM CPU? Yeah. That would
• 29:57
  Speaker 4: be the On an m, like an on an Apple.
• 29:59
  Speaker 1: On an m one, that would be, that would be a go for it. That would be that that's not, yeah. There's no there's no
• 30:05
  Speaker 3: No no one is gonna tell you it's impossible, but it's not It's definitely not. Yeah. It's not gonna happen before dinner.
• 30:11
  Speaker 4: No. But yeah. Sure. Nothing happens before dinner. Right?
• 30:16
  Or a lot of thing can happen before dinner, but I guess
• 30:18
  Speaker 3: I mean, if you had a thick pad, it could happen before dinner.
• 30:22
  Speaker 4: Right? We can get it in the pod before dinner.
• 30:24
  Speaker 7: It's just an ARM PC, isn't it?
• 30:26
  Speaker 3: Yeah. We don't
• 30:27
  Speaker 4: really do is
• 30:28
  Speaker 3: We don't really do ARM today.
• 30:30
  Speaker 4: So my question is there there could be different solutions to that. Right? There could be, like, q m QML or they could be just booting up an operating system.
• 30:44
  Speaker 1: I mean, the the thing that
• 30:45
  Speaker 4: we The story on m one is, like, I I kind of have, like, a virtual box. Right? Because it's m one and,
• 30:54
  Speaker 3: you you would need a full system CPU emulator, and it would run very slowly, I would think.
• 31:00
  Speaker 1: And Laura's Laura's our resident arm expert. She can she can weigh in.
• 31:04
  Speaker 5: Yeah. Not really m one is fast. M one's really fast, and Kim Canvoo is pretty good. So you you can do an amazing stuff with pure emulation.
• 31:13
  Speaker 1: Laura, hit it. The thing
• 31:14
  Speaker 8: with the m one is is that, like, it's it's it's not fully standards compliant. I know I'm sending people around hissing whenever I say this, but it actually turns out the fact that Apple is kind of doing their own thing makes trying trying to boot up, you know, an operation that is not OSX, like, a lot of work. See, also the work with for Linux to try and get what's actually merged with the Linux kernel, for example, moved in is basically just a bare skeleton. And how long it'll take anything else, you know, remains to be seen. But I think there's been a lot of controversy there, so I mean it's certainly a labor of love, I think, to try and do it.
• 31:46
  But Well,
• 31:48
  Speaker 3: Laura, can I ask you, how do
• 31:49
  Speaker 1: you feel about, like, personally feel about m one? Is it does m one feel vindicating or does it feel, like, threatening to kind of the principles of armed from terms of the openness or maybe both?
• 31:59
  Speaker 8: I mean, I'm happy to see it going forward, but, I also kind of have strong opinions about running, non Apple operating systems on the hardware. Like, I support everyone's right to try and do so, but I think it is not a the best use of effort to try and do this mostly because exactly Apple has, you know, chosen to make it difficult to try on anything anything else. So I think you're you you you're fighting an uphill battle, and it's just it doesn't seem like it's worth it to try anything else.
• 32:29
  Speaker 3: Which was also true on their x86 systems. I mean Yeah. Like, it it's always just it's always been kinda crap to run Linux OBS 2 or something else on on, like, even relatively old MacBook Airs or something. Like But
• 32:44
  Speaker 4: what but what if there was, like, a virtualization layer? Something. Like, I don't know.
• 32:50
  Speaker 1: It's hard. I mean, it's hard to because it's hard to get the the machine really properly emulated. And I mean, I really, as Laura can attest, when we were starting to do Cortex M based development, I really, really, really wanted QEMU to work for full machine emulation. And it it's it's really spotty and tough. So it will be I think an m one emulator would be great.
• 33:15
  I think it would be hard, to be have a complete emulator.
• 33:21
  Speaker 5: The thing that amazes me is AWS has a macOS offering for people who wanna do cloud development. And, it's just a bunch of Mac minis thrown together.
• 33:33
  Speaker 4: Yeah. I think I saw a promo video.
• 33:35
  Speaker 1: You you you would think
• 33:36
  Speaker 5: if anyone could get Apple to virtualize macOS, it'd be anyone AWS.
• 33:41
  Speaker 3: I you know, I I don't believe that Apple virtualized inside a whole lot necessarily even. Like, you would you would you would think
• 33:48
  Speaker 5: VMware Fusion, right, where you could have
• 33:51
  Speaker 3: Yeah. But that's gotta run on OS 10 already.
• 33:54
  Speaker 5: Right.
• 33:55
  Speaker 1: So maybe some long time Apple watcher can give kind of insight into this, but it is kind of amazing that the world's most successful computer company, single computer company in terms of Apple, has really never kind of grokked or had interest in the service space, other than X1. Adam, did you have an X1? I think we had an X1.
• 34:11
  Speaker 2: No. No. I mean, I know I was a big, like, Apple super fan back in the day, but no. I never had an X1. I I was gonna do a throwback to, like, the chirp systems because I'd like
• 34:20
  Speaker 1: Oh, chirp. Yeah.
• 34:23
  Speaker 2: Like, the and their clones. Like, my brother had a, a, like, a Power Mac clone. I can't remember the name of it. Power Computing. There it is.
• 34:33
  And then just yanked it all back, when when Steve Jobs showed back up.
• 34:38
  Speaker 1: Yeah. So the Chirp was the the common hardware reference platform. Right? That was
• 34:41
  Speaker 2: That's right. It it was, like, ostensibly, I think, with IBM to create this open power PC kind of standard thing. I I think I I don't think it got very far.
• 34:52
  Speaker 1: I mean, this is obviously the value for Mike Sullivan's laptop. I don't know if you're trying to take me there
• 34:57
  Speaker 3: or not. But the so,
• 34:59
  Speaker 1: a colleague that I worked, it was in the next office at Sun, was responsible for a bunch of build infrastructure. And, his laptop was stolen, which is very weird. This is the what is now the Facebook campus. So if you're on Facebook, you're in m p k 17. I wish Facebook had renumbered the buildings.
• 35:15
  I just feel like it was that's a disrespectful for the dead. Like, you're living in the carcass. Like, can you at least like to anyway.
• 35:22
  Speaker 3: Have you seen the sign
• 35:23
  Speaker 1: at the time? Know the sign that that it's it's very it's fetishizing the dead. I feel like, let's bury the dead. Let's not fetishize them.
• 35:30
  Speaker 3: You can go on a pilgrimage to what feels really honestly like a graveyard. It's some kind of Sleepy Hollow thing.
• 35:35
  Speaker 1: A graveyard that is, like, up to 1
• 35:37
  Speaker 3: There's 3 little trees, there's a clearing, and then you see, like Right. There's a body hanging from a tree and, like yeah.
• 35:44
  Speaker 1: It's Day
• 35:45
  Speaker 2: of the Dead vibe as you walk around certain places. Yes.
• 35:47
  Speaker 1: So the anyway I don't know the idea. Mike's laptop was stolen. And Mike's laptop was a was actually a PowerPC laptop running Solaris 251, a very ill advised experiment because we've done the endingness wrong. We did a little endian port, not a big endian port. And so we were the only little endian PowerPC implementation on the planet.
• 36:13
  There were only, like, I mean, we had fewer than 20 machines, like a boot Solaris 251, and one of them was stolen. And I would have loved to have been like, what happened to that laptop? If I could have, like, the webcam on the adventures of that laptop as the thief realizes that they have the actual most worthless laptop you could possibly steal in 1997. As they attempt to it's like, the this is the only operating system it can run. It's a terrible operating.
• 36:39
  It's an operating system for which there are no binaries other than, like, CD. It's it's
• 36:42
  Speaker 3: in a it's in small paces at the bottom of the bay at this point, I assume.
• 36:47
  Speaker 5: But you are vindicated because years later, Open Power is little Indian.
• 36:52
  Speaker 3: Open Power is
• 36:53
  Speaker 1: little I don't think I realized that, that Open Power is little Indian.
• 36:56
  Speaker 5: Yeah. To get to get the Linux stupidity compatibility.
• 37:01
  Speaker 1: Really? Yeah. Sorry. Go ahead.
• 37:04
  Speaker 4: Brian has a talk that is, on YouTube that is, like, a gathering of a lot of old sound folks, I think.
• 37:12
  Speaker 1: Yeah.
• 37:12
  Speaker 4: It seems like it. And it's and there's a story of a laptop there. Is that the same laptop?
• 37:17
  Speaker 1: That's gotta be the same laptop. I'm sure. Are you are you accusing me of of retelling stories? Because
• 37:23
  Speaker 4: I No. No. No. I I wanted to get the No.
• 37:25
  Speaker 2: No. I think that's the same.
• 37:26
  Speaker 4: I found that talk really, really interesting. I I don't know I don't know what I found about it interesting, but I found a lot of things about it interesting.
• 37:35
  Speaker 1: We're all living fossils talking
• 37:36
  Speaker 2: about it.
• 37:37
  Speaker 7: Accuse him of reusing stories or we'll start hearing about the h programming language.
• 37:40
  Speaker 1: Exactly. Hey. Listen. That's first of all, that's called language h, first of all. And and, you're gonna really you you don't you
• 37:47
  Speaker 3: know, you'll play with fire, Aaron. You know what I mean?
• 37:49
  Speaker 1: You can't just bring up language h and then walk away. The in the we were talking about, the, The Friendly Orange Glow, this great book by Brian Deer last week. And one of the characters in that goes off to NCR and works on a COBOL like language at NCR. And I'm like, oh my god. This is this is language h.
• 38:07
  This person went to work on language h. So I'm gonna I I wanna go yes. I wanna go hunt down some language h.
• 38:15
  Speaker 4: I mean, it's sorry. I have to I have to just bring this up. There is also I don't know about language age. I guess, like, I need to, like, do a little bit of No.
• 38:27
  Speaker 1: No. You don't. Actually, no. You really don't. It's a it's a it's a language that you haven't heard of for reason.
• 38:31
  Don't worry.
• 38:31
  Speaker 4: There's there's another language that I've heard of, and it's, the d language. Yes. It's a d language that is so I I actually looked at the d language, the d trace d language. And, I I don't think I've coped around that much because, unfortunately, there is a bunch of d trace hooks, in, Mozilla's spider monkey, but they're not maintained anymore. Yes.
• 39:01
  So that was, like, the closest that I had an encounter with, and I was like, okay. Time to time to do some d trace. But, I I couldn't get that experience. But I did learn a little bit about the e language and the demo with, diagnosing the, twanky behavior in one of the components in GTK or X11, which was delivered at Google?
• 39:32
  Speaker 1: Yes. That was the detrace of your Google. Yeah.
• 39:35
  Speaker 4: Yeah. So I I feel like the d language, that, kind of, like, programs the trace?
• 39:45
  Speaker 1: Uh-oh. Is that me?
• 39:49
  Speaker 2: Or is that him? Cliffhanger. No. That's him.
• 39:50
  Speaker 1: That's him? Okay.
• 39:51
  Speaker 3: Makes sense.
• 39:52
  Speaker 1: You so I've upgraded my app. I and I am you'll notice do you notice that we're 40 minutes in and I've not died?
• 39:58
  Speaker 3: Impressive.
• 39:59
  Speaker 1: It it is impressive. And
• 40:01
  Speaker 3: Yeah.
• 40:01
  Speaker 1: The, one of the the Spaces folks DM'd me saying, hey. You may wanna upgrade your app. So I so I did that. And so here we are.
• 40:08
  Speaker 5: There you go. Good.
• 40:09
  Speaker 1: So, of course, when when when he's when he when Nivo stopped, I was just like, oh my god. It's happening again. It's I'm I'm traumatized by, I thought he was actually gonna talk
• 40:20
  Speaker 3: about It is about the time it usually happens. It is
• 40:22
  Speaker 1: about the time. I I thought he was gonna talk about Walter Bright's d. The other d the the the, frankly, the more famous d d language. And do you remember our our interaction with Walter Bright, Adam? No.
• 40:34
  No. I don't. He Walter Bright's d Walter Bright of Borland and c plus plus fame. And the d language, like, again, like, much more d lang, much more, I think, much more famous than DTrace. But the our d and his d were developed in parallel and in secret.
• 40:51
  And when we made we talked about d trace in d, he's like, well, this is interesting, but you're gonna have to change the name of the language because I've got a language called d. And, like, we're like, our language, first of all, is not Turing complete. So, like, go I mean, come on. You can't take you can't take us very seriously. 1, 2, it's a letter of the alphabet.
• 41:08
  Like, I don't think you can copyright those. I don't think you can trick I think you got to pick a more distinctive name if you wanna have something defendable. Defensible. So
• 41:16
  Speaker 2: Plus it's like, it's the one after c. Like, we're both going for the same pun here.
• 41:20
  Speaker 8: Right. So
• 41:21
  Speaker 1: And it's, like, not that funny. I mean, it's not.
• 41:23
  Speaker 2: Right. Right. That not that funny of a joke.
• 41:26
  Speaker 1: Right. It's not that funny of a joke. Right?
• 41:27
  Speaker 3: We don't
• 41:27
  Speaker 1: need to fight over it. We can both just have the same not funny joke. But d is RD, very inspired by Och. Yeah. Of course, I mean, it all comes it all comes back to Och.
• 41:39
  I mean, Tom, Och must have felt like a just an absolute, like, blast of modernity when that happened.
• 41:46
  Speaker 5: Yeah. That was very cool. I was I was right there that summer. It was all coming together. So, yeah, people were doing crazy stuff with it.
• 41:58
  Speaker 1: So you were you're there with you got, what, Weinberger and Ajo and Kernaghan all developing this together.
• 42:05
  Speaker 3: Right. Right.
• 42:07
  Speaker 1: Was it named at that point? Are they
• 42:10
  Speaker 5: Yeah. Yeah. Pretty early on.
• 42:14
  Speaker 1: Because I I don't
• 42:14
  Speaker 5: remember when.
• 42:16
  Speaker 1: I mean, it's it's kind of, like, genius to use the the I mean, I'm are are there other Unix commands that are named after their that have the, an o to the authors in its name? I don't know. So you
• 42:31
  Speaker 3: The born shell?
• 42:32
  Speaker 1: It's true. The born I guess, maybe yeah. Maybe this was, like, a thing.
• 42:36
  Speaker 3: But I
• 42:37
  Speaker 5: mean But
• 42:37
  Speaker 3: it was for that, the Thompson shell. Right? I mean, I guess it's not in the name of the command, technically.
• 42:42
  Speaker 5: Yeah. Yeah. Because the Born shell because of the way it was written in Algol 68.
• 42:48
  Speaker 3: It is not the greatest c.
• 42:50
  Speaker 1: No. It is not the greatest c. I I love Steve, but was born, but Jesus Christ, Steve. It's not Algon. You can't do what you'd like.
• 42:59
  It's it's wrong. What you did is wrong. For those of you who have not been to the source code, pound to find begin as open brace and pound defined end as close brace. If I recall correctly. John?
• 43:11
  Speaker 3: Yeah. And and, like
• 43:11
  Speaker 5: worse than that.
• 43:12
  Speaker 3: A bunch of the rest of the language. Like
• 43:17
  Speaker 1: Yeah. What was worse than that, Tom?
• 43:19
  Speaker 5: Well well, he he had all kinds of Algol 6 a constructs.
• 43:23
  Speaker 3: So the, yeah, the source code
• 43:25
  Speaker 5: was not immediately identifiable as c.
• 43:29
  Speaker 3: Because it was all in, like, capital letters and, like, if and phi and then and a bunch of other stuff that
• 43:36
  Speaker 1: See, in the in the world of barfel is solved.
• 43:40
  Speaker 5: I mean, the the use of the shell also
• 43:42
  Speaker 3: has has all that was algorithm. I'll I'll know if it is in
• 43:46
  Speaker 1: it. In the actual language?
• 43:49
  Speaker 5: Yeah. The s and 5. I don't know what to
• 43:52
  Speaker 3: Case and ESAC. Right?
• 43:53
  Speaker 1: Right. Right. Those are algorithms. Yeah.
• 43:56
  Speaker 3: I guess I hadn't really put 2 and 2 together on
• 43:59
  Speaker 4: that. I'm so sorry. My phone died, and I just did I just lost the past 5 minutes of the conversation.
• 44:05
  Speaker 1: Don't worry. We're just waiting for you to rejoin having a fight or flight reaction because I assume that Twitter Spaces had actually died on me again, which it hasn't because I've upgraded the app. So I'm optimistic that's not gonna die.
• 44:14
  Speaker 4: I guess it's okay because we experienced that last week, but and it was fine.
• 44:19
  Speaker 1: And the week before and the week before. It just died every time.
• 44:22
  Speaker 7: It is still amazing how bad the visibility is on Android. If I wanna know is this app leaking memory and slowly linearly growing to its own death, That's actually kinda hard to tell on an Android.
• 44:34
  Speaker 1: Okay. So that's not just me because I actually have tried it's it is unfortunately, with Twitter spaces, it's very hard to do anything else on your phone while you're in the Twitter space. But the, yeah. So Android, I I I I wanted to look for better tools for that. It they don't exist?
• 44:48
  Speaker 5: Uh-oh. There he went.
• 44:49
  Speaker 7: I don't know. What is people's preferred htop for Android?
• 44:54
  Speaker 1: Yeah. I've
• 44:55
  Speaker 3: I think we lost you.
• 44:56
  Speaker 1: Tom, I we've not lost you. Have you lost me? This is
• 45:00
  Speaker 5: I I lost you for a minute. You're Okay. Breaking up.
• 45:05
  Speaker 1: I, so sorry, Nima. You you you you took a a 5 minute vacation, but I think you're you're asking about the the detours of you talk at Google circa that's such a
• 45:14
  Speaker 3: yep.
• 45:15
  Speaker 4: Yeah. I guess I built a lot of I I built a lot of context because there was this probably very selfish of me, but I guess it's a question that I care about a lot. I guess in terms of you know, DTrace is, like, such a highly domain specific language. Right? But in terms of ergonomics and how you can sort of declaratively just say what you want and just collect data in various different sorts of way, I would I was feeling more comfortable in the detrace language than I was like.
• 45:58
  Speaker 3: Oh, is
• 45:58
  Speaker 1: that me again?
• 46:00
  Speaker 4: So I was wondering
• 46:04
  Speaker 1: Yeah. But Twitter spaces knows you're trying to praise us, so it's trying to cut you off. I kind of admire it. It's got it's got some new new filters for it, knows you're trying to praise Dee.
• 46:13
  Speaker 4: Okay. I guess
• 46:14
  Speaker 3: Yeah. Yeah. We can hear you. Can you
• 46:15
  Speaker 4: hear me now? So what what I was saying is that in terms of, like, working with working with a language that aggregates data and formats it and prints it out, I have found the entire, like, DTrace experience very somewhat ergonomic, compared to SQL or VisiCalc. But it is it surprises me that DTrace is such a domain specific language. You know? Even though it's like querying operators and aggregating operators are pretty generic.
• 46:50
  You know? Like, you can think of the same kind of syntax over a relational database. Right? So I was, wondering what what went into designing that sort of, like, high level layer that programs dtrace? Like, who was behind it?
• 47:08
  Like, was it a iterative thing, or were you inspired by SQL and gave it, like, a different syntax? Like, what was what what was it?
• 47:16
  Speaker 1: Not inspired by SQL. Although, it may I mean, it actually in terms of Adam's personal career, Adam, I think I dare say, like, sequel came later for you. Right? I mean, I don't think you I think
• 47:25
  Speaker 3: Absolutely. Yeah.
• 47:25
  Speaker 1: So the For sure. Very much inspired by Oc, for sure. We were talking about Oc. I mean, definitely took great inspiration from Oc and its ability to very quickly, put things together. And then in terms of the aggregating, operations, you know, honestly, that just came out of using it.
• 47:46
  And I I do feel like this is one thing that is, like, really important for anything to be ergonomic. I feel its creators really need to make use of it. And we used DTrace a lot to debug lots and lots of things. I mean, we were the biggest users of dtrace, Adam, by the way. So I mean, correct me if Yeah.
• 48:02
  Speaker 2: No. I I totally agree with that, Brian. And I think that and and if you agree with this, but I don't think any of us was really a programming languages nerd No. Of any description. And, and one of the things that was very stark for me as I look at some of the d trace clones and sort of, offshoots emerged is how much more structured some of the language was and how much more boilerplate they required and how much more fussy they were about about a lot of things that the d language is not that fussy about.
• 48:35
  Mostly to its benefit, to its ergonomic benefit, but occasionally just to, like, total misfortune. But to your point, Brian, it was it was all based on, like, us exploring some phenomenon, something being kind of a pain in the ass or impossible, and then venting something that was easy to use.
• 48:54
  Speaker 1: And it was all about that actual use. And so we were optimizing constantly for our own use, and I think it was like Auk in that regard. And what I didn't realize at the time and I'm do you remember when going in from our architectural review board? They one of the comments was, you know, honestly, this reminds us a lot of OCC. And we were like, that is high praise.
• 49:14
  Thank you so much. I am, you know, they'd be these Peter Weinberger is an idol. And then you realize, like, oh, wait a minute. That's not intended for this praise. That was actually intended as, like, damnation because Ock from a PL perspective, I think is viewed somewhat pejoratively.
• 49:29
  I mean, if you're a PL person, like, if you're gonna correct me if that's that's an incorrect inference. But I think Ock is viewed somewhat pejoratively, and it shouldn't be because it's so pragmatic. And it's it is so easy to to stitch things together. And it obviously and, Tom, I would love to know, you know, you were there in that in in that summer. I mean, you get the sense that Aqua is being used to by its creators to do things.
• 49:57
  Speaker 5: Oh, yeah. It's basically it it's kinda like how what's the most powerful one liner you can crank out with ARC.
• 50:05
  Speaker 2: Right. Yeah. That was that was exactly what what, you know, we were up to
• 50:08
  Speaker 1: with These rights for sure.
• 50:10
  Speaker 5: Yeah.
• 50:10
  Speaker 1: Yeah. It's so funny to think about that exact same kind of echo, you know, exact Tom, that's the summer of 77?
• 50:19
  Speaker 3: Right.
• 50:19
  Speaker 1: And so and we're in we're, like god. I actually can see what's amazing, Adam, is it's only, like I know. It's crazy. It's only, like, it's only, like, 20 3 or 4 years later that we're doing that.
• 50:29
  Speaker 2: I know. I know.
• 50:30
  Speaker 1: Like, 1977 is almost closer to when we were developing DTrace than when we were developing DTraces than now. That's not quite true that that's close.
• 50:38
  Speaker 3: That's right.
• 50:38
  Speaker 2: That is insane.
• 50:41
  Speaker 1: But it is it's it's an interesting kind of echo. And, you know, the there was a, remember the some CTO, actually, Greg Papadopoulos, we we like despite the fact that despite NEA's view on oxide. Sorry, Greg. The, but, you know, he had the this line that he that he felt that any just to, Niamh, to your point about about the domain specific language, that the biggest advances in computer science had a language that went with them, which I think is actually really interesting. Right?
• 51:10
  Because you see that in CUDA with NVIDIA and the GPGPU. I mean, certainly, we see it where things are more excited about, like, BlueSpec. I mean, obviously, I think ROS is a big deal. But I think it's like I'm not a PL person, but it's really hard for me to deny that that language is really stitched in closely with our biggest innovations, in computing systems, not even not just computer science.
• 51:32
  Speaker 5: It's just too damn hard to figure out how how to use something without the language to guide you.
• 51:37
  Speaker 1: Yeah. There's it it is. It's something like that. It's where it's like you need that that domain specificity. It does it.
• 51:44
  It guides you, Tom. That's it. That's, and that's an interesting way of thinking about it.
• 51:50
  Speaker 2: But but there's an interesting lesson there in terms of the the survivor bias bias, and and that statement, which is, you know, not every every new language has that kind of durable, like, persistent effect. Is anyone hearing this very strange buzzing? Anyway, just
• 52:13
  Speaker 3: No. That's But
• 52:14
  Speaker 2: but That's just me.
• 52:15
  Speaker 1: No. That's the microchip that Josh planted in your brain he's
• 52:18
  Speaker 3: acting in his eyes. Just say we're just I hear
• 52:21
  Speaker 2: you talking. But right. Same thing. But, but, like, the the languages do persist that do persist, like, have this this long shadow. Whereas but having a new language can mean your technology is very hard to adopt.
• 52:34
  Speaker 1: It it it I actually this is also one of the things that I know, Adam, you and I both love proc macros. I don't know how, like Laura, you Laura, Josh, do you do you Laura, I know you but you you've got the you've gotta have the same love of of proc I mean, I just feel like proc macros in Rust are gonna allow for a new level of domain specificity that's also rigorous? Am I
• 52:59
  Speaker 3: Yeah. I mean, it's like you're gonna be able to do amazing, crimes and other things, I
• 53:07
  Speaker 5: think,
• 53:07
  Speaker 3: at the same time. Right? I mean, it's like some some depending really depends on your point of view when you pick up that tool, I think.
• 53:14
  Speaker 4: So diagnosing, I'm I'm not a Rust person, but I had, like, a brief experience with Rust. But diagnosing error messages with the way macros expand is somewhat difficult. Oh. And I I I sort of under I I get the power of, macros.
• 53:34
  Speaker 7: I think that's every language with macros though.
• 53:36
  Speaker 1: No. It's amazing in Rust. Rust. I can't look. It's amazing.
• 53:40
  Speaker 5: It
• 53:40
  Speaker 1: uses a We're about to be very on brand.
• 53:42
  Speaker 4: Rust makes like a heavy Rust makes a heavy use of macro in a way that I've really never seen in any other language. And this the thing is, macros in Rust expand into things that the that the compiler can handle really well. Like, I don't know the details. Right? All all the way to, like, how the structs are layered in memory and whatever.
• 54:07
  But, that kind of macro system is really needed across all compiled languages. Like, c plus plus needs that a lot. Like, if you ask any, like, GLSL folks, like, anybody who's, like, interested in, like, a staged method programming, a lot of a lot of GPU folks may may may relate to that. And they may appreciate Rust's, macro system. And they're like, they're a bunch of c plus plus efforts by, say, like, NVIDIA folks who try to do, like, const expert.
• 54:44
  That's, like, future of const experience c plus plus. I don't necessarily, like, follow that, the kind of the use cases that our program doesn't involve those kind of things. But, the the so it's like, meta programming is, like, generally, it's like a very difficult language
• 55:02
  Speaker 1: It is. But so the the error messages you can get in because I think, historically, you're onto, like, a a big historical problem. Namely, when you got when you got errors in that macro expansion, how do you
• 55:14
  Speaker 4: Well, I I personally got super confused. Like, my experience was, with the this library called salsa, which I needed it for So you'll be Things between build systems and so on, but it it it's some sort of like a a It'd be
• 55:36
  Speaker 1: It'd be sorry.
• 55:37
  Speaker 3: It'd be interesting to take
• 55:37
  Speaker 1: it apart because the I can tell you that the Rust from the proc macro, from the Rust language perspective, gives macro authors incredible power to generate ridiculously good error messages. I mean, error messages that are I mean, honestly, I thought they're eye popping. Adam, it sounds like you had to say a similar kind of experience.
• 55:56
  Speaker 2: Absolutely. I mean, I think on one hand, I totally agree with the macros that are completely undebuggable, where the error message that you get will be indecipherable compared with what you had typed into your program. On the other hand, the latitude that you have to know what, what span, like what, what section of code the user typed in, was responsible for a particular input value to know to be able to correlate with the compiler and say, please underline this statement and print out the following value. It
• 56:30
  Speaker 4: it I didn't have any Marley, we
• 56:33
  Speaker 3: we call them we call them proc macros, but it it may be better to think of them as compiler plug ins, really. Right.
• 56:41
  Speaker 4: I didn't even know the thought went went into that in Rust because, yeah, I didn't know, like, people have thought about, like, error handling at the macro Oh.
• 56:52
  Speaker 1: They it is it's amazing.
• 56:54
  Speaker 4: I guess, like, I'm in for a great Yeah.
• 56:56
  Speaker 3: It is you're effect you're effectively writing a Rust program that has access to the the the POS tree and the token stream. Right? I mean and and can then control the compiler's behavior and its output.
• 57:09
  Speaker 1: And and then on top of that, can when it has bad input, can and it was it sounds like Adam and I had the same reaction to the same thing, namely your ability to underline within a line the particular span that actually caused you to not be able to continue processing. And it's just, like, crazy how what we can
• 57:26
  Speaker 6: do.
• 57:27
  Speaker 2: And then on top of that, the on top of that, the Rust compiler
• 57:31
  Speaker 3: will will
• 57:31
  Speaker 2: also, you know, present to the user, this is what the macro told me what was wrong. But tell you what, I don't necessarily know this guy. So I'm gonna tell you what I think is wrong too. So even as a macro author, you can't override the compiler. You can only amend to like, try to try to specify in greater detail what's going on, but the compiler is gonna tell the user its honest thoughts as well.
• 57:54
  Speaker 1: And I think it it
• 57:55
  Speaker 7: So is it fair to say that Rust thinks the quality of life of a macro user is more important than the quality of life of a macro writer?
• 58:02
  Speaker 1: I think it does not wanna pick between its children, but, it it it thinks that both are very important. And in particular, what it thinks is the debugability of macros all around are very important. The things that's important to people and I can tell you from a having done things that were much less powerful, but as powerful as CPP could do, the CPP processor, where you got none of this. And, you know, we made use of preprocessor very heavily, and I've made use of preprocessor heavily in lots of things. And you've got no net, and what you can do is
• 58:33
  Speaker 3: It's completely completely hygien free.
• 58:35
  Speaker 1: Yeah. It is high it it is so nonhygienic that I kid you not, I did not know how to spell hygienic before Russ's macros. I didn't there's another I in there. Like, it feels like there should only be one I, but there's another I hiding out in there. I'm like, oh, well, there you go.
• 58:49
  That's the filth I've been living in. I don't know how to spell hygienic. Hy hygienic? Different name. Plug ins sounds too, I don't know, sounds too like Well,
• 59:06
  Speaker 3: because Rust also has a slightly more hygienic version of the CPU Processor as well in in terms the the macro rules, macros.
• 59:15
  Speaker 1: The macro rules, macros, and they are great. And you're like, wow. What is it we're talking about with proc macros? I think it's like I view it as like a compiler channel programs. You've you've got this, like, little program that is gonna be executed.
• 59:26
  I it is though, like, when you first start experimenting with it, you're like, wow. Is this, like, legal? Is this I mean, I feel you know, this is a language that's made such a big deal about being safe, and that's gonna wow. It just allows me to be right on the brain stem, and I get to do whatever I want. I mean, not whatever I want, but
• 59:43
  Speaker 2: it did reading about it for the first time. It was, it felt like the forbidden fruit.
• 59:47
  Speaker 1: It it is the forbidden fruit. It really is.
• 59:50
  Speaker 3: And I think in the same way, we we
• 59:52
  Speaker 2: have some colleagues who, who feel like it it it should be more forbidden than we're treating it. That's because it it, I mean, it does mean like when you're writing Algo 68 as your DSL in your Rust program, it all sort of just works.
• 1:00:07
  Speaker 3: Presents a a formatting challenge for for one thing. It's like you've created a different language other than Rust. So how do I even indent that now inside my broader Rust program? You know?
• 1:00:19
  Speaker 1: Well and and, actually, I think it's a hacker news story today about effectively writing Python within Rust about with a and I think we're gonna see more and more of this. I think we are gonna see
• 1:00:28
  Speaker 3: more and more Python, there's a a Java proc macro
• 1:00:32
  Speaker 1: Right.
• 1:00:32
  Speaker 6: Experience on this one?
• 1:00:34
  Speaker 1: Surely. Oh, and it would be it's something that would be highly attainable. And I think it's actually I think it's great because it allows we're just talking about how important it is to have language innovation to just to Tom's earlier point about, like, showing you where to go and to add DSL kind of specificity on top of Rust, I think it's gonna incredibly powerful. I think it's it's gonna be really neat.
• 1:00:57
  Speaker 3: All all all of my subsequent libraries are gonna be, accessible only through TCL. There you go.
• 1:01:04
  Speaker 1: In in your RISE program. You know, I I I knew you were gonna try to punish me with with
• 1:01:08
  Speaker 7: I I have written more It's too soft than
• 1:01:10
  Speaker 1: I have in my entire
• 1:01:11
  Speaker 5: career. It's time to bring back InterCal.
• 1:01:16
  Speaker 1: I do. I feel like we are bringing back InterCal with some of the Tickle because all of the the EDA world is still in Tickle.
• 1:01:23
  Speaker 5: Oh, yeah.
• 1:01:25
  Speaker 3: Well, and and So
• 1:01:26
  Speaker 4: it's so sad because, Jonathan sorry. I just have to mention this and then likely, take myself off from this because Jonathan Turner was joined this space. He was, like, a person who knows a lot about rust and ergonomics of rust and various features of rust. But then they they joined this space, but then they left so early. But I feel like they could have had, like, a great response and guidance about, like, all the points we brought up about macros and drives or
• 1:01:54
  Speaker 2: errors we do.
• 1:01:55
  Speaker 3: Just it goes to goes to show when you talk about featurized people.
• 1:01:58
  Speaker 1: I know. That's it. I want the analytics. I wanna be able to demonstrate. You have a shadow of a doubt that it was vfork, I think, that did it.
• 1:02:03
  Speaker 3: I think Vfork started out at it started out at 13,000,000 households, but then,
• 1:02:07
  Speaker 4: like, you
• 1:02:08
  Speaker 3: know, after the ptrace discussion, no one came back after the commercial.
• 1:02:11
  Speaker 2: I I think we've been very restrained for the past few weeks to not just have a completely complete Rust loving. And and I'm sure
• 1:02:20
  Speaker 4: I'm proud of the DTrace conversation we have. I got the answers I wanted.
• 1:02:25
  Speaker 3: So So the feature request
• 1:02:27
  Speaker 7: there is time series data on who's listening?
• 1:02:31
  Speaker 1: I've got lots of feature requests. If we're talking 200 spaces, I wanna lie I have a, like, a lot of analytics, and I wanna be able to see the v fork effect, as everyone, like, hangs up
• 1:02:41
  Speaker 2: on v fork. Node.
• 1:02:43
  Speaker 1: Right. Francesca, you're trying to get in here. Just to be able to pronounce it correctly.
• 1:02:48
  Speaker 6: Can you hear me?
• 1:02:48
  Speaker 1: Yep. Yeah. Sure. Whoops.
• 1:02:56
  Speaker 6: Oopsie. I I was just trying to complement the thing about DTrace that we liked it so much. I mean, plan 9 and 9 front that we implemented it Not fully for sure because we're missing, like, stack captures and some of the batteries batteries kept caption features, but it's so nice. And, like, there's people that don't like the syntax, but if you can explain it in less than less than a page, like, it it's useful. You know?
• 1:03:35
  Speaker 1: I I obviously Did
• 1:03:36
  Speaker 3: plan did plan 9 contain orc? I can't I I mean, I know that there were things like make were re remade with less vowels and stuff, obviously.
• 1:03:46
  Speaker 6: It contained the the first of implementation, I think. Compiled to see, obviously, and then compiled to.
• 1:03:58
  Speaker 1: So I think we're we are at about and we're actually a little over an hour. We are unaccustomed to being able to go through the entire hour without having to, like, reboot and get, like, chase out of where we were and have to get into a new space, but it all works. So, hey, Kudos. Spitterspace team is getting better, I think. Yeah.
• 1:04:14
  Speaker 3: But I can still touch my phone. It's not melting my fingers.
• 1:04:18
  Speaker 1: And and I I know we we said this last week, but I think we mean it this week, that we are going to, wanna talk about Silicon Cowboys. We'll kick off with Silicon Cowboys next week with the I know that Steve talk, our boss, wants to join for that discussion, and, couldn't make it today. He's working on a board deck, which is definitely much more important. But the, Steve wants to join us so we can talk about Silicon Cowboys and Compact, which is a super interesting kickoff. So we will do that next week.
• 1:04:46
  I promise.
• 1:04:48
  Speaker 3: Is that the Netflix documentary or is that a book?
• 1:04:51
  Speaker 1: It is a there's a book called Open by Rod Canyon, which is also worth reading. And there is the, it's a I believe this is on Netflix. The the documentary can be found on Netflix.
• 1:05:02
  Speaker 3: I think I've seen that one. It's good.
• 1:05:03
  Speaker 1: It's good. It's worth watching worth watching with the fam too. It's, it it's it's a good watch for sure. A lot of it still rings true.
• 1:05:12
  Speaker 5: And they thought they had a bug for bug compatibility issue too.
• 1:05:16
  Speaker 1: Yeah. They definitely did.
• 1:05:17
  Speaker 2: Yeah.
• 1:05:17
  Speaker 1: They definitely did. Adam, any parting thoughts?
• 1:05:20
  Speaker 2: Really looking forward to next week because Silicon Cowboys is awesome and touches on exactly as Tom was saying, these kinds of, like, systems problems of, of making Magic Lair and, making your system look like somebody else's.
• 1:05:33
  Speaker 1: Alright. Hey. Thanks so much, everyone. Really looking forward to next week. But really, we're we're having a a blast doing this.
• 1:05:39
  Hopefully, you are too. And hopefully, we'll see you next week.
• 1:05:42
  Speaker 2: Take care. See you next week. Bye.
• 1:05:44
  Speaker 5: Bye bye.
• 1:05:44
  Speaker 4: Goodbye.