Latest / The MonkCast / "Don't Submit Code You Don't Understand": Angie Byron on AI and Open Source Trust

Transcript

"Don't Submit Code You Don't Understand": Angie Byron on AI and Open Source Trust · The MonkCast

• speaker-0: Hello and welcome to this Red Monk Conversation. My name is Kate Holterhoff. I'm a senior analyst at Red Monk. And with me today is Angie Byron. She is a senior manager of community and advocacy at Temporal, and she's also the lead of applied AI at Drupal. Angie, thank you so much for joining me on the MonkCast. speaker-1: Thank you so much. I'm so happy to be here. speaker-0: Well, let's just begin by introducing what you do at Temporal and what you're working on these days at ⁓ Drupal. speaker-1: Yeah, sure. So ⁓ so by day, ⁓ I lead a developer advocacy and community team at ⁓ temporal. And what that means is I have a bunch of very smart, very awesome technologist folks who travel around to events, who do YouTube videos, ⁓ workshops, all manner of kind of developer education around temporal. ⁓ temporal is basically An execution layer around applications to make sure that they don't fail and they don't time out. And a great use case for temporal is for example, if you ever ⁓ ordered something on your phone, like a pizza, it's actually a miracle that the pizza actually shows up at your house because there's like 1500 things that could go wrong along the way. Your credit card might not work, the thing that checks your credit card might be down, all kinds of stuff. ⁓ and so temporal essentially wraps all that. logic in a way that it can be retried seamlessly and ⁓ you know it c it can pick off where it left off if it gets interrupted and all that kind of stuff. So so my team has been wrestling with, you know, AI as a topic for, you know, a while now. ⁓ you know, we it was like February of 2025 was the first time when we started to get folks in our community who were like, Yeah, I came to you because the AI told me to, like that kind of stuff. And we were like, what? You know, because most of our users at that time were like, You know, people who came out of the womb clutching a Unix Bible, like these kind of people, or like distributed systems, geniuses, like that kind of thing. And so it was very like, whoa. And so we kind of had to get up to speed on that in a hurry. ⁓ and then I was a core committer for Drupal for a very, very long time. I went to DrupalCon back in March, kind of more as a social thing. It was like ⁓ you know, DrupalCon Chicago was happening within a six-hour driving distance of where my mom lives. So my mom and I were gonna hang out all week. I was gonna show her, like, here's all my friends. that kind of thing. ⁓ and instead, I kind of like, you know, was was sitting in the audience listening to like say the the Drees note, they it the keynote for the conference, where he's talking about how AI is impacting their open source project, right? And this is like a 25-year-old open source project filled with people who have it like their core identity. I am a Drupal developer, right? Or I am ⁓ you know, like and I've been doing this for for 15, 20 years, like that kind of thing. And they're Freaking out about AI and what it means for them and everything else. And and Drupal is just an example of like all these open like older open source problems or projects have the same kind of issues. And you have this spectrum of people who are like adamantly opposed to AI on moral and ethical grounds. And you have people over here who are like, yay, let's do all the things. And everyone in the middle is just kind of like scared and doesn't really know what it means. So coming out of that conference, I decided to start two projects. One was ⁓ what's called Drupal AI Learners Club. And that is basically just a safe space to admit no one knows anything, right? Like it's like, you know, this stuff changes way too fast, and there are no stupid questions, and we're just gonna talk, right? Because I think there's there's this, you know, there's this worry about being left behind. There's this worry that this stuff is moving way too fast, and I'm so focused on shipping features, I don't have time to keep up and stuff. And So it's just like every week we just talk and like there's people that come on, they show and tell what AI can actually do as opposed to what it's hyped up to be able to do and ⁓ and we live vibe together and like do little silly projects and stuff and and that kind of thing. And then on the more serious side of things, it's you know, we have a huge as as all major open source projects have ⁓ at that scale, this huge influx of, you know, like AI generated the slop. Both on like the the issue discussion and comment side as well as the contribution, like code contribution side. And so AI can now churn out these things faster than any human could possibly ever review them. And it's not like maintainers were like well known to be this bastion of not burned out people before this happened. And so, right, so that So I as a former maintainer myself, this is something that's near and dear to my heart. And so I started another complimentary project called the AI Best Practices Project. Because I'm not very good at naming things and just name things what they are. But that is essentially a skills and eval framework around LLMs to help them write less stupid code because Drupal does a bunch of weird things and it's written in PHP, which is already weird because most LLMs are trained on like Python and TypeScript. Right. It's basically an attempt to gather up. ⁓ information that only lives in people's heads in a lot of cases and provide guidance in the form of skills files. And then that helps both the agents write less stupid code and also helps the humans understand how some of these things work. Like what are the best practices around writing automated tests or what are the best practices around how to contribute to Drupal, that kind of thing. So so that's sort of kind of the two worlds that I live in. So by day You know, my my team's running around, you know, hosting AI meetups, being at AI meetups, interfacing with the AI community quite deeply on like what's happening in 2026 in the San Francisco world, right? Which is sci-fi compared to like, say, the average town in Europe, you know, which is where a lot of Drupal people are based. And then by night, I'm trying to more or less prototype how how this AI stuff can get introduced into an open source community in a way that works. with that community in a way that is not threatening to people who have built a lifetime on developing expertise and in a way that ideally we prototype it in Drupal, but this can then expand out and become sort of like a playbook that any of these like kind of more senior open source projects can apply, you know, to help their their community level up on these tools. Because the tools, unfortunately or fortunately, whatever your perspective is, are they're not going away. And ⁓ and I think there's a big like digital divide right now between the people who are like on the bleeding edge and the people who are still in like I tried chat GPT once in 2023 and it sucked and I'm not going back. And and that that divide needs to close quickly. And so trying to come up with scalable, repeatable ways of closing that digital divide across open source projects because Drupal is just like one example of a project that has this this issue. speaker-0: Fantastic. Okay. So much to unpack there. Sorry, I'm gonna wrap. Yeah. All good. because I ⁓ you know, I think you're setting the scene here for folk. I mean you're you're letting everybody know where this is going and why you are such a domain expert when it comes to to this subject. ⁓ and of course a red monk, we've been grappling with this as well, trying to keep up on how maintainers are dealing with the AI slop situation, trying to think through what is a speaker-1: And then there we go. speaker-0: What is the future gonna look like in terms of this AI that indeed is not going away? ⁓ and that, you know, can be gamed in all these different ways. And it's it's really forcing folks to to redefine how they're looking at trust and how they're looking at a responsible ⁓ stewardship model. for these projects that have been around for many years and ⁓ are are hopefully going to be ⁓ can continuing to serve the community for years in the future. So I also want to point out that I think what's exciting about having you here on the on the show is that you can speak to both of these levels, both the the case study of like what how Drupal is grappling with this, because what I have noticed in my research of these different communities is that all of them are different and unique and that there's no one size fits all. ⁓ but also that you can speak more broadly for what you've been seeing, ⁓ you know, not only in your your sort of your day job talking to a lot of different companies, but also, yeah, like ⁓ just the open source world in general, what you what you hear at conferences, ⁓ what you're discussing ⁓ with other folks in this community. So just wanna begin by saying how jazz I am and that yeah, ⁓ the info dump was useful. speaker-1: Yeah, the the fun thing about working at a DevTools company is that like all of the SDK languages are our target audience. All of the, you know, like we have to talk to AI engineers and platform engineers and so like we have quite a broad based view, which is why it's so interesting to then go into one particular community where they're very like focused on their own stuff. Right. And then you see this stuff happening everywhere, but they don't necessarily. And so that's the yeah, it's it's it's interesting to see it play out more at that macro level as well as like the micro level of a specific open source project 'cause to your point, they they all have their own community norms, they all have their own ⁓ trust, you know, systems and that kind of thing. And so it's very interesting to see how these problems, you know, some are unique to certain communities, but there's a lot that apply across as well. So speaker-0: Okay. So I think a good place for us to dive in maybe with is with this idea of trust. So Drupal has this strong culture of earned credit and contribution status. I wanna hear more about how AI is complicating that situation. So for folks who aren't familiar, what does that credit system look like? speaker-1: Yeah. So ⁓ the credit system is actually really cool. Basically, on every issue ⁓ that we do, there is the notion of a contribution record. And that is where you can indicate in your profile, like this contribution was me as myself as a volunteer. This one contribution was me on behalf of a customer or client, or this ⁓ contribution was funded by my employer, and you can give them like attribution on the thing. And the and the person who decides the attribution is the maintainer of the project because you do see people try and game this, right? They'll fire hose a bunch of docs fixes and it's just like, no, no one asked for that. No, thank you. And that was happening before AI. Now it's just much easier. So ⁓ so yeah, that trust model is very interesting too because they work it into to everything. So for example If you go to like the Drupal marketplace and look for, you know, different ⁓ companies to help you host a Drupal website or develop, you know, a module or whatever, those companies are ranked by their contribution credits. And the credits are scored through an algorithm that they don't make public so that they can make tweaks to it, you know. So like the AI initiative in Drupal, for example, is very strategic right now. So they have like a multiplier for people that are working on AI related things ⁓ versus docs fixes and stuff like that. So it make they try to make it harder to game. It is something we've tried to propose upstream into GitLab, which is what we're migrating to as a as a platform, because it is a really cool system and it allows you to bake in incentivization for like ecosystem support from a from a corporate level, right? Because it's like that if they contribute to open source, it makes their business more visible. It leads to more money for them, and it's like this this nice virtuous circle. ⁓ so that's how Drupal handles it and a lot of other. ⁓ projects though it and and really every project it's a lot more like human to human relationships and like I see this person's name attached to this you know pull request I know that stinky foot 23 and I go way back right and I know that I can trust them and they they know their stuff versus this one I always have to apply a little bit more rigor. You know, this person I've never seen before in my life so I'm definitely gonna like read that more. The big challenge is now you don't even know if the Person you're talking to online in open source ⁓ ecosystem is a person, or if it's an agent acting on behalf of a person, right? ⁓ you don't know if that person wrote that comment or if Claude drafted and posted it for them, right? And this is where things are starting to get all jumbled up because it used to be that even if a maintainer was kind of annoying, you know, I was like, you could take them aside and be like, hey. That's kind of not how we do things here, right? Like can you maybe like dot dah dah dah da and kind of set them on a good path? But ⁓ when an agent's going awry and it's doing it at like, you know, agenc scale, right? You might get 47 of these pull requests that are nonsense and now you're just left with a big mess to clean up yourself. And so a lot of people are like, Well, use AI to solve this problem, but it's like, yeah, but like imagine that you are a left leaning. Open source values believing, you know, 30-something software developer, years of experience person who doesn't trust AI, and you're told that the solution to your problems is use more AI about it. It's just like that's that is not really resonating, shall we say, among this audience? It's like not. So yeah. ⁓ so what is starting to happen though is is starting to like, so automation does help, right? As much automation as you can do in front of these things to make sure that by the time it gets to a human, it's actually worth the human's time. ⁓ there's also open source projects that are grappling with this in a couple of different ways. So I don't want to misspeak, so I hope I'm not misspeaking, but I want to say the Linux kernel wants you to disclose this is the LLM and specifically the model of the LLM that I'm using to contribute this thing, right? So people aren't hiding their usage so that they are transparent about what's going on. Whereas like in the Kubernetes community, they also want you to be transparent, but like the tools like clawed code that like add their little decorator into commit messages, like, get rid of that stuff. No. Because we don't accept a commit unless it's from a person and you are the one signing that commit. And I it it's like you're responsible for the code, right? ⁓ in Drupal, the thing they're they're they're pushing on is this mantra of like. Don't submit code you don't understand, basically. ⁓ and it's interesting because it I think that makes sense for like where that project is at and like, you know, the scale that it's at, right? For them to have a rule like that. But the fact is the you know, the hot kids in San Francisco, they're not they haven't written a line of code in eight months, right? And ⁓ don't review any of it. And, you know, what they do is they set up a you know, kind of a like they'll have a like a judge, you know, agent that reviews the code before it goes up. Like there's different approaches. ⁓ but when you have kind of this this influx of new people experimenting with new AI tools, you have an influx of projects that have a lot of like tribal knowledge that is only kept in like very sequences' heads therefore the LLMs literally can't be trained on it even if they wanted to, it's ⁓ it's creating a really dicey situation. A couple of projects have have taken different approaches on this. So there's some projects that just ban it outright. Like I want to say Gen 2 is one of those where it's just like, nope, we don't allow it. No LLM contributions, buzz off with that. All humans all the time. But what that tends to just do is people just hide it. They still use the tools they want to use. They just don't disclose it. And now you're still dealing with the same problem, but you're not aware of it, which is why I think the The be transparent but a human signs off on it is a better approach. ⁓ because then at least we know that these AI tools are be doing are are being used. We can also like look at that data over time and we can see, okay, well, you know, clawed, it was all clawed all the time. Now we see codecs is starting to come up that right, and it can help us to write better onboarding materials and that kind of thing. ⁓ and then, you know, on the other side you have ghost. I don't think that's how you pronounce it. What they do is they just auto-close anything by a drive-by contributor, just auto-close it immediately. But like a hand selected pick of issues that are like, here, ⁓ if you want to help with these specific 10 things, contribution's welcome. Right. Because they've pre fixed those as like these are things that are not time sensitive. You know, we don't need deep subject matter expertise. If they don't go anywhere, it's not gonna waste a bunch of people's time. Like Kind like stuff that would make great Google Summer of Code projects, I imagine, right? Like those would be great. Like here, new contributor issues, you go over here. So every open source project right now is trying different things to see what's gonna work. And we might see some like patterns emerge, but we might not, right? Like it it because like to your point, it's gotta work for the community and the trust model that they have. But those are some different approaches that are being tried out there right now. And ⁓ and and Drupal's approach is sort of trying to meet people in the middle where it's like we want you to disclose, we expect you to not waste people's time. And then I'm working with a bunch of other people on some automation-y stuff to at least make the code that these LLMs generate suck less. So that ⁓ once they do hit a human, it's like at least, you know, not not a bunch of like nonsense that that waste their time. speaker-0: Okay. Yeah, that is super helpful. And ⁓ I've been collecting AI maintainer policies actually. So I am I am familiar with the the sort of bent of these. ⁓ but it's good to hear the rationale from someone who's kind of in the trenches. ⁓ what I'm curious about is Angie, when we've talked about this ⁓ historically, you've talked about the difficulty of like onboarding and getting like junior often developers. ⁓ You know, to up to speed, I guess. So so these ideas of like understanding and onboarding in a way that's going to help the commute community. ⁓ is that part of these initiatives that you've been working on? And I I guess relatedly, have you seen the demographic of contributors change in the past what, six months, a year, as these agents have gotten good enough to actually, you know, contribute meaningful ⁓ code potentially slop, but you know, ⁓ actually contribute? speaker-1: Yeah. ⁓ so the demographic changes are really interesting. So I think I manage at the top of this call or this podcast that I am a manager now, right? So the last was able to sling code around in VI was quite a long time ago, is how we say. ⁓ but these tools actually make it so that I can code again. It's really awesome. So what we've seen is I've seen code contributions from, say, ⁓ solution architects, right? I've seen code contributions from or not solution architects are pretty technical, but like Like the like product managers, I've seen them from like marketing folks. I've seen them from you know what I mean? Like and they're doing ⁓ real engineering work, right? They can't review necessarily the internals of that work, but you also don't need that for everything, right? Like for something like Python, the Python project, you want pretty in-depth review. But we all have like that abandoned spiderweb open source project over here that we pick away at as we can. Those things don't need that level of of thing. And like, you know, a a really interesting example for is ⁓ so the AI Learners Club that I run is basically just a bunch of people get together and we just talk. And it was like the first one of those was terrifying. Let's, we're gonna do it live. And I have like one plan, which is I I synced ahead of time with this one guy. And I was like, if no one raises their hand to talk, at least one guy can just talk the whole time and it'll be fine. But three people raise their hands and I was like, okay, we're gonna do this then. And it was neat because one of them was talking about like, ⁓ here's like my setup and I use these skills and I got them from here or whatever. But another one was a front end developer and works on Drupal themes, like the the the web design part. And ⁓ and he was working on a new theme and he's like, in order to like make sure I wasn't introducing problems, I needed a visual regression testing tool. And I'm like, I don't know anything about that, but I bet Claude does. So he like said, Hey, Claude, can you write a visual regression testing tool? And it did. And so he was able to parse out his his contribution in like little enough chunks that it could be reviewed by actual humans and then went on to use that visual regression testing framework to find issues in core, right? Like because it was like, you know, it was a before and after kind of kind of check. And I just think that kind of stuff is awesome. And I think like more open source maintainers freely sharing what they do ⁓ and how they're using AI to help reduce maintainer burnout is is so critical. Another example of this is I had a conversation. Sorry, the the gentleman who did that ⁓ demo was named ⁓ Mike Herschel. And the the gentleman I talked to about ⁓ so I have one of those spiderweb covered open source projects, it's all dependabot spam. That's like all that's in there because it's abandoned. And I, you know, asked the question, I was like, is anyone using AI to manage Dependabot? And of course, most open source managers are like, absolutely not. What what blasphemy is this? My goodness, right? This is the good. And I'm like, yeah, I know, but like. On man, like 37 of these, I'm just gonna YOLO merge them. And at the time, Mini Shy Halud was running around. I'm like, I'm definitely not right. So, ⁓ but yeah, so you know, he shared this skill that was basically like if you had like the open SSF sitting on your shoulder while you're reviewing these, I'm like, this is so cool. So I built a thing called like Dependabot, ⁓ what's it called? Dependency Scout. And it and it does it. It's got like, you know, it's got like salsa assa attestations and then it's got like, you know, like a a supply chain thing. It checks the tarball that you downloaded to against the tarball on the thing and it compares and it just like that's the kind of thing. ⁓ and it does it across like 13 language ecosystems. Like I could never write that. Even when I was like a super, you know, hot, hot crap engineer, I could never write something like that. I could maybe write it for PHP, but that's it. And so the ability for people who have a vision on a problem that they want to solve. Are now equipped to do that much more effectively. But what it means for open source maintainers is you kind of had a picture in your mind of what that person on the other end looked like. And they probably are like you, right? They're engineers, they're developers because the bar was too high for anyone else to contribute, right? Drupal's better than a lot of open source projects where they have explicit carved-out contribution paths for, say, designers, UX folks, accessibility folks, like that kind of thing, but many don't. And so their bar up is up here because you have to know how to code. You have to at least know how to figure out GitHub. You don't anymore. Your agent can do that for you. And so that has completely changed people's perceptions. It's also changed their perceptions geographically where these people are located. ⁓ GitHub did like the Octoverse thing, and what they found is like India is exploding, right? Brazil's exploding like these places. And you know, these folks may not share a language with you, like a human language with you, but they're still contributing to your project. And so it's just become a lot to contend with. So the projects that are faring the best in this whole transition are the people who really, if anything, overinvested in things like their governance, their contributor onboarding, this kind of a thing. Right. So, so projects like, say, Python, Drupal, like those projects are probably going to be fine because they already had, you know, gates ⁓ in front of like how how how much you can contribute and like this kind of a thing. And they had well trod out processes and stuff like that, which AI will stomp all over, but it's very clear when they have, and it's like we can but most open source projects are not like that. Most open source projects were I had an interest to scratch, I threw it on GitHub, have fun with that. And they haven't even begun to think about contributor onboarding and they don't have time to do that, right? And so I think there's an area where and this may be going on, I don't know, but like say the CNCF In the same way that they have like governance templates, ⁓ updating those for like the world of agentic, where like if you're like, I don't have time to think about governance, I just need something, right? Like there's starting to be like patterns about this that sort of emerge. I think that would be very helpful. ⁓ and then the other angle of that is again, the person might not be a person, it might be an agent. So, like, what is your contribution flow look like if there is no UI at all and it's all done through the CLI and it's all Like, you know, JSON responses back and forth. Like, what does that flow look like? Right. And this is where a lot of people are, you know, SaaS projects have to deal with this too. Like, what does your cloud sign up flow look like if it's an agent doing it and not a person? Right. ⁓ so it's just like, yeah, th th I don't know if I'm answering your question. I'm mostly just rambling, but it's just like this is these are all the different changes that these poor folks are dealing with. And most of them are volunteers. Most of them are doing this off their nights and weekends budget, right? And it's it's a lot, you know. So speaker-0: Yeah. I am curious about one demographic in particular that I don't think you mentioned, which is academics. I mean, academics love Drupal. Have you noticed a change in that? I mean, I used to work at something called digital humanities. I know many academics who would have loved to have spun up ⁓ a CMS and and actually been able to to manipulate it without the help of a ⁓ an engineer. ⁓ are are you seeing that at all? speaker-1: I have not, but I have to warn you that with my nights and weekends projects. So that could very well be going on and just not on my radar. But yeah. But yeah, it's it's it's like it's up level the kind of folks who can, you know, or I'm I don't know what I'm trying to say. It's it's lowered the floor, is what I'm trying to say of of of contribution. And that's amazing, right? ⁓ it's it's amazing. It's sort of like what blogs did, right? Before that, you had to work for a newspaper, but now I can just put whatever random stuff online and there's good and bad with that, right? You know, but but nevertheless, ⁓ and so I get excited about this. Like I haven't been this excited about technology since like the early days of the web when that kind of democratization was going on. But I do recognize that the people with concerns around AI are are very valid concerns, right? It is causing all kinds of thrash in the job market and nobody knows how that's gonna land. It's stealing from creators. Like, you know, ⁓ I wrote a book at one point. I'm in some case lawsuit against anthrop you know what I mean? It's just like so like this stuff is happening. but yet this stuff has just kind of crossed a threshold of it's like too dang useful to ignore anymore. And yeah. And so I think, you know, working with that reality, ⁓ and then trying to protect our poor maintainers who are doing so much work. I think that's that's a big area of focus that I think the open source world needs to contend with ⁓ in in the very short and medium and long term. speaker-0: Mm-hmm. Okay. And you have alluded to the gaming of the trust system. And of course, some of the benefits that folks would get from doing so would be like, now we've got, you know, a a line on my C V for having contributed or whatever. I you know, I want to dig into that more broadly, but I am curious, does does Drupal have one of those you you mentioned the summer of code. Do you pay for folks to contribute to Drupal? speaker-1: No. No. So Google. ⁓ I don't know if we've done ever done outreachy, but like that kind of a thing. ⁓ Right. Yeah. So more aligned with that. And then, you know, kinda everybody it's interesting, everyone dealt with this issue a few years ago with the first time DigitalOcean ran Hacktoberfest. Because it was basically like win a t shirt, make an open source contribution, and then poof, it was like the entire GitHub. caught on fire and ⁓ anyway, that was a learning experience for everyone. But ⁓ so we've kind of dealt with this before, ⁓ you know, in that respect, but not at this scale. And again, Hacktoberfest was a lot of one line changes, right, here and there, just to kind of get the credit. Now people could spit up entire like entire functional things that probably duplicate other things, right? So it's just But the code at first glance looks correct. And that's what makes it really tricky, is it looks correct, but unless you have ex like expertise and domain knowledge, right, to know this thing just rewrote this entire subsystem that already exists and it should have called two functions instead, right? Like that thing and and again it's pushing that down on humans because while these maintainers could use AI to take a first pass anyway at reviewing these, the a lot of them won't on moral ethical grounds and it creates a really tricky situation. So yeah. ⁓ speaker-0: ⁓ I'm glad you brought up Outreachy, 'cause that is one of the programs that I I had spoken to a maintainer from another project who had said that yeah, they were pretty sure that ⁓ the folks that were gonna get paid via Outreachy to contribute were submitting slop. That you that they they ⁓ they didn't know how to program. But like It's such a burden on the folks at Outreachy to try to figure out whether or not they are ⁓ a legitimate contributor, like if they're actually programming or, you know, somehow gaming that ⁓ so so try I I I feel like there's a lot of these incentivization programs. I mean, bug bounties not being the least among them. You know, there's all these ways that folks are trying to get paid through Slop. ⁓ and I I I feel like trying to turn that fire hose off is still something that ⁓ folks are working on. 'Cause you want the good, right? We wanna discover these C V E's, we wanna help with these projects, we wanna create new features, right? But when you do have perverse incentives in the mix, ⁓ trying to pare those down could be a problem. speaker-1: Yeah. So there's a notable ⁓ his name is Daniel Stenberg. He's the maintainer of the Curl project. And like curl is used by everyone for everything. It's like basically go fetch a URL and do things to it, kind of th ⁓ kind of program. ⁓ so it's at the base of a whole bunch of stuff, like the entire web services field. ⁓ so they it's interesting to watch his LinkedIn because basically I wanna say I'm gonna get the dates wrong, but basically, say in February, Daniel was like, We are Cutting the bug bounty system from curl. It's ended as of today. There's too much slop. We can't keep up with it. It's low quality contributions. No, thank you. Then mythos happened. Then it was the opposite problem. It was like, ⁓ no, these are like super high quality secure. ⁓ wow, they found like 20-year-old bugs. However, we're still strapped for being able to actually do it because like it was like, and it again, this stuff changes so fast. It was like within like, say, two months now. ⁓ it had gone from this slop kind of s like generation stuff just to like win a thing to like, ⁓ actually, good point. That's actually ⁓ it's obscure, but that is definitely a vulnerability that people could chain to other vulnerabilities and get, you know, get access to stuff. And now in July or June, wherever we are, what month we are, ⁓ you know, he's like, Okay, it's it we're taking the summer off. Like, no more but it's just like, you what mean? So like people are again, they're just experimenting with different ways to deal with this volume. And and yeah, but but it is very interesting because, you know, that that again is arguably a an argument for like AI for good, right? Is like if the AI is is sussing out the security vulnerabilities in your project before a bad actor does, though bear in mind the bad actors have access to AI too. So that's a fun game. ⁓ but yeah, that that that becomes an area where it can actually be really helpful. ⁓ to maintainers and and there's a couple of notable projects that have posted about this. I want to say Mozilla is one who posted that like ⁓ they basically in the month of April or some month like that found more vulnerabilities in that one month than they had the entire like 10 years previous to the project. It's just like this is the scale at which AI can help if it's pointed in the right direction. ⁓ and so a lot of it is is you know it it it's it's solving that trust issue once again and and It's difficult, right? Because again, when your community is small or you have like a a a a team of people that are like the onboarding happy people that welcome people onto the project. Like I'm contributing to GitLab now because ⁓ Drupal uses it and like they have like this neat little automated onboarding process you get assigned to like a customer service representative, like, you know, help you get onboarded. It's cool, right? It's like but like most open source projects don't have bandwidth for that, right? So Yeah. ⁓ And if you can't do contributor onboarding at scale though, right, you're you have a tough time, right? And and especially because, you know, and this is just a perennial volunteer management problem, you don't know if the time invested in onboarding that volunteer is gonna be worth dividends because they're gonna stick with your project for ten years and lead to major subsystem overhauls and all this kind of stuff, or if you invested that two hours of time and it's lost because they moved on to the next thing, right? ⁓ I think gating that a little bit too on like watching, you know, what happens and and you know, again, surfacing signals for maintainers so that they can try before they allocate their precious brain space on anything, just to try to understand, okay, this already got coding standards checks done, right? This already got ⁓ we've already checked that the issue summary has all the required fields. We've already done like this this kind of thing. The more we can automate there so that once it gets to the point where it's at a maintainer, it's actually worthy of their time. I think. That that's a major area that I I think ⁓ well we're all need do things. And I think like say GitHub starting to tackle some of this will be helpful, right? Because then they change it once, they change it for all of the projects. But that that's also gnarly because everybody's got their own workflows and they want to customize them and stuff. So ⁓ but yeah, there's smart people thinking about all of this. speaker-0: Okay. So you mentioned Mythos. Has have many of the contributions to Drupal been security related? speaker-1: I'm not on the security team anymore, so I don't know. I do know that this is public. We do have an ⁓ a grant from Alpha Omega project, which I imagine is doing something around security, like that would make sense. and ⁓ but I I'm not privy to that information to know. But I know like there's a broad based effort, you know, ⁓ especially like earlier this year, is it's like, ⁓ MG, once bad guys get access to these tools, right? open source is gonna explode. So let's try to be proactive and get in front of it. And I know that the folks at the Linux Foundation and elsewhere, they were working with a bunch of sort of like strategic open source projects to try to like get in front of it. And I think that's where some of the curl work came from, the Mozilla work and the the Drupal work. But I can't really comment on that 'cause I'm not I'm no longer on that team. speaker-0: Fair enough. Fair enough. ⁓ okay. Well, ⁓ from a a higher level, ⁓ you know, as a as a connoisseur of fine content management systems, ⁓ I'm interested. speaker-1: A connoisseur. I'm sorry, what was the question? I laughed too much. speaker-0: I didn't ask it yet. This is good. I'm letting you laugh at my hilarious jokes. ⁓ yes, that's why I invited you here to to, you know, to make make me feel good about my my ⁓ my bad my bad humor. so the question is, ⁓ do you feel like there are unique sort of I don't know, benefits and and challenges to this agentic era we're we're living in when it comes to ⁓ CMSs? Well, I I know WordPress is often in the news, so sometimes for things very very unrelated. ⁓ the drama's often not not related to the the tech itself, but regardless, I I I'm just interested in if there's some unique challenges that you've experienced ⁓ in in this particular domain, or is it just software's software here? speaker-1: No, ⁓ it's a great question. So ⁓ so Drupal as a representative example of of this is is contending with this. So I will say like at the broadest spectrum of things, what what I predict you're going to see is a ⁓ in the same way that like Multi-cloud is a good idea in case one of your cloud vendors goes down. You're going to see a move towards multi-frontier modelslash open model like stuff, especially when like Fable gets yoinked out from under you with no notice because the US government's in a fight with Dario. Like this is insane, right? You can't plan an infrastructure around that. So ⁓ and the open models, they lag the frontier models, but not by much. Like there is some benchmark about, ⁓ geez. Is it GLM 5.2 or something is like the highest performing open model? You know what I mean? It's just like so these things catch up quick, is is what it seems to be. And they close the gap ⁓ tightly. So what's probably going to happen is there's going to be, you know, sort of a best model for the job. There's going to be like failover modes for like different models. So like if Claude is out of tokens, let's switch to codex. If if both of those are out of tokens, let's switch to something else. ⁓ you know, that kind of thing. And and what that means is that. There's renewed attention, particularly in the EU, around like data sovereignty, because the tools are all trained on data from the internet, usually unethically, and certainly not legally, but they sort of like, well, whatever. It's too useful. We're gonna just do our thing and let the lawsuits come, right? So and so open source, I believe, is going to make a resurgence, or that you we'll we'll look back on this podcast and be like, Angie, you clown, you like that's of but I think. People have kind of learned a couple of ways the past couple of months not to put all of their eggs in either in in any one AI provider basket, right? ⁓ and so for a project like Drupal, it gets very interesting because Drupal is ⁓ like a you know it's a content management framework. So like you could use it as a dev harness if you wanted to. ⁓ there probably people are doing that, right? You could use it to power your ⁓ like rag on top of whatever llm you know to to train ⁓ an llm for your organization that knows everything about your business you can use ai tools outside of Drupal to like I don't want to look up how to make a landing page just make it for me right and it does all the things right so ⁓ so AI is changing CMSs in a couple of significant ways one is ⁓ the question becomes do you need a CMS for that right? There's a whole need a CMS for maybe anymore. Sorry, Drupal. But like, you what I mean? If you can give lovable a prompt to make you a quick little splash page for your little marketing campaign and it goes out there and you take it down when the campaign is over, you maybe don't need Drupal for that. What you need Drupal for is something where it is, for example, ⁓ it's like for across your organization, you want standards and this kind and and best practices. So like a lot of higher ed institutes, like you were saying academics, use Drupal for that reason, because you have the computer science department and the art department and the da-da-da-da-da. And they all want to look very different, but they all want to share the same functionality. They all want the calendar of events and they want the staff director or whatever they have. So Drupal is great in like those use cases. Drupal's also great in a case where it's like I have ⁓ really ⁓ robust like permissions systems. But the AI initiative in Drupal is structured around like inside out, which means we're building the AI tools into Drupal. Like something called context control center, which is effectively context engineering for your Drupal site, and you load up your brand guidelines and all these other things that now it knows to apply it whenever AI is used to to develop content. And then there's the outside in, which is I'm an AI engineer and I use Claude Claude code for everything else, and now I'm coming to Drupal. What's that about? Right. And and and trying to tackle both of those audiences at the same time. So it's a very exciting and scary time to be in the website building business because It's it's like blown up the agency model as well, right? You can't how do you charge an hourly rate anymore? The AIs can generate code at a fraction of a second, right? And it's your trust, your expertise, your taste. Like that's what they're paying for. But how do you bill for that? Right. So that's been a huge challenge that that a lot of the agencies in the Drupal community have been contending with. And I assume in WordPress as well, I just don't know that community quite quite as much. ⁓ but yeah, so there is still places for CMS, but I think that the CMS market is moving up. So in a way, you know, Drupal got criticized a lot for for kind of catering to like enterprise use cases and needs and and these massively, you know, huge organizations that need, you know, all this different kind of stuff. And that's like a good investment. Another good investment is having like structured fields and content, which Drupal has and you know what I mean, like this kind of stuff. It's like these investments made. eons ago, the separation between data and presentation layer, that kind of stuff. All of those are paying dividends now because ⁓ you know, you you can have a clean data model and repurpose it in a bunch of different ways, including outputting as JSON for your agent person, you know. ⁓ and you can, you know, build out an admin dashboard to like look at all the data that your company's servicing, upvote, downvote, conversions, like all kinds of stuff. And so people are doing really innovative things. ⁓ by both working AI tools into like the content author workflow, like AI translations, for example, and then also trying to make it so it's accessible to people who don't have a CMS background, but you know, got thrown into something where they now need to come up to speed on Drupal really fast and like but they already have a a developer workflow that works for them. How do we how do we combine those two worlds? So speaker-0: Okay. And so if I were to go into Claude Code and say, Build me, ⁓ you know, I I I want to create a thing, some sort of archive or whatever, ⁓ have you set up any SEO so that it's going to recommend Drupal above, I don't know, Strappy or something? speaker-1: Yeah, someone definitely needs to do that. I have not the AEO of of Drupal yet, but yeah, it's a it's a challenge, right? Because it's like if d so I have an issue in the issue queue for that, but haven't gotten to it yet. But yeah, it's it's important because like that consolidation is happening, right? If you say, I want to build a website, click, right? It's gonna recommend ⁓ it's gonna recommend like Next.js, it's gonna recommend SHA CDN, like whatever. It's got its tool set already figured out and it knows that from like reading the entirety of the internet. So if you want to change what LLMs say about your thing, you have to actually do like a concerted effort around that. And ⁓ and that is still pretty premature in the Drupal community, from what I understand. But but it is something that a lot of companies are ⁓ companies, I should say, like it's an open source community. So that's a little different, but they, you know, it but it's a it's a challenge, right? Because without that opinionated like, this is the guidance on when this is the right tool to use it, and the agents actually being able to listen to that guidance because it's in a format that they can read. ⁓ that's like one of those one of those like areas that that all open source projects that want to be used in two years should be focusing on now. ⁓ so that they don't get replaced by whatever hot, cool new kid stuff came out in like twenty twenty three and twenty twenty four. Cause that's ⁓ yeah, there's a lot of that. It temporal started that way too, where originally you had to like really cajole it and be like, no, no. Right, because it would recommend all these other things 'cause they were more common, like out there on the internet. And that's that started to change as well. We have smart people working on AEO. We've also like done a lot of work to our docs to make them agent friendly and stuff. And I think all those those efforts are helping. speaker-0: Okay. Amazing. ⁓ yeah, so you you've got a roadmap around there. I mean, ideally, even a vibe coder would look into what CMS they want ahead of time and be like, build me a Drupal project, right? As opposed to just like build me a thing. Here's what it looks like. ⁓ but but I know that that's possibly optimistic. Yeah. speaker-1: ⁓ yeah it well again, because you know, yeah, I I heard about temporal because the AI told me to, like that's that's an audience. I heard about Drupal because the AI told me to is is definitely gonna be an audience. But a lot of times people just sit down to solve a problem. And vibe in particular are very technology agnostic. They don't care, right? They just want the website to do the thing that they want it to do, right? They they could care less about like long term maintenance and things like that. That's someone else's problem. Then they'll huck it over to an engineer who's like, What have you done to me? Like, right? Like that's That's what's gonna happen in a vibe coding situation. ⁓ the horror yep. speaker-0: Okay. Well, as we wind down this conversation, ⁓ are there any CTAs you'd like to share with the community? I'm interested in hearing w what is what's the sort of advice you're giving to developers right now, maintainers, ⁓ folks who are interested in in not only the Drupal community, but you know, where software's going right now. speaker-1: God, that's a great question. It's hard. It's so fragmented. I'm not aware of a nice single place to point you to where this is all being discussed and it's actually a huge problem. So there is, for example, the Agentic AI Foundation, which is starting to collect together a bunch of open source projects to make a truly like open stack around ⁓ agentic AI, but I'm not aware of them focusing on maintainer sustainability issues. I don't know. That's problem, right? I should know and I should be able to rattle off, like, yep, this is the organization that's focused on this instead of being like, well, I know a couple specific individuals in different projects doing this in different ways. ⁓ so maybe that will be the next thing we talk about because I'm gonna go start that this weekend. I'm saying, ⁓ because yeah, that's it's a huge problem. I'm not aware of one. ⁓ I I really am not. The closest thing is ⁓ GitHub has like a maintainers community that maintainers can ⁓ apply to join. But even there, like again, it's a bunch of hardcore open source folks. And a lot of them are just like, no, I couldn't possibly use AI for that. What are you talking about? And they just view AI as a problem because for them it's a problem. And ⁓ as far as like a place up a level where we could all talk as open source maintainers across projects, ⁓ I don't know of a place like that that exists. But if your listeners do, I would love to hang out in that place and I would love to help with that place. And then yeah, once again, maybe this will be something that I start in my other nights and weekends. I got some PTO coming up. Maybe I'll ⁓ start a foundation in that amount of time. No. ⁓ but it's a colour, right? It's a great colour because it's like even in Drupal, the reason I started AI best practices is because if you go to skills.sh and you search Drupal, there's like 96 things in there. And it's like that's cannot happen. That's not an onboarding experience, right? You know what I mean? We need like one opinionated set of things. That is fine for 80%. And then if you want to go off the rails and do your custom handcrafted organic, you know, grass-fed evals framework on top of it, fine. Right. But like this is gonna work for 80% of people and get them up and running, right? So ⁓ but yeah, but that problem that you're highlighting, that is what I'm interested in tackling. So hey, if you're listening to this and you want people to to partner with on solving this problem at a broader open source ⁓ perspective, look me up and I would love to talk to you because it's ⁓ It's critical and there's very few people that can work on it, I think, because you need a lot of depth of cultural knowledge of how open source works. You also need credibility. So you don't necessarily have to have been a maintainer, but you at least need to really empathize with what a maintainer feels and and how this whole onslaught is affecting them, like on both the both the just a practical level, but also like on an I on a core identity level. ⁓ but yes, I don't know how to point you at a great resource. I will say The Drupal AI Learners Club meetings are pretty Drupal specific, but they're fun. So you could always join there and and we'd love to have you. We did a vibe coding session the other day where I was trying to port my very old crusty Drupal 7 blog and we ⁓ made it so that whenever the migration failed, it printed like an Oregon Trail dysentery, like, you know, kind of like thing. You know, it's like, yeah. So we have fun. ⁓ and I think that's what you have to do. You have to make it fun. You have to make it not scary. You have to make it approachable to people, right? Because It's it's it's it's it is scary, right? It's changing the way people work. It's it's making people who loved the artisanal craft of coding really fundamentally go through a grieving process about that, right? And and it's and it's up leveling them to, yeah, you used to write the code. Now I need you to supervise people writing the code. And that's a whole different job, right? Supervising developers is a discipline unto itself. And you know, you don't necessarily thrive in that environment. So But that identity crisis that that that that developers broadly and open source maintainers in particular are are going through right now is is really tough. ⁓ but anyway, yeah. So I don't have any easy answers for you, unfortunately, but maybe by the next time we talk I will. That'll be a goal of mine. So speaker-0: I like it. I like it. ⁓ so for folks who do want to maybe reach out to you, Angie, where are you directing folks? Is there a social media ⁓ platform that you're using these days? You know, how can folks get in touch? speaker-1: Yeah, I would say ⁓ if fortunately or unfortunately, LinkedIn is probably the easiest one 'cause I'm a hiring manager and so I'm sort of contractually obligated to hang out on that one. ⁓ and then ⁓ and then Drupal Slack, but it doesn't make sense for people to join that unless they ⁓ you know want to. Or temporal Slack. You can go you can reach me on many Slacks. But yeah, LinkedIn is probably the best. And then ⁓ webchick dot tech is my it's a blog called Tech Arounded Find Out. So my like nights and weekends, three in the morning projects, I post about those there sometimes and ⁓ so yeah, but I would love to love to meet other people working in this space and and collaborate together to to solve some of these challenges at scale. speaker-0: Fantastic. All right. Well, I have really enjoyed this conversation. ⁓ been super eye-opening and you've confirmed and challenged me, which I I I I love it when that happens. You know, we're we're digging deeper. We're gonna figure this out together, you know? ⁓ okay. ⁓ so again, my name is Kate Holterhoff. My guest today is Angie Byron. ⁓ if you have j enjoyed this conversation, please like, subscribe, and review the monthcast on your podcast platform of choice. If you're watching us on Red Buck's YouTube channel, please like, subscribe, and engage with us in the comments.