Latest / The Ruby AI Podcast / Rails After the Robots: Chad Fowler on AI as the Next Abstraction

Transcript

Rails After the Robots: Chad Fowler on AI as the Next Abstraction · The Ruby AI Podcast

• 0:00
  Hey, everybody. Welcome to another episode of the Ruby AI podcast. I'm your host today, Valentino Stoll, and joined by co-host Joe Leo.
• 0:09
  Hey, everybody.
• 0:10
  And we have an incredibly special guest today, Mr. Chad Fowler. I really appreciate you coming on. Just one of those blind reaching out to you we had never met before. But, you know, I was at RailsConf and I heard you talking about how you feel about all of these AI changes with respect to Rails. And I found it really compelling. And so I would love to hear your opinions on so many things here. I want to hear yours too. Let's argue a little bit too, hopefully. Well, Valentino is actually, he's being nice and I appreciate that he is completing the gambit, which is to get you on here under the auspices of talking about artificial intelligence and Ruby.
• 0:51
  That's not true. I actually, I specifically wanted you to come on the show so that I could talk to you about February, 2013, when you left Ruby world as I knew it and the country to go work on a startup and write it in Cappuccino.
• 1:05
  And I would like you to explain that, please. In Cappuccino? What is Cappuccino?
• 1:12
  Exactly. So I think you're talking about when I went to Wunderlist.
• 1:17
  What is Cappuccino? Is that a thing we use? That's the Objective-C library.
• 1:22
  Okay. Right, right, right. Yeah. You know, it's funny.
• 1:25
  I did leave the Ruby world then.
• 1:27
  So, you know, we were talking about our mutual friend, David Allen Black, Ruby Luminary, right before we started this.
• 1:33
  And I said, I hadn't seen him since RailsConf 2012.
• 1:35
  That's because the last time I went to RailsConf was 2012.
• 1:38
  Right after that, I left the country.
• 1:40
  I went to Germany.
• 1:41
  I worked on this thing called Wunderlist.
• 1:44
  And they hired me because I was a Ruby person. And they've done this big rewrite and done a backend in Ruby. And the first thing I did was start talking about tearing up the Ruby thing and replacing it with other languages, which I think was much to their chagrin. Like, oh, great. We got this crazy guy. We moved him from the US and now he wants to use Scala of all things.
• 2:03
  But it is true. I did. Yeah, I totally did that. I think we'll probably end up getting back to that at some point, so I won't go too much into it. But the reason I did that was actually largely architectural.
• 2:16
  And it's when I was forming some of the biases that lead to how I'm viewing AI today and generative code specifically. Yeah, no, I definitely want to get into it.
• 2:26
  I mean, it was such a conflicting moment for me.
• 2:28
  I know I'm dating myself, but there were all these people who started writing Scala at or around that time because it was like, oh, what Chad is doing and look at this cool thing.
• 2:35
  Because obviously, if you don't know the story of Wunderlist, it was an incredible success sold to Microsoft.
• 2:40
  And you that went and had several positions at Microsoft, obviously, the re-architecture was a good idea.
• 2:45
  But of course, me as a rebudist at that time, I was like, wait, wait, chat's doing something else.
• 2:50
  Why? Why?
• 2:52
  And of course, you know that you get older and you make different decisions.
• 2:54
  I did other languages as well.
• 2:56
  So I'd be interested to hear the kind of architectural underpinnings behind that.
• 3:01
  Yeah, sure.
• 3:02
  You know, it's funny you're talking about that feeling of, oh, no, I'm doing Ruby and I at the time.
• 3:25
  And we all felt like we were at the top of the world because we got lucky and we were in this tech that everyone was using.
• 3:29
  It's kind of like investing in a company that is very early and then getting lucky.
• 3:33
  And it won't be long before we're all old people wishing that we had learned something else.
• 3:37
  So how do we avoid getting stuck in this thing?
• 3:40
  And so that is actually one of the reasons that I stopped doing Ruby when I did, because it felt too good to do Ruby, ironically.
• 3:47
  And I knew I could get stuck there.
• 3:49
  I didn't want to get stuck.
• 3:51
  I can understand that because I could also, I also had that same feeling right around that time, but I was much earlier on in my Ruby writing career.
• 3:58
  But I was like, man, can we actually keep this thing going?
• 4:01
  Your analogy to a startup you invested in, I think is a really good one.
• 4:05
  I just double checked and isrubydead.com says no. I have so many questions for you. I mean, especially around all of this AI stuff is very Python central or what now is TypeScript central in some ways. And there's so much value, I think, from the Ruby language in this particular space, as far as transformers go and natural language processing that I think, to be honest, makes Ruby well-positioned work better in a lot of ways for these transformers specifically.
• 4:40
  I'm curious what your opinions are kind of with language specifics around all this stuff.
• 4:46
  Is that something that you think about even or is it all abstractions at this point?
• 4:50
  I think about that a lot, partially because being a nerd, of course, I want to be one of the people who figures out how to do it.
• 4:56
  The Python thing is really irritating. Irritating as a Rubyist, of course, because there really is no reason to be both a Python programmer and a Ruby programmer.
• 5:06
  And unless it's just like you want to use one of the libraries from one of them or something, you know, they fill the same hole.
• 5:13
  But just the very basic fact that a language that's white space sensitive in an era where everyone's copying and pasting is just really stupid.
• 5:20
  And I hope everyone involved in that decision, which might just be Guido. Maybe he's not even experiencing this now, but in retrospect, it's a mistake. And even at the time, it felt like a mistake. So, ugh, gross Python. Just for that reason. That said, I want to hear your perspective on how Ruby is better for transformers. I will tell you first, since you asked me, the way I think about it is I think probably it would make sense to even have a language that's geared toward generative AI, generative code.
• 5:51
  My first hunch was that something like Haskell would be perfect for this.
• 5:56
  Because with Haskell, I used to joke we had Haskell stuff, if it compiles, it works because the type system is so rich and you can embody so much in the type system that it's beyond just syntax and grammar.
• 6:10
  It's semantics that get baked into these things.
• 6:13
  Ruby doesn't have that and Python doesn't have that.
• 6:16
  TypeScript sort of does. Python as being like the default thing that these LLMs vomit up, then great. Because now, in a way that I used to argue against when dynamic languages were becoming the rage in the early 2000s, the programmers we're talking about now actually are stupid, they're LLMs. And Dave Thomas of Pragmatic Bookshelf and Pragmatic Programmer, he used to say that Java is the Duplo block of programming.
• 6:45
  You're trying to protect the dumb programmers for themselves by putting all these things in place.
• 6:50
  Joe Armstrong actually even told me once about the Erlang system. What is the Erlang system?
• 6:56
  The framework is called OTP. Yeah. So we were talking about OTP and he's like, oh, I don't use that. We built that for bad programmers to use. That's really funny.
• 7:06
  He would literally not use OTP.
• 7:07
  He would always code everything on his own.
• 7:09
  He's one of the inventors of Erlang.
• 7:11
  Just for context, if people don't know, most Ruby people seem to know Erlang.
• 7:16
  So my point is guardrails actually matter more than they used to, or at least more than I used to argue that they used to.
• 7:22
  There were people that never moved on from that.
• 7:24
  But like the programmers are pretty dumb, the LLMs.
• 7:27
  They're good at spitting out lots of stuff.
• 7:30
  So there's language-specific stuff like that.
• 7:32
  I've done some research into formal verification and languages for that for doing specifications, so like TLA+, and there's another one called Quint that's new and open source and looks more like Ruby and JavaScript.
• 7:45
  It's really nice.
• 7:46
  I feel like there's a place for those sorts of tools.
• 7:49
  And then ironically, things like ontologies to describe domains in some sort of codified language, like OWL was the thing from the semantic web, ontological web language.
• 7:59
  I think it's like these sorts of things that come together that can create and document constraints in a computer checkable way that become more valuable, usually than very specific language features.
• 8:13
  And then the problem is, I think it would be great if there were languages built for LLMs to use because I think they'd made things better. However, you also have the whole other side of reality, which is LLMs work by indexing the internet, which means the best thing that you could do is use something that everyone else used two years ago or before, right?
• 8:35
  So the best documented things are the best ones.
• 8:37
  Or here's something in the favor of Rails, not necessarily Ruby.
• 8:41
  Languages that have very strong conventions or frameworks that have very strong conventions.
• 8:46
  That's a great thing about Rails.
• 8:48
  Rails is an architecture more than a framework.
• 8:51
  It's an architecture embodied in Ruby.
• 8:53
  And then people have completely embodied that in other languages.
• 8:57
  And so the things that reduce decisions that a human would need to make that are not of value are also good for LLMs because you also want to reduce places for them to make mistakes, places for the context window to grow so they become more stupid as they go. Anyway, this was like a totally unhinged babbling session.
• 9:16
  Why do you think, Valentino, that Ruby is better for transformers? What's the argument?
• 9:21
  I mean, more specifically, I guess, is the training data.
• 9:25
  Right now, things are mostly like, you know, like you said, scraped from the internet, but a lot of it is pure textual. I would say a very small portion of the training data is actually code in a lot of ways, especially for the larger ones, models that everybody uses the code, right?
• 9:43
  Is that in perspective to the amount of data that they're actually scraping and training and preparing, the amount of code is very small.
• 9:50
  And so what you end up with is something that more desirable is aligning your inputs to be more textual and more English reading and sentence structure and things like that matter more than how the code is specifically formatted or parameters and things like that. This is all hearsay. I don't have any empirical data to support any of this that I'm talking about, but this is just from what I've been reading about it, transformers over the years and following the patterns and training data that they at least are admitting to. This is kind of where I'm thinking is the more you can get something to look like actual sentences and structures and reporting and things like that, the better your outputs end up being.
• 10:36
  And I think a lot of this can be proven simply by Markdown. Like Markdown is an incredibly well performing format, even compared to JSON as an example, which is what a lot of people use, but it doesn't perform as good. And I think that kind of leads into a lot of just how transformers work. And when you get a lot of these superfluous tokens of hash signs or begin, end, and maybe some of this stuff that doesn't make sentence a sentence.
• 11:07
  For Ruby's case, like those begin ends maybe make it less desirable.
• 11:09
  But when you get ones where there's like curly brackets and lots of semicolons and even lots of this extra tokens, it doesn't really align as well.
• 11:18
  And then unless the model is like heavily trained in that specific language, you end up getting kind of blended results where it's, okay, it has to assume that you're using this language. is like heavily trained in that specific language, you end up getting kind of blended results where it's okay, it has to assume that you're using this language. And then based on that language, it gets into a different layer and then tries to have to reason internally about how to align those layers and chains through everything. And so my perspective is that Ruby is very easy to read and it is more like language specific from a grammar standpoint.
• 11:48
  And so if you can align your code, which I think Rubyists do kind of best practices of trying to make it more legible, right?
• 11:56
  And even from an abstraction standpoint, very practiced at trying to make your abstractions work well from a readability standpoint.
• 12:04
  And so like the more readable your code is, really people say that's the better Ruby.
• 12:08
  And I tend to agree with that.
• 12:10
  That's interesting.
• 12:11
  And it's also another take on the raising of abstraction where Rubyists will like create a DSL for something that doesn't need a DSL.
• 12:19
  And I don't mean that in a negative way.
• 12:21
  It's just like stylistically what happens because we all fell in love with this idea of being able to raid the level of abstraction to where it's almost like human language to the point where we did a bunch of stupid things too in the past.
• 12:33
  Weird chained sentence structures just because you can do it because the result of every call is some sort of value that you can chain on and overload and blah, blah, blah.
• 12:42
  And that's an interesting one because those are really bad for people.
• 12:45
  Ultimately, they're good for the reader and not good for anyone who needs to write it or modify it.
• 12:51
  But that might not be true for LLMs.
• 12:54
  Yeah, and I mean, this is all just assumptions.
• 12:56
  There's definitely more training data on the models for code for all of the code that these model producers adhere to.
• 13:04
  And that's clear, which in some cases may be why something like Scala or Rust or something like that, maybe Rust is a bad example. I heard it doesn't perform very well. But I feel like there's just a lot of opportunity for Ruby to really dig into the current state, at least. But I definitely want to hear more about your idea of abstractions because I was hoping the WebAssembly application of language, like you're mentioning some kind of specification that converts to something that is LLM performant, where like maybe there's like some kind of WebAssembly version of cross-language compilation into some kind of spec that then the LLMs understand universally.
• 13:45
  That's something I think about too. These are all hypotheticals. Is anybody working on this stuff?
• 13:50
  I haven't seen it. Maybe you have. I don't know. I've talked to a lot of people doing CodeGen-related stuff. Yeah. And my sense is that probably they don't live in LLMs. Ultimately, these models, parts do and parts don't, but they're like MCP calls into things that have like better graph oriented structures behind them or whatever.
• 14:13
  So I don't know.
• 14:14
  I don't know if the LLMs care about this sort of uniform thing either.
• 14:17
  Like one lesson I've learned from talking to founders as an investor now is sometimes I'll ask people, well, what's the shape of this API and how do you get these two things to talk to each other? people, well, what's the shape of this API and how do you get these two things to talk to each other? Like, oh, it's a prompt actually, you know, at this point, we don't even need to think about that anymore. And for small enough problems, that's fine. It actually works today where you can tolerate some error. You were asking about abstractions. I remember when Rails came along. Well, I remember when Ruby came along in fact, and before that, like there was a time in my programming life where I really thought in terms of the program counter, basically. When I learned basic, it's 10 print, hello world, 20 go to 10, literally about the steps that the program is following and store this thing in here and then print it here or assembly was even lower level than that of course every time we've seen a major level of abstraction get raised in programming there's a community of people that are absolutely terrified angry think it's stupid not going to work whether it's compilers or even assemblers or the JV app or Ruby as a high level language with dynamic typing, as opposed to using Java or C at the time where everyone's like, well, it's going to go crazy.
• 15:32
  You're not going to know what any of these values are and it's not going to work and it's going to perform terribly.
• 15:36
  Well, that ended up being false. Rails comes along and says, you know what?
• 15:41
  You don't need to spend several days figuring out the project structure of this web app because none of these are valuable questions to even answer.
• 15:49
  So just get past that by typing one command and we're done with how to set up the test framework, which one to use, what directories things go into, how you do routing, all this stuff is just answered for you. That is the miracle of Rails sitting on top of the miracle of Ruby at the time.
• 16:07
  It's all about convention over configuration and being opinionated.
• 16:10
  Yes, I agree.
• 16:11
  So you look at active record, that looks insane to a person in the year 2004. Someone who spends all day working on database stuff, they look at that and think, that is never going to work.
• 16:25
  It sounds like a fun trick, but it's never going to work.
• 16:28
  It worked.
• 16:29
  It works.
• 16:29
  Of course, there are all sorts of corners and you end up fixing them.
• 16:32
  And for a while, there's a job, and there still is, but for a while, there was really a job for people to figure out how to get something like ActiveRecord to work without destroying your database when you start getting people hitting it because the first thing that you do is N plus one problems. and there were so many things that were just like by default you were going to screw up no longer the case though and no one expects to start a job on a software project now and type sql all day right it just doesn't happen someone has to do it sometimes if you're lucky and i say if you're lucky because it means you have a performance problem because people are using your app right so if you're lucky you get to do it sometimes if you're lucky. And I say, if you're lucky, because it means you have a performance problem because people are using your app, right? So if you're lucky, you get to do that.
• 17:08
  It's not everyone on the team. I'm looking at these LLMs like this, and I know there'll be people who disagree because it's not deterministic and blah, blah, blah. There's all sorts of arguments, but it's just a continuation of the same thing we've been doing the entire time we've been in this very still nascent industry is raising the level of abstraction to a point where we can ignore the things underneath it.
• 17:30
  So I think there's soon a time very, very soon.
• 17:33
  I think we should stop saying the phrase vibe coding unless it's a pejorative, actually, because vibe coding implies people who don't know what they're doing, just like letting the thing go.
• 17:43
  Right. people who don't know what they're doing, just like letting the thing go, right? There's a time pretty soon where people like us that have spent our entire careers understanding exactly what's happening also ignore what the thing develops.
• 17:52
  As long as it's doing the thing we want, it's creating the thing we want in the same way that I have not recently read the source code to any of MRI to see what happens when I call a function with a certain type of argument, because I don't care. Ruby's doing that. That's Ruby's job, not my job.
• 18:09
  Well, so this is an interesting point here because you've hit on something with Rails, the convention over configuration concept or feature. What it does is it eliminates cognitive load, right? I no longer have to think about how I'm going to group my models, my views, controllers.
• 18:27
  I no longer have to think about routing, right?
• 18:29
  There were a lot of things.
• 18:31
  Soon after that, we got things like Heroku.
• 18:34
  And then all of a sudden, we didn't have to spend two weeks figuring out how we were going to deploy this thing that we had built to the web.
• 18:40
  We could do it overnight.
• 18:41
  We could do it in an hour. So if that is an abstraction, both of those are abstractions that reduces cognitive loads that we can work on, presumably higher creative problems.
• 18:54
  Then what is the cognitive load that gets reduced with the introduction of AI as an abstraction?
• 19:01
  I'll tell you a story instead of answer the question real quick.
• 19:04
  Right after Rails was released, there's a guy named Glenn Vanderberg that all the Ruby people used to know. I'm not sure if they still do, because he also left around the time I did. And I think he got closure, but I remember vaguely he did a bunch of great talks at Rails comps and stuff. He's a really good thinker in architecture. He had moved from doing Java stuff. I think he'd written books on Java and enterprise Java before that too, to Rails.
• 19:27
  And he told me, I remember where I was sitting, I was in 2005 and I was helping him with some like Rails windows problem.
• 19:34
  But he said, yeah, I've just started working in Rails and it's been like a month and I find myself to be exhausted at the end of the day.
• 19:42
  And the reason, because I thought, well, that's weird. That's not what I expected. I expected you to say like energized and excited, you know, not dealing with all this crap anymore.
• 19:47
  He's no, I'm exhausted because in the Java world, I would think of something I want to do, and then I'd spend an hour typing it into the thing and having it compile, and then I'd be done.
• 19:58
  And then I'd think of the next thing.
• 20:01
  And the hard part is thinking of what you want to do and how you want to do it.
• 20:04
  Then the easy part was sitting and typing. So I was like, well then I'd think of the next thing. And the hard part is thinking of what you want to do and how you want to do it.
• 20:07
  Then the easy part was sitting and typing.
• 20:09
  So weirdly, it actually increased his cognitive load to raise the level of abstraction.
• 20:15
  And I recognize that to be true for me too.
• 20:19
  Like, oh, I can go so fast with this thing.
• 20:21
  I have this exact feeling today when I'm using cloud code because I structure things in such a way that it finishes the thing I wanted to finish and I need to decide what's next.
• 20:32
  And I'm dealing with complexity much more quickly than I would if I was typing it in because the system is creating is huge. So you asked me the opposite. I'm telling you, it sort of increases the cognitive load, but I think it does it because it removes the boring stuff and the mindless stuff from my process.
• 20:48
  The things that it would decrease though.
• 20:51
  So just to make this believable, imagine you have a system you're building and almost the entire system is made of pure functions.
• 21:00
  So, you know, no side effects, no memory being changed.
• 21:03
  It's just deterministic inputs, outputs.
• 21:06
  So the first one is reverse a string.
• 21:09
  You could write that.
• 21:11
  You could also use a built-in thing for the language, but let's imagine the language doesn't already have it.
• 21:16
  If you had to reverse the string right now, you'd sit down, and if the language doesn't already have it, you'd sit and you'd think, okay, how do I need to do this?
• 21:22
  How does it need to perform well? Blah, blah, blah.
• 21:24
  And then you'd finally type it out, and you'd be, okay, how do I need to do this? How does it need to perform well? Blah, blah, blah. And then you'd finally type it out and you'd be done.
• 21:26
  It's a simple thing.
• 21:28
  But I think we all agree we could ask an LLM to do this and we could probably trust the output.
• 21:34
  It's a pure function.
• 21:35
  Complexity is reduced by the surface area of it.
• 21:38
  So I think with the right architecture, many more problems can be solved in such a way that the solution is trivial.
• 21:48
  And therefore, we can reduce cognitive load by not having to think about it.
• 21:53
  And that leaves us thinking about architecture.
• 21:55
  And, you know, another thing from my past is when I got into test-driven development, I've only been to two software courses in my life.
• 22:03
  One of them was the extreme programming immersion that Object Mentor used to do. And I went to that. I convinced my company to send me and a whole bunch of other people and we were going to do XP. And so I was sitting with Kent Beck, because that's what you get to do, the inventor of extreme programming, pair programming on some made up problem. And I remember telling Kent as we were working on this thing, I see what's happening now when we do TDD the right way, we take a complex problem, we break it into simple pieces, we declare what those pieces are, and we allow the hard part to happen lower in the stack.
• 22:39
  So now we go lower in the stack, we do exactly the same thing, and we TDD all the way down to the bottom where nothing was ever actually hard to do. I think what we need is architectures now that mirror that sort of a thinking for the LLMs. Because the LLMs can't do hard stuff in a way that's reliable.
• 22:56
  They certainly can't maintain the system after they generate it. But what if we had a system design and architecture that was just made up of hierarchies of things that were all trivial and we could all trust the most idiotic programmer we've ever met to build it as long as there's a test for it so this is how i think now and it's how i've always thought i told you there's biases that i bring from back before i went to wonderlist which led me to do this crazy heterogeneous back-end microservice, which I didn't talk much about. But the reason I left Ruby was I wanted to explore these ideas of decoupling and composing systems of trivial pieces. And even back then I started talking about immutable infrastructure. Yeah, I remember. That was a big influence for me. Sorry to cut you off, but I loved that, what you wrote about, And I neared it at the company I was working for.
• 23:46
  It was a big influence on me.
• 23:47
  Yeah, continue.
• 23:48
  I think I coined that term.
• 23:50
  I've been taking credit for it for years.
• 23:52
  I'm sorry.
• 23:53
  Oh, no, it was you that coined the term.
• 23:56
  But after I heard it, I wrote a post about it.
• 23:59
  And what I used to always talk about was immutable infrastructure and disposable code. And it's the latter that's more powerful that didn't catch on as much. So at Wunderlist, what I would tell the team is you can write your code in any language you want on the backend, mostly. Yeah, that's where I was really working, mostly. Any language you want, but it had to follow a certain set of guidelines. And there were the obvious ones, like it works with the monitoring system and the build system, whatever else. But the one that was the most important is I would hold my fingers up like the size of a small page or something and say, as long as the code is this big or smaller, you can write it in any language you want.
• 24:39
  The reason was we had built a system with a consistent set of calling conventions and API conventions. So anything could plug in. It was all via HTTP, of course, and a message bus thing that was sitting on top of RabbitMQ. So we had like two modes of calling things. We had an architecture where it was very obvious where something should sit. And so if your code is no bigger than a page long and it doesn't work or someone doesn't like that language and they're on call, they can replace it.
• 25:09
  So if I'm doing some crazy stuff in Haskell and I could only find one other person at the company who would work on Haskell with me, which is the case.
• 25:17
  Well, that was one of the rules.
• 25:18
  You had to have at least one other person who was excited to work on and maintain the thing with you.
• 25:22
  So if neither of us was around and the Haskell thing went down, we'll just rewrite it in Ruby.
• 25:27
  It's fine.
• 25:28
  You know, or rewrite it and go or whatever.
• 25:29
  And we did that sometimes.
• 25:31
  So like, oh, I've got an upgrade request for this service.
• 25:34
  I don't like Clojure.
• 25:35
  I'm just going to rewrite that function and TypeScript or whatever and do the upgrade for it, the feature enhancement for it at the time.
• 25:42
  That's pretty fascinating.
• 25:44
  The reason I'm talking about this is what if you had a complex software system where all of the pieces of code followed those rules and therefore anytime you needed to modify the code or maintain the code, the system just figures out which piece it needs to change and it replaces it because it's trivial and it has like a clear inputs and outputs and side effects are well known so it doesn't matter if your lom can figure out how to maintain your code as long as it can find the thing and replace it and you don't care you're not like married to the implementation of the thing you just need it to do what it's supposed to do i really love that that response i'm glad you rambled because yeah love the idea of the hierarchy of composable objects. I think that if you can abstract a concept about something, then you don't have to define it everywhere anytime you call the LLM.
• 26:49
  And that seems to be true even for a lot of the stuff coming out in the industry as well, even from Shopify, a lot of Obi's work from like Cloud on Rails or the specification driven development, right?
• 27:01
  That's coming out now.
• 27:02
  It does seem to be like, okay, if you enter this directory even, then this is how you should behave as a coding assister. And that can be even, like you said, drilled down further down the stack to be more specific to, okay, I'm working on database related things in a Rails architecture. Here are things to follow, right? But there is definitely, it's hard to manage that kind of abstraction on top of all the other abstractions. And I think that's where we're kind of missing some tooling and infrastructure, even from the industry, not just Rails and Ruby, right, is we have all of these kind of cobbled together things that work kind of, and they should work very well because of how tightly bound the conventions are already defined, right? So in a Rails stack, if you want to do anything database active record is very well defined and it has very simple instances and interfaces to like do most things with, right? But if you just drop that in an LM, you're going to have to like give it some handholding or hook up an MCP and then it's going to have to call the MCP who knows how many times to figure out just how to implement the thing that's already implemented. And so what I'm looking forward to is more of this context expansion, right? Like how do we take these abstractions that are very big, but when the ask comes, it's very specific.
• 28:29
  How does it like then walk the stack?
• 28:31
  How does it find its way up to build up more context to figure those things out, which is ultimately what we do kind of in reverse in a way.
• 28:41
  Yeah.
• 28:41
  Well, we do it in both directions because we're doing in both directions.
• 28:45
  Yeah. And we have the same problems that LLMs have too. Right. We also create the same problems that LLMs create for that matter. Totally. I feel like Rails, like you mentioned, outside of the conventions, all the generators and things like this are just so helpful in this generative process. I feel like there's something there too, where we could really expand on to drill into these kinds of concepts.
• 29:09
  I'm curious if you've thought about from a Rails perspective, or even Ruby's like modules includes and things like that.
• 29:16
  If you've thought about maybe how this architecture would look.
• 29:20
  A bit, yeah.
• 29:21
  I mean, I think I lean toward, you know, I use the word bias a lot in this conversation already because I have this bias that everything needs to be small and decoupled.
• 29:30
  And this is why I was also one of the crazy microservice people, and I probably still am.
• 29:35
  It's not because I want everything to talk via HTTP and be slower.
• 29:40
  It's because I want forced decoupling, and I want it to be harder for things to be badly coupled than easier. Like the default choice should be the good choice. And that's what microservices do.
• 29:52
  Whereas in Rails and Ruby, there was a speaker, I should remember his name, a speaker at Nordic Ruby back in 2014, who said, paraphrasing, Ruby lets you create beautiful coupling or something.
• 30:08
  Like that's the purpose of Ruby.
• 30:10
  And it was just like, he was coming from the outside and he had this really eloquent way of saying, Ruby's features are beautiful and they increase coupling in so many different ways.
• 30:20
  You don't know if there's a new method on string, for example, you don't know what's going to happen.
• 30:25
  And that's one of the things that never would have worked, that people thought Ruby is crazy, you can't be doing this.
• 30:30
  So I think more about like, how do I not have to worry about tons of context more than I think about how to increase it?
• 30:37
  But I think you're right that like generators, it's not just conventions, it's like actual code that does things.
• 30:43
  And to your point about text being a big input, well, the input, but textual descriptions versus code being more of an input into LLMs and the training data, all the text tells you to generate things.
• 30:55
  So the LLMs already know that's what you're supposed to do, which is good.
• 30:58
  That's a good start.
• 30:59
  The worst case would be like, oh, I know how to make a Rails app.
• 31:02
  I'm going to, as an LLM, hand code all the components, which means they're going to be like the two-year-old version and not work with Rails 8 or whatever. We've been talking about how Ruby and Rails in certain ways lend themselves well to generated applications. And I agree. I think that's true. I agree with myself. No, I think that's true. Though the pessimistic side of this from a Ruby perspective, when I was thinking about going to RailsConf and how I was going to perceive being there after all these years, the pessimistic side of me says, well, the language that wins is the one that there's an ease of generation, ease of deployment piece, but ease doesn't matter anymore when humans are out of the loop.
• 31:45
  So what really matters is the runtime that's the cheapest to run and provides the best uptime and speed, right?
• 31:55
  So they're like just really boring metrics that are all about machines that ultimately we need to care about.
• 32:02
  Probably there is a bit of like how verbose is it, Valentino, to your point of like more tokens means larger contacts. You know, I'm sure there are ways that brilliant people can compress that stuff out, but succinctness is probably important.
• 32:14
  Is that kind of like another vote in favor of something like Haskell as a better alternative?
• 32:19
  Yeah, or Haskell Rust.
• 32:22
  I mean, it could even be C. I think that would be stupid with LLMs because you have no guardrails.
• 32:27
  Rust is kind of best of all worlds.
• 32:29
  You've got the compiler can check things for you.
• 32:31
  It smells sort of like Haskell in that way, but you can create things that are like super lean, perform really well.
• 32:38
  Yeah, Haskell is a great example.
• 32:39
  We had a service in Haskell that when we were scaling up, we had massive performance problems before a big rewrite that we did that I was leading.
• 32:47
  With all of our servers, we have like at least nine instances of the server running on nine EC2 machines back then.
• 32:55
  With Haskell, we never needed more than one because it was just like everything happened in a millisecond and used no memory and it never crashed.
• 33:04
  You could run it on a laptop, probably.
• 33:06
  The rest of it, we were spending hundreds of thousands of dollars to run this infrastructure.
• 33:10
  That's why we're gaming the show to the Haskell AI podcast coming to you next week.
• 33:16
  I think Haskell has not won.
• 33:18
  By the way, Haskell is an interesting one where when I said, if someone hates my Haskell code and they don't want to modify it, or they don't like Haskell, they can rewrite it.
• 33:27
  Ultimately, the code never broke.
• 33:30
  We never had to change it once it worked because, you know, the type system and there weren't any bugs in it.
• 33:33
  But the build system changed around it in a way that was incompatible with something we had done.
• 33:38
  And we got to a point where we could never compile it again.
• 33:41
  So we had to replace it just because we couldn't get it to compile, which was a very annoying thing for me with my Hasbro project.
• 33:47
  Yeah.
• 33:48
  Yeah.
• 33:49
  You know, I had a follow-up question based on your idea of computation performance at that point.
• 33:56
  So I think what I took away from that is that if Ruby wants to dig in to take more market share, we need to be more performant?
• 34:07
  Yeah, definitely.
• 34:08
  Or the pessimist says, too late.
• 34:11
  Doesn't matter anymore.
• 34:13
  The optimist says, there's a bunch of Ruby code in production, so people already are dealing with the operational cost of Ruby.
• 34:20
  So maybe it's not too late.
• 34:21
  There's tons of great documentation that's easy to generate things and all the stuff we said about Rails before with conventions. So maybe it's not. I. There's tons of great documentation. It's easy to generate things and all the stuff we said about Rails before with conventions.
• 34:26
  So maybe it's not.
• 34:28
  I do worry a little bit, though.
• 34:29
  I know we're on a Ruby-themed podcast now, but the main thing that I was thinking about going to RailsConf is like, you know, I spent so much of my career really identifying with a programming language, whether it be Ruby or something else, or with some sort of like guts of how things work concern, most of us in the future won't have the luxury to do that. If that's even a luxury, the fact that there's a conference called RailsConf, this feels like a blip in time.
• 34:59
  Of course, now it no longer exists. So, you know, maybe... More right than you realize, yeah. It's a blip in time where humans were concerned with this sort of thing at levels that would have thousands of them go to other countries and talk to each other about it. I think it's a dangerous, worrying thing to identify now with an implementation detail. And I think Ruby and Rails are implementation details in the near future versus like cultures and full-time vocations and hobbies. Unless it's like, I guess I still do like racket programming sometimes on the weekend for no reason. So there's probably a place for a hobby. But yeah, it's not just Ruby and Rails.
• 35:46
  Any language community beyond the hobbyists, you say like we want Ruby to be adopted.
• 35:52
  Well, you know, I used to be a director of a nonprofit that was all about advocacy, of course, for Ruby.
• 35:57
  Now I think that isn't a thing that we should be thinking about so much.
• 36:00
  I think we're graduating to another level.
• 36:03
  And it's more like, how do we make these machines build the code for the machines better? And there's a whole bunch of stuff around that, as you know, you know, you've been talking about MCPs for this and that. To me, that's the place that people like us live in the future, as opposed to down in the layer of abstraction of even a web framework and how you implement things in it. Well, so you could be right about that. And Obi Fernandez was on the show a few weeks ago, said something similar, like, hey, you know, I used to really pride myself on, what did he say, the assembler code that he put together. And that's over. You kind of let those things go. But I am curious to know, I probably have a less serious question than Valentino, but it is serious. What do the enthusiasts, the professional enthusiasts do? Where do they go to celebrate what they do together and learn from each other?
• 36:49
  Or is it just a bunch of like, well, we all just kind of direct the machines now and that's good enough.
• 36:54
  We don't have to go to any conference about it.
• 36:56
  Well, there's probably a transition period.
• 36:59
  Obviously, for a long time, there is going to be Ruby and rails and people are going to be typing ruby and rails code and reading ruby and rails code if for no other reason than there's all this legacy code people have to maintain so i think communities continue to exist around this stuff but i think in the future there will always be enthusiasts but it's just a matter of what they're enthusiastic about i think it's going to be like the tools around this stuff and the techniques.
• 37:28
  I'm sure you see this as well.
• 37:29
  The reason I said earlier, I think we should stop using vibe coding as a term, except for as a pejorative.
• 37:35
  So I think there is like an art which will be evolving for quite a long time of how to orchestrate systems so that they get built well. Cheapest, most effective, best user experience. It's still a creative thing. I wrote a short story recently. And when I say I wrote one, I had multiple LLMs collaborate with me on a short story. And ultimately, I wrote none of the text. But I feel deep ownership of the output. No one else would have created what I created in that session.
• 38:07
  It's actually deeply disturbing.
• 38:08
  I'm going to use it for a music project.
• 38:10
  I won't go into it because I ramble too much.
• 38:12
  But at the end of that, I thought, I'm exhausted.
• 38:16
  I feel like I've done something amazing.
• 38:17
  I want to share it with people.
• 38:19
  And I did not write this story.
• 38:21
  I'm not an author of fiction.
• 38:23
  I didn't write the words, but I still feel like I created this thing. And there's a craft to it. And I don't think you could have done it, either of you, because you're not me. You didn't approach it the way I did. And I think there was actually even something like novel and clever of the way I did it. Of course, in programming, I think that to a much greater degree, because I'm so-called expert in software development and architecture. So the way I deal with these LLMs is different than most people.
• 38:50
  And the way you deal with them is different from most people. And it's really exciting. And what I find is when I catch up with my friends in such situations like this, my previous call was with a friend named Jess Martin, who used to be in the Ruby community too. We spent the whole time talking about how we're dealing with LLMs to generate code and how we're orchestrating and making tasks get completed in a way that works, like integrating with GitHub, and that's where it goes now.
• 39:14
  It's just a different thing that you're worried about at a higher level of abstraction.
• 39:18
  Sometimes we'll look at the code that it implements and probably comment on that too, but we don't get together at RailsConf and all of us talk about the specific implementation of some feature like how has many works that's not interesting anymore it used to be in 2008 that was the thing we would debate not anymore it's done curious there are so many problems technically that haven't been solved too. And so where do those problems go in this new state?
• 39:46
  We almost have to have those solved too, right? Before we can get to that ideal state of the LLMs just doing everything. So to me, I worry that that is like something promised early and delivered late and everybody's just ends up having to deal with it. And how do you move fast in that environment, right? The Rails and Ruby, they're really great at moving fast. That's why people choose Rails to begin with is you want to get a business set up on the internet. Rails is fantastic for doing that at lightning speed, right? And so how do we keep that momentum in this new layer of abstraction, right? And so that's something I worry about because Repl.it is great for doing quick things. Somebody wants to deal with UX and try out a new thing, let your design team at it and have them build up an interface and solve your UX for how you want it to work and then hand it off to something else, right? That's going to be more performant longer term. I feel like I'm a little suspicious of that actually being solved near term, even medium term, like in the next three years. I feel like that's a long shot. So I think about like, everybody's going and using all these code generation tools and it's great, but do you still have this review process, right? Trust but verify is going to be a thing for a long time. And when you have to read through thousands of pages of code that it generates at a time, there are ways to reduce that amount with some work. Then you just have more code to review in smaller chunks. And you're not going to review it. It's still reviewing, right? How do you continue to move fast, right?
• 41:25
  Like in this iterative process that we've built up, that we've proven that works, right?
• 41:31
  I know that we're talking about building new processes, but I feel like that generative aspect of things is a huge bottleneck.
• 41:40
  And everybody that I've seen introducing code generation on a large scale introduces a bottleneck that is unseen and it is seen by top people in some regards but it hasn't been realized.
• 41:55
  It's trash most of it. I have so many reactions to what you just said one of them being there is this obsession with one shots which is the wrong way to think about things.
• 42:05
  That's good for demos. The other one is the entire modality of assuming that people are going to sit in an editor and have an assistant. That's silly. That was a complete waste of time and will continue to be. I think it made us all go faster for a little while for those of us that like to use those IDs. I don't, ultimately, I think we have to think about agents as members of the team, you know, not like we have to humanize them, but GitHub, GitHub was built for agents.
• 42:33
  They didn't know that at the time, but I remember the first time I saw Snyk, the security company, S-N-Y-K.
• 42:39
  I talked to the founder.
• 42:40
  He showed me a demo of Snyk making changes to code to like upgrade from security vulnerabilities. I was very skeptical back then. This is like 2017 until he showed me, oh, it just generates a pull request. And I thought, oh, wow. Okay, great. It's a pull request. I'm comfortable with that. And the reason that that turned on the switch of like, oh, computers are going to be able to generate code because I see how it fits into the process.
• 43:05
  So I would like to see agents get introduced to teams in this way where they're just dealing with the programmable API surface areas of all the things that we interact with when we create code.
• 43:17
  We don't need to look at their editor when they're working on it, just like you don't have to look at mine.
• 43:22
  But then you're right.
• 43:23
  If it's going too fast, you're not going to be able to review it quickly enough. That's where it's about new processes to me and architectures that make this okay. Sending the right things to the agents so that you can trust them in the same way that if you work for a big company and you bring on a large contracting company for lower cost labor to do software development, which I have done a lot in my career, you don't get to talk to them all the time. Sometimes you're not speaking the same native language. So there's a bunch lost in translation. They go off and they do a bunch of work and they deliver it all at the same time. And then you've got tons of code to review. You work on a system for 10 years and you're in an environment with rotating staff members and rotating contract companies. You have the exact same problem already that you have with LLMs. You're just getting to the problem state faster, but you're also getting to the positive state faster with LLMs. So it's not all bad. So I think you should worry about it. You know, I was interrupting like excitedly saying we're not going to review the code. Not because I think we shouldn't, although I also think we have to accept it and not review the code because we can't we also can't slow down because no one will tolerate that the incentive structures in the world don't align with us all saying we got to slow down let's be like half AI speed and half human speed not going to be okay that's just not how it works right so look into the future and look what business is going to tell us about how we need to operate on code and we need to be thinking about how that's going to be okay i think you're totally right that there are a bunch of promises and they're not going to be delivered and the whole thing's going faster the cart's going before the horse and all this stuff we have to figure it out and there are going to be people who care about this and who don't have their heads in the sand that are the ones that figure it out. And they're the ones that have a programming community that enthusiasts rally around because they're figuring this stuff out in the same way that like the first like rail scaffold thing, all the Java enterprise people are like, haha, that's ridiculous. That's junk. I don't trust that. I'm worried about that. Yeah, you should have been. But also there are a bunch of people who saw that and said, we're going to make that okay. And we're going to solve those problems as opposed to say, eh, we got to slow down and not use that. Let me know when it's done.
• 45:32
  Because those are the ones that are still doing Java enterprise programming at a terrible corporation. Yeah, I really like that answer. I had something similar to say in a talk a few months ago about the number one reason that AI exists at all when it comes to software is so that humans could go faster. Businesses just want us to program faster.
• 45:53
  So yeah, you're right. At some point, we will have to create a system or get comfortable or something with the fact that we can't review all the code that the AI is writing because our competitor is not going to review all the code that AI is writing and look at all the money they're making according to the owners of the business. Yeah. I mean, that's maybe that's capitalism or that's economics, but I think that's absolutely the way we have to be thinking about it.
• 46:16
  But I also like there's some hope in there, I think from both of you, because you're saying, well, okay, if that's the case, then we need to get a lot better at building the thing to make it okay.
• 46:26
  As you said, Chad, to make it okay means to be innovative and creative to find solutions for that rather than tucking it away or pushing it away.
• 46:35
  Yeah, it's kind of like regulation and alignment and all this stuff.
• 46:38
  If a country slows down on this AI stuff, then the other ones aren't going to.
• 46:44
  Unfortunately, it's like an arms race.
• 46:46
  I think it's not capitalism as much as it is economics, to your point, that it's all about incentives and mechanisms and game theory.
• 46:54
  And by the way, we all have a business.
• 46:57
  You know, you were mentioning the business owner.
• 46:59
  Well, you're the business owner of your career and you're competing with other software developers as well.
• 47:05
  10 years ago, it felt like that would never actually mean anything other than how rich can I be as a software developer?
• 47:11
  It might mean, can I have a job as a software developer in the future?
• 47:15
  And so you need to be making those decisions, not you specifically, Joe, but you know, you as proverbial you, wow, need to be thinking about this, like how do you compete in the industry?
• 47:24
  And if you're not going fast, someone else is. Are they writing perfect code? No, but neither are you.
• 47:30
  So like, what's the gradient? What's the Delta? What are you bringing that's special this time in the world? And I will argue that for most people, the thing you bring that is special and differentiates you will not be your expertise in a specific programming language or framework.
• 47:44
  It might be in a database. It might be in a runtime. If you're an expert at deploying, it's like the containers and the rails on which these agents will work. That will continue to be valuable for some period of time, but the creation of the thing, not so much, not in the details anyway. One thing I've seen recurring in the Ruby community is this idea of a runtime self-tolerant system where it kind of just rebuilds itself as it goes.
• 48:14
  And I can see that becoming a thing as these things become more performant.
• 48:18
  And maybe that is a potential future for Ruby in general. If somebody can go build one of these things where it's safe to run and modify, right, in real time, I feel like there is potential there to see maybe a future, not necessarily for writing the language of your own, but at least using it.
• 48:39
  Yeah, that's interesting.
• 48:40
  Like you could modify in memory with monkey patching in memory.
• 48:44
  If it generates something and it had an issue doing whatever it was trying to do, just having the thing reflect on itself and be like, what should we have done differently and modify and then re-execute?
• 48:53
  I'm afraid of that.
• 48:55
  Yeah, everybody's excited about it.
• 48:56
  Yeah, yeah, yeah.
• 48:58
  I mean, my crazy idea is we could definitely build up a standard library of known good stuff really quickly, whether it's in Ruby or probably Wasm or some other, maybe it's on a domain and tech specific thing. Imagine an incentivized contest where you are rewarded for the best challenges and the best challenges, like the most useful, best specced version of a function that needs to exist that lots of people want right you're rewarded on that end you're also rewarded for the solution and the solution is based on metrics and kpis and of course assertions and you know actual test framework now people could come play this game and they could type code that wouldn't happen for very long if there's a reward right so then it becomes a bot war whose bot does the best thing so imagine now you win the contest for one specific function or library or whatever the unit is now there's an ongoing thing where you're incentivized to make a better version so this has now been canonicalized into the standard library of whatever. And now you've got thousands of agents competing to make the more performant version or the version that uses less memory. And now just like pour tons of money into this thing and watch agents build up an amazing new system of interoperable libraries and components that you can trust because they're all spec'd out in terms of how you verify them.
• 50:26
  I'm sensing a cryptocurrency scheme here.
• 50:29
  Well, it just, it feels that way because that would be a way that makes sense.
• 50:32
  It's really because crypto and mechanism design go together and this is an economic problem.
• 50:37
  To be honest, I could see that incentivizing people to solve challenges and earn from that over time being very fruitful.
• 50:47
  I know I've done coding challenges myself back in the day, no longer, but there's a certain reward you get from it, achieving it. And I imagine agents would feel that in a way.
• 50:58
  Well, you would still be a person trying to coerce the agent, trying to program the agent.
• 51:03
  Yeah, exactly. You're trying to create the bot that'll do it.
• 51:06
  Yeah.
• 51:06
  If the system existed and was successful, people could just make their living asking bots to do the best, most clever version of this.
• 51:13
  I think it'd be an interesting way to like race to some new understanding of how to generate code well.
• 51:19
  Let's do it.
• 51:20
  All right.
• 51:20
  After this call, we'll hang up and start on.
• 51:24
  Botchallenge.com.
• 51:26
  The first thing I will type is Claude space dash dash dangerously skip permission check, whatever it's called. It's amazing how much that is getting adopted that I've been seeing people do just like, yeah, whatever. YOLO. It makes me nervous. Welcome to the future.
• 51:45
  You should be nervous.
• 51:46
  Yeah, I should be nervous.
• 51:47
  And it's just what I was saying.
• 51:48
  You should be nervous and you cannot stop it.
• 51:50
  So we have to just assume this is the case.
• 51:53
  We really loved having you on here.
• 51:54
  We'd love to have you back again, especially after this new chatbot challenge is up and running.
• 52:00
  And you can talk about the winning of the purse.
• 52:02
  We'll be too busy raking in the money after that.
• 52:06
  It was fun.
• 52:07
  Yeah.
• 52:07
  And anytime.
• 52:09
  I love to talk about this, obviously.
• 52:11
  Yeah, us too.
• 52:12
  Thanks so much.
• 52:12
  I appreciate it.
• 52:13
  If you've got any more time, we do this little segment at the end where we just share something that you're interested in.
• 52:18
  It doesn't have to be specific to what we're talking about.
• 52:21
  We'll just give you the opportunity to share whatever you want. Because of AI, I am finally learning 3D modeling and 3D printing after wanting to do it for years.
• 52:31
  It is amazing how quickly you can go from nothing to like actually solving a problem in your house.
• 52:36
  So the first project I did was I created a new end cap for my saxophone that protects the octave key because I have one that I love.
• 52:44
  Then I had another one I didn't love and I have two alto saxophones and I protects the octave key. Because I have one that I love, then I had another one I didn't love, and I have two alto saxophones, and I lost the bad one.
• 52:48
  So I measured with calipers and created a new end cap.
• 52:53
  It was my first project.
• 52:54
  It took me like three hours.
• 52:56
  And it was all because I was asking ChapGPT and like showing it my screen.
• 53:00
  I just knew what to do.
• 53:01
  It's amazing.
• 53:02
  I sound like such an AI show, but it's just such an exciting new world.
• 53:07
  It really is.
• 53:07
  You know, I have the similar one with circuit printing and configurations.
• 53:12
  You know, hey, chat GPT, what would you do for this?
• 53:14
  I even took a picture of tiny hardware components that I had lying around.
• 53:18
  I'm thinking about making this thing.
• 53:20
  How would you lay it out?
• 53:21
  Oh, yeah.
• 53:21
  I got an ESP32 dev kit for doing IoT this weekend, too.
• 53:25
  It's all related to 3D printing and all this.
• 53:28
  And it had a QR code and a URL to get instructions and a tutorial.
• 53:33
  Of course, the URL's dead.
• 53:34
  And I think the company doesn't exist.
• 53:36
  I was getting a knockoff.
• 53:37
  So I dumped out all the components on a table, and I took a picture of it, and Pat PPT sorted through and told me what it all was.
• 53:45
  Amazing. That's awesome. And it walked me through projects. It's the future, man. Yeah. You can't not talk about it. Right. Awesome. All right, Chad. Thanks very much. Thanks again, Chad.
• 53:54
  And it's great having you. Take care. All right. Thank you.