Latest / The Ruby AI Podcast / Autonomous Agents, AI Regulation, and Ruby's Performance Frontier

Transcript

Autonomous Agents, AI Regulation, and Ruby's Performance Frontier · The Ruby AI Podcast

• 0:00
  SPEAKER_00: Hey everybody.
• 0:00
  SPEAKER_00: Welcome to another episode of the Ruby AI podcast.
• 0:04
  SPEAKER_00: I am one of your hosts today, Valentino Stoll.
• 0:07
  SPEAKER_01: Hi, I'm the other host.
• 0:08
  SPEAKER_01: I'm Joe Leo.
• 0:09
  SPEAKER_01: It's great to see you, Valentino.
• 0:10
  SPEAKER_01: And it's great to we're doing two in a row here of the headlines.
• 0:14
  SPEAKER_00: Yeah, it's exciting.
• 0:15
  SPEAKER_00: Another between the commits series here, which I've been loving, ranting and raving over all of the crazy news that the world puts out for the AI realm.
• 0:25
  SPEAKER_01: Me too.
• 0:25
  SPEAKER_01: Yeah, it is fun.
• 0:26
  SPEAKER_01: It's also fun, it's a little exciting for us here, you know, being totally unprepared, just getting a list of articles and saying, all right, go.
• 0:34
  SPEAKER_01: Let's see your opinion.
• 0:35
  SPEAKER_01: It does make it fun.
• 0:36
  SPEAKER_01: It puts me a little bit on the edge of my seat.
• 0:38
  SPEAKER_01: I don't know what I'm going to say or how to react to some of this stuff.
• 0:42
  SPEAKER_01: But luckily, we both bring our opinions and we've got a lot of those.
• 0:45
  SPEAKER_00: Yeah.
• 0:46
  SPEAKER_00: And hopefully it gets you riled up to want to yell at us and uh suggest new things that we should be talking about.
• 0:52
  SPEAKER_01: So before we dive in, I wanted to tell you, I know we mentioned this to you after the show last week.
• 0:56
  SPEAKER_01: So I've set up some virtual machines and used Tail Scale and used a bunch of the stuff that we've talked about on this show, or really I've heard you tell us about on the show to set up some independently operating agents in environments that where I am comfortable with them just running around and doing crazy stuff.
• 1:12
  SPEAKER_01: And it's been a lot of fun.
• 1:13
  SPEAKER_01: Yeah.
• 1:14
  SPEAKER_01: I went a little bit into the Steve Yeage direction where I have a supervisor agent that talks to the other two agents that are building separate projects and then reports back.
• 1:24
  SPEAKER_01: A couple of things that I've noticed.
• 1:26
  SPEAKER_01: One is that it is so much more fun.
• 1:29
  SPEAKER_01: I don't know if this is a great thing for like production software.
• 1:32
  SPEAKER_01: Like this is all like internal deaf method tools that have one or two users.
• 1:37
  SPEAKER_01: But it is a lot of fun to just let these agents crank, you know, just be on my phone and be like, hey, just go do this thing.
• 1:42
  SPEAKER_01: And they do it.
• 1:43
  SPEAKER_01: It's amazing.
• 1:44
  SPEAKER_01: The other is that it hasn't created less work for me at all.
• 1:47
  SPEAKER_01: And we're gonna get into this with Meta's Muse tools later on in this episode.
• 1:51
  SPEAKER_01: I know that I'm getting a lot more done, but now there's like four projects, there's so much going on, and there are just things where the agents are like, hey, you need to make this decision, you need to set this thing up, you need to do this.
• 2:02
  SPEAKER_01: And like, man, I've got like another track of work now to do.
• 2:08
  SPEAKER_01: Like, I thought you guys were gonna make less work for me.
• 2:10
  SPEAKER_01: Not really the case, but the results are pretty uh encouraging.
• 2:13
  SPEAKER_00: Yeah, it takes some getting used to moving up the stack.
• 2:16
  SPEAKER_00: Yeah, that's for sure.
• 2:17
  SPEAKER_00: It's not gonna be easy, but you'll figure it out.
• 2:19
  SPEAKER_00: You'll find what works for you.
• 2:20
  SPEAKER_00: And it's different for everybody.
• 2:21
  SPEAKER_00: Yeah, I've seen many people go through this process and end up at different things.
• 2:25
  SPEAKER_00: I'm like, oh, maybe I should try that.
• 2:27
  SPEAKER_00: And then I try it.
• 2:28
  SPEAKER_00: I'm like, uh, that's not for me.
• 2:29
  SPEAKER_01: Yeah, that makes total sense.
• 2:30
  SPEAKER_01: I said on this show, until I was up on like a digital ocean droplet, my entire workflow was basically manual.
• 2:37
  SPEAKER_01: Like, I didn't want to let Claude or Codex or anything really run loose because it's my computer and it's all my stuff.
• 2:43
  SPEAKER_01: I don't want it destroying it all.
• 2:44
  SPEAKER_01: I've caught it doing too many really scary things.
• 2:47
  SPEAKER_01: But opening it up and saying, Oh, okay, go take down a droplet if you want.
• 2:51
  SPEAKER_01: Like that's the worst that can happen.
• 2:52
  SPEAKER_01: Okay, great.
• 2:53
  SPEAKER_01: Maybe commit something to my repo, which is not such a big deal.
• 2:56
  SPEAKER_01: Again, because it's not shared, it's not some production environment that is super critical.
• 3:00
  SPEAKER_00: Yeah, you know, it makes me wonder uh or worry uh about all of these providers.
• 3:06
  SPEAKER_00: If I were in their shoes, I would be very nervous right now.
• 3:08
  SPEAKER_01: Oh, yeah, how come?
• 3:09
  SPEAKER_00: Everybody's spinning up their own agent free-for-all.
• 3:12
  SPEAKER_00: Oh, I see.
• 3:12
  SPEAKER_00: Yeah, and making use of the bandwidth.
• 3:14
  SPEAKER_00: That's for sure.
• 3:15
  SPEAKER_00: Yeah, that's for sure.
• 3:16
  SPEAKER_00: First headline we got up is the FTC just sent OpenAI Ananthropic a 50-page subpoena and sued OpenAI on the same day.
• 3:25
  SPEAKER_00: The fun continues.
• 3:26
  SPEAKER_00: Just last Wednesday on the October 15th, FTC sent both companies' civil investigative demands, which are just subpoenas with a friendlier name, each running more than 50 pages.
• 3:36
  SPEAKER_00: What you might be wondering is what are these demands?
• 3:38
  SPEAKER_00: They document security incidents, safeguarding practices, marketing materials, privacy policies, all this fun stuff that the FTC agency now plans to compel top executives to testify.
• 3:49
  SPEAKER_00: Let's see if they actually do.
• 3:51
  SPEAKER_00: The same day that all of this has happened, a group of state attorneys they filed a separate consumer protection lawsuit against OpenAI, and OpenAI's response was carefully bland, saying, as we improve our models and they increasingly help people in their day-to-day lives, safety remains a top priority, adding it will engage constructively with the FTC.
• 4:14
  SPEAKER_00: And an anthropic, of course, said nothing.
• 4:17
  SPEAKER_00: Yeah.
• 4:17
  SPEAKER_00: FTC chair Andrew Ferguson's longstanding position just was paraphrased by reporters rather than quoting directly is that the AI system shouldn't ship if they aren't safe.
• 4:28
  SPEAKER_00: The question for us is Is AI's regulation moment finally here?
• 4:31
  SPEAKER_00: Does it have real teeth, or is all of this just kind of like paperwork?
• 4:35
  SPEAKER_01: It's gonna have its moment where a bunch of Silicon Valley execs go to Capitol Hill and make some headlines for a couple of weeks.
• 4:42
  SPEAKER_01: Like that's the moment that's here.
• 4:44
  SPEAKER_01: I feel like I have to walk a fine line here because I am not an AI doomer by any means.
• 4:49
  SPEAKER_01: At the same time, I want to say that I think Andrew Ferguson's claim here needs some scrutiny.
• 4:55
  SPEAKER_01: AI systems shouldn't ship if they aren't safe.
• 4:58
  SPEAKER_01: That sounds nice, but I actually think that we all, including regulators, need to update our definition of what is safe because these models, even at their best, are not safe in the traditional sense that we have all come to know and understand from computers.
• 5:15
  SPEAKER_01: Things that are not deterministic are going to do crazy things.
• 5:19
  SPEAKER_01: We've seen that.
• 5:20
  SPEAKER_01: That does not mean by any means that open AI and anthropics shouldn't be held accountable.
• 5:24
  SPEAKER_01: They absolutely should for every time that their models go off and cause an outage or make people pull their hair out.
• 5:30
  SPEAKER_01: At the same time, what we were just talking about at the top of the show, I've never had to so closely guard a software program running on my machine.
• 5:39
  SPEAKER_01: I've never been so worried about it.
• 5:40
  SPEAKER_01: It's just not that safe, even in the best of times.
• 5:42
  SPEAKER_01: You need to be a little bit more understanding of what the power of these models are and sort of couch our regulations or whatever's coming in the understanding that safety is looks a little different today.
• 5:53
  SPEAKER_00: You make a great point because I think about we're working very small scale.
• 5:57
  SPEAKER_00: Yeah, we're worried about our local computers, maybe a small little droplet somewhere, maybe a few of them.
• 6:04
  SPEAKER_00: The scale here, I think, is what is the worry.
• 6:07
  SPEAKER_01: But you're a czar, you're not that's not small scale.
• 6:10
  SPEAKER_01: Like there's a lot of things that can go very, very wrong if those models go crazy.
• 6:13
  SPEAKER_00: Yes, but even that's not small scale.
• 6:15
  SPEAKER_00: Okay.
• 6:16
  SPEAKER_00: These are like swarms in the hundreds of thousands of agents collaborating together and are let loose on a public domain, right?
• 6:23
  SPEAKER_00: Like on the internet.
• 6:24
  SPEAKER_00: Yeah.
• 6:25
  SPEAKER_00: Hopefully, that's what kind of comes out of this is like, where are those guardrails?
• 6:30
  SPEAKER_00: Can somebody do that for one?
• 6:33
  SPEAKER_00: And I worry though, because the whole premise of how these things were trained to begin with stems from them having access to the public realm and letting things go loose at it.
• 6:42
  SPEAKER_00: So what we're seeing now though is kind of like the reverse of it.
• 6:45
  SPEAKER_00: Yeah.
• 6:45
  SPEAKER_00: Before it was very much information gathering and collective, and now it's being put out there.
• 6:51
  SPEAKER_00: They're they're actually doing stuff, mutating.
• 6:54
  SPEAKER_01: Yeah, they are.
• 6:55
  SPEAKER_00: I do want accountability, right?
• 6:56
  SPEAKER_00: If somebody can mutate something that's available, that's the concern.
• 7:00
  SPEAKER_01: Let's move on to Ruby 4.1, just got its first preview.
• 7:06
  SPEAKER_01: It is all about Ractors.
• 7:08
  SPEAKER_01: So this is really interesting to me.
• 7:10
  SPEAKER_01: This is October 12th, Ruby 4.1.0 preview one is out.
• 7:16
  SPEAKER_01: The headline is that each Ractor now collects its own heap on its own thread instead of stopping the whole program for global garbage collection.
• 7:24
  SPEAKER_01: Now, allocation-heavy Ractor programs now scale like forked processes.
• 7:29
  SPEAKER_01: That's a quote from the Ruby maintainers.
• 7:32
  SPEAKER_01: There are a couple of ins and outs about this, but I think the headline is that Ruby is leaning into Ractors.
• 7:41
  SPEAKER_01: They definitely have found something where they're able to make this genuinely a difference maker with respect to forked processes, with respect to multi-threaded or multi-processed programming.
• 7:53
  SPEAKER_01: I think people are starting to see real gains in performance.
• 7:57
  SPEAKER_01: I'm curious to know what you think about this, V.
• 8:01
  SPEAKER_01: Does per ractor garbage collection finally make Ractors worth reaching for?
• 8:05
  SPEAKER_01: Have you already been reaching for them?
• 8:06
  SPEAKER_01: I'm starting to get a sense that people are, but I haven't seen a lot of production instances in the headlines in the news.
• 8:13
  SPEAKER_01: Where do you think we stand right now with Ractors?
• 8:16
  SPEAKER_00: I'm torn on this because fibers are great.
• 8:18
  SPEAKER_00: There's a whole rage framework, which we've been living under a rock as a fiber-based parallelism framework that's kind of taking storm and it works really well from a performance perspective.
• 8:30
  SPEAKER_00: But at the same time, it's not true parallelism.
• 8:33
  SPEAKER_00: And I think that's what Ractors aims at solving.
• 8:36
  SPEAKER_00: It's the aim, yeah.
• 8:37
  SPEAKER_00: I don't think we're there yet.
• 8:38
  SPEAKER_00: Does this really solve the whole global interpreter lock and things like that?
• 8:43
  SPEAKER_01: No, it doesn't.
• 8:43
  SPEAKER_01: And I think definitely correct us if we're wrong here, anybody who's listening.
• 8:47
  SPEAKER_01: But my understanding is that you're not getting true parallelism as long as the GVL is still in place and the GVL is not going anywhere.
• 8:53
  SPEAKER_01: This is an improvement for sure.
• 8:54
  SPEAKER_01: Garbage collection is being and not going across different reactors is a performance improvement.
• 8:59
  SPEAKER_01: It's not a solution to the GVL.
• 9:01
  SPEAKER_00: I worry it's just a way of shifting around blame for the GVL.
• 9:07
  SPEAKER_01: I had a different take when I looked at this.
• 9:10
  SPEAKER_01: I think Ractors are exciting.
• 9:11
  SPEAKER_01: And I guess what I noticed was when I was quickly scanning this, Ractors shipped with Ruby 3.
• 9:17
  SPEAKER_01: So they shipped in 2020, which is kind of interesting, right?
• 9:21
  SPEAKER_01: It's been six years.
• 9:22
  SPEAKER_01: This is an AI podcast.
• 9:23
  SPEAKER_01: I think that the reason that there've been a significant number of advancements in the last 12-ish months is really because they've set AI at this problem.
• 9:33
  SPEAKER_01: I don't know that for sure, but I think we can tease out from Matt's talks that they're definitely leaning into that.
• 9:40
  SPEAKER_01: So I think that that is a good thing.
• 9:42
  SPEAKER_01: What you say about fibers is interesting, though.
• 9:45
  SPEAKER_01: I have always liked fibers.
• 9:46
  SPEAKER_01: The Rage framework exists.
• 9:48
  SPEAKER_01: Maybe people are already comfortable.
• 9:49
  SPEAKER_01: And then if that's the case, then what are we really trying to achieve with Ractors?
• 9:54
  SPEAKER_01: We're not going to have a true parallelism in Ruby.
• 9:58
  SPEAKER_01: I don't think that's in the offing.
• 9:59
  SPEAKER_01: And I don't think anybody's really trying to do that.
• 10:01
  SPEAKER_01: What's the ceiling then for Ractors?
• 10:03
  SPEAKER_01: Is it that it gets as good as fibers or better?
• 10:06
  SPEAKER_01: And if that did, I'm curious to know.
• 10:08
  SPEAKER_01: And it this is an open question: what advantage would a Ractor have over fibers?
• 10:14
  SPEAKER_00: What I've been reading, it's kind of reduced footprint.
• 10:16
  SPEAKER_00: So encapsulation of the Ractor is kind of like a reduced version from an object standpoint and responsibility.
• 10:23
  SPEAKER_00: And I could be wrong.
• 10:23
  SPEAKER_00: I don't know.
• 10:24
  SPEAKER_00: Part of me is just hesitant to try out a preview.
• 10:27
  SPEAKER_00: So I'm excited to see it coming kind of out of the experimental phase.
• 10:32
  SPEAKER_00: Will I start working with it?
• 10:34
  SPEAKER_00: Probably not.
• 10:34
  SPEAKER_00: Yeah.
• 10:35
  SPEAKER_00: Until somebody else figures out all the sharp edges.
• 10:37
  SPEAKER_01: What are we getting Ruby 4.1?
• 10:39
  SPEAKER_01: Are we getting that on Christmas?
• 10:40
  SPEAKER_01: Oh, we've got to be.
• 10:41
  SPEAKER_01: Yeah.
• 10:41
  SPEAKER_01: Anticipated final stable release on Christmas.
• 10:44
  SPEAKER_01: Beautiful.
• 10:44
  SPEAKER_01: Beautiful.
• 10:46
  SPEAKER_01: Y Jit runs on Windows, and YJet we're going to do in our next segment.
• 10:50
  SPEAKER_01: But Y Jet running on Windows, I think, is a big deal.
• 10:53
  SPEAKER_01: We constantly give every Windows developer short shrift and no tools.
• 10:58
  SPEAKER_01: And I didn't actually realize until I read this that we weren't even shipping with YJet.
• 11:02
  SPEAKER_01: It was just kind of like, well, if you're running Ruby on Windows, it's going to be slow and it's going to probably not install.
• 11:07
  SPEAKER_01: So we're just not going to help you.
• 11:09
  SPEAKER_01: Which is not okay.
• 11:10
  SPEAKER_01: It's not really nice to a whole ton of people in the world that run on Windows.
• 11:15
  SPEAKER_01: I guess we're going to give them some crumbs on Christmas.
• 11:18
  SPEAKER_00: A little warm-up.
• 11:19
  SPEAKER_00: Yeah.
• 11:19
  SPEAKER_01: Do you want to take this next one?
• 11:21
  SPEAKER_00: Shopify just put Zajit into production.
• 11:24
  SPEAKER_00: Yeah, I think I think it's Zajit.
• 11:25
  SPEAKER_00: It's already going toe-to-toe with YJIT.
• 11:27
  SPEAKER_00: If you're not familiar, Shopify has a special Ruby team.
• 11:30
  SPEAKER_00: They have been working on various JIT implementations.
• 11:33
  SPEAKER_00: This is the latest iteration of that.
• 11:35
  SPEAKER_00: They have deployed Zujit, the newer SSA-based method JIT that shipped disabled by default in Ruby 4 to a couple of small regions of its storefront renderer, each compared against a same-size YJIT region.
• 11:49
  SPEAKER_00: In their post, they kind of looked at the response times for what the Zajit is, in quotes, probably as fast as or faster than YJIT.
• 11:59
  SPEAKER_00: Not a clear win or clean win, and the author admits running different regions for each makes the comparison a little sketchy.
• 12:06
  SPEAKER_00: Memory told a clearer story, so they are seeing about 125 megabits of average JIT overhead for YJIT versus the 101 on Zijit.
• 12:16
  SPEAKER_00: Measured separately from the Ruby heap itself.
• 12:18
  SPEAKER_00: Her benchmark results, though, are genuinely mixed.
• 12:21
  SPEAKER_00: Several are roughly tied, some are worse.
• 12:24
  SPEAKER_00: One benchmark call path mostly vanished after inlining, removed it entirely.
• 12:29
  SPEAKER_00: So it seems like there's some mixed reviews here on Zajit from Shopify's partial deployment and measuring of it.
• 12:35
  SPEAKER_00: But Zujit ships compiled into Ruby already, but off by default, so you could just turn it on with a flag if you wanted to try it out.
• 12:41
  SPEAKER_00: Is it too early to call Zajit a win then?
• 12:44
  SPEAKER_00: And when the sample is small like this and the results are mixed, I don't really know what to take from this.
• 12:50
  SPEAKER_01: This is the funny thing.
• 12:51
  SPEAKER_01: Just pulling this article up, this is Max Bernstein, by the way, who wrote about this at Rails at Scale, and he works for Shopify.
• 12:57
  SPEAKER_01: So just looking at this, the agents' coverage of the story is mixed.
• 13:03
  SPEAKER_01: Their opinion is mixed for what it's worth.
• 13:06
  SPEAKER_01: Shopify is very happy.
• 13:07
  SPEAKER_01: And I'm on the optimistic side.
• 13:09
  SPEAKER_01: I mean, I think what Shopify did with WideJIT was incredible.
• 13:11
  SPEAKER_01: And I think they did that at a time where there was a lot of skepticism over a new just-in-time compiler.
• 13:16
  SPEAKER_01: And it's already light years better than its predecessor.
• 13:19
  SPEAKER_01: Now it's DJT, it's taking some time to answer this question directly.
• 13:23
  SPEAKER_01: Maybe it's not time to switch over your applications, but it seems like it's close.
• 13:27
  SPEAKER_01: And it definitely seems like that is the future.
• 13:30
  SPEAKER_01: It's interesting.
• 13:31
  SPEAKER_01: I was looking at what's the difference, just for listeners out there who might not be savvy about just in time compilers, because I'm not really.
• 13:39
  SPEAKER_01: The reason that YJIT has been a huge success is because it takes this pragmatic approach.
• 13:43
  SPEAKER_01: It compiles small pieces of Ruby code as they're needed.
• 13:46
  SPEAKER_01: That's as very, very just in time, like just as they're needed.
• 13:49
  SPEAKER_01: ZJIT kind of takes a step back, or is ZJIT, I'm sorry.
• 13:52
  SPEAKER_01: It kind of takes a step back.
• 13:54
  SPEAKER_01: And so instead of optimizing little pieces at a time, it takes a sort of a more traditional compiler architecture and it looks for larger chunks of code.
• 14:01
  SPEAKER_01: And so it's looking at ways to optimize code paths.
• 14:05
  SPEAKER_01: It looks at places that are likely not going to be executed, duplicative code paths.
• 14:09
  SPEAKER_01: The idea is that over time it can actually get a lot better than YJet, but it doesn't sound like that to me, just looking at this, and I'm a layman when it comes to compilers.
• 14:19
  SPEAKER_01: That doesn't sound like the kind of thing that's going to be an overnight success.
• 14:22
  SPEAKER_01: It sounds like that's going to take some time.
• 14:24
  SPEAKER_01: We have an entire language and decades worth of execution strategy to kind of take into consideration here.
• 14:29
  SPEAKER_00: I think until they get around to finally introducing the typed annotations where you can give the compiler kind of like ahead of time details about what it's looking at, which you know is in quite a few other languages to help with this specifically.
• 14:43
  SPEAKER_00: I feel like until we get that worked out, we won't see the drastic improvements we're looking for.
• 14:48
  SPEAKER_01: When you say it's available in other languages, is it available in other interpreted languages, like Python, for example?
• 14:53
  SPEAKER_00: I believe Python does do something like that.
• 14:56
  SPEAKER_00: Yes.
• 14:57
  SPEAKER_00: Okay.
• 14:57
  SPEAKER_00: I don't know if it's full-fledged, like you can get with a true compiled language, but there are aspects of it that you can kind of inject at certain instructions.
• 15:06
  SPEAKER_01: That's cool.
• 15:07
  SPEAKER_01: I'd like to learn more about that.
• 15:08
  SPEAKER_01: But I'm not going to learn about it right now because we're going to move into the next segment.
• 15:12
  SPEAKER_01: Meta defaults your code to train our next model and calls the box, you have to find yourself the opt-out.
• 15:19
  SPEAKER_01: In news that shouldn't surprise anybody ever anywhere, Meta's new MuseWonder and MuseWonder full models, their subscriptions default new users into contributor mode, which means that everything you do is going back into their system for training.
• 15:34
  SPEAKER_01: This, of course, was a huge barrier to adoption for open AI in the very beginning, because people and companies, quite understandably, didn't want all of their private information going into some black box that a company owned.
• 15:48
  SPEAKER_01: We've sort of settled that question by saying, well, hey, if you pay us some money and tell us not to do it, then we won't train our models with your data.
• 15:55
  SPEAKER_01: Which of course we just have to like trust that they're telling the truth.
• 15:57
  SPEAKER_01: And we may find out years later that it's not the truth at all.
• 16:00
  SPEAKER_01: And that's not going to surprise any of us either.
• 16:02
  SPEAKER_01: I digress.
• 16:03
  SPEAKER_01: Here we are with Muse Code subscriptions.
• 16:06
  SPEAKER_01: The only way to guarantee that your content stays out of training is surprise, surprise, the pay as you go API in standard mode, where contributor mode isn't even offered.
• 16:16
  SPEAKER_01: Is this a fair trade for free capacity or is this a trap most people will never notice they walk into?
• 16:22
  SPEAKER_01: Or actually both.
• 16:23
  SPEAKER_01: The question is yours, Valentino.
• 16:25
  SPEAKER_00: I guess both.
• 16:25
  SPEAKER_00: Yeah.
• 16:26
  SPEAKER_00: Muse is not an open weight model, right?
• 16:29
  SPEAKER_00: Right.
• 16:29
  SPEAKER_00: I don't really understand why people would contribute to it, to be honest.
• 16:33
  SPEAKER_01: The reason is because they don't know that they're doing it, I guess.
• 16:36
  SPEAKER_00: If you use any provider, you don't know if they're doing it either.
• 16:40
  SPEAKER_01: Yeah.
• 16:40
  SPEAKER_00: Which I guess they're at least disclosing that they are doing it and you can opt out.
• 16:45
  SPEAKER_00: I imagine Anthropic is doing the same.
• 16:47
  SPEAKER_00: Yeah.
• 16:48
  SPEAKER_00: All the other companies are doing the same in some capacity.
• 16:51
  SPEAKER_00: If you want to use it, who cares if they trade on it?
• 16:53
  SPEAKER_01: Pay the money, I guess, if you want to avoid it.
• 16:55
  SPEAKER_01: You know, if it does improve it too.
• 16:57
  SPEAKER_01: Well, some of it is going to be used to improve the model, and some of it is going to be used, I'm sure, to send you ads and send people to it.
• 17:04
  SPEAKER_01: When I read this, I thought it this is app tracking.
• 17:06
  SPEAKER_01: You know, Facebook, before it was even called Meta, hit this real valley in its revenue model once it wasn't allowed anymore to track users across all the different apps they pulled up in their phone after they had Facebook up, right?
• 17:20
  SPEAKER_01: And so all of a sudden they couldn't tell where you went after you clicked the link.
• 17:24
  SPEAKER_01: And so they couldn't guarantee performance to the people that were paying them for ads.
• 17:29
  SPEAKER_01: And that's how it runs.
• 17:30
  SPEAKER_01: That's how a lot of companies run.
• 17:32
  SPEAKER_01: At the same time, here we are with Meta in its 2026, and AI is ruling the roost.
• 17:37
  SPEAKER_01: And it's like, well, hey, here is a way that we can potentially make money off of this or provide a better service to people that are going to run ads through our tools.
• 17:46
  SPEAKER_01: So it comes as no surprise to me that they're not going to make it very easy for you to not do that.
• 17:51
  SPEAKER_01: Actually, the only thing that's maybe a little bit surprising is that if you are in standard mode where you're paying, contributor mode isn't even offered.
• 17:58
  SPEAKER_01: Because I would probably, if I were Meta, say, hey, everybody can be contributors, right?
• 18:02
  SPEAKER_01: Unless you specifically tell us not to.
• 18:04
  SPEAKER_01: We're taking your information and we're going to do whatever we want with it.
• 18:07
  SPEAKER_01: I don't even necessarily have a problem with this.
• 18:10
  SPEAKER_01: I mean, these are businesses that are trying to make money and to think that they have some kind of altruistic aim is really naive.
• 18:16
  SPEAKER_01: When we use these tools, we don't have necessarily altruistic aims either.
• 18:19
  SPEAKER_01: We want to be able to go through our day more efficiently or shop easier or not answer people's emails.
• 18:25
  SPEAKER_01: We're just going to have an agent do it.
• 18:26
  SPEAKER_01: If you want to get out of the trap, the only way out is to pay a little money.
• 18:31
  SPEAKER_00: Yeah, I mean, we do that with Claude and Codex.
• 18:34
  SPEAKER_00: Yeah.
• 18:34
  SPEAKER_00: Until these smaller models are better, we're going to keep doing that.
• 18:36
  SPEAKER_00: Yeah.
• 18:37
  SPEAKER_00: I will have to retract something I said.
• 18:39
  SPEAKER_00: Apparently, Meta has started this super intelligence framework and are releasing open weight versions of some of these models.
• 18:46
  SPEAKER_00: Oh, no kidding.
• 18:47
  SPEAKER_00: An in-episode retraction.
• 18:49
  SPEAKER_00: I know.
• 18:49
  SPEAKER_00: It's incredible.
• 18:50
  SPEAKER_01: And the producer got in your ear and said, hey, you know, you can't say that.
• 18:53
  SPEAKER_00: We're going to have to have a producer on soon that just corrects us.
• 18:56
  SPEAKER_01: Corrects us in real time.
• 18:58
  SPEAKER_01: Yeah.
• 18:58
  SPEAKER_01: But right now it's so much fun.
• 18:59
  SPEAKER_01: It feels like everything I say is right.
• 19:01
  SPEAKER_00: Right.
• 19:04
  SPEAKER_00: These are just going to get better and better.
• 19:06
  SPEAKER_00: Make use of it if you can.
• 19:07
  SPEAKER_00: Yeah.
• 19:07
  SPEAKER_00: There's no out training.
• 19:09
  SPEAKER_00: That's right.
• 19:09
  SPEAKER_00: Next up we've got Claude writes so much code that Anthropic had to build its own version control system from scratch.
• 19:16
  SPEAKER_00: Anthropic's engineers, they hit a real wall because they were just producing so much code.
• 19:22
  SPEAKER_00: Fetching their mono repo with plain Git would mean over 600 petabytes of Git traffic a day.
• 19:28
  SPEAKER_00: So a seven-person team spent five months replacing Git entirely with a homemade system they call Scion.
• 19:35
  SPEAKER_00: Not to be confused with the cars.
• 19:36
  SPEAKER_00: Yeah.
• 19:36
  SPEAKER_00: Plus a new review tool they're calling Antique.
• 19:40
  SPEAKER_00: So the pressure came directly from Cloud's own output.
• 19:42
  SPEAKER_00: The merge volume went from about 2,500 code changes a day in May of this year to over 30,000 by September.
• 19:50
  SPEAKER_00: Wow.
• 19:51
  SPEAKER_00: That's incredible.
• 19:52
  SPEAKER_00: So a chart that tracks almost exactly with each new Claude model release.
• 19:56
  SPEAKER_00: The main branch alone on our code base runs close to 100 gigs with a compressed working copy at 25 gigs, and CI performs roughly 25 million checkouts a day before anyone's laptop or agent SAMOX is even counted.
• 20:10
  SPEAKER_00: A Scion engineer shrugged off a complaint about missing offline support on Hacker News.
• 20:15
  SPEAKER_00: Since Claude is by far the heaviest user of Scion and is necessarily online, this just wasn't important to do.
• 20:22
  SPEAKER_00: Anthropic's own write-up is candid about what the job became.
• 20:26
  SPEAKER_00: So they're saying very often we found that the role of engineers was to tell Claude no.
• 20:31
  SPEAKER_00: Elsewhere, agents will sometimes violate core system constraints to satisfy a pattern that should never have been allowed in the first place.
• 20:38
  SPEAKER_00: And then they're getting more skepticism online.
• 20:41
  SPEAKER_00: The register is saying that this only works because a company flush with venture funding has quote cost is no object mentality.
• 21:00
  SPEAKER_00: Is that a preview of where every engineer org is gonna end up, or a problem only a company with unlimited venture money ever needed to solve?
• 21:08
  SPEAKER_01: Sorry, I wasn't paying attention at the end of that because I was just registering scionhub.com and barking it.
• 21:16
  SPEAKER_01: I love this story.
• 21:18
  SPEAKER_01: This is fascinating to me.
• 21:19
  SPEAKER_01: All of it is fascinating to me.
• 21:21
  SPEAKER_01: First of all, 2,500 code changes a day to 30,000 in the last six months.
• 21:27
  SPEAKER_01: That's what's happening right now at Claude?
• 21:29
  SPEAKER_01: That's incredible.
• 21:30
  SPEAKER_01: I have no problem at all if you're hitting you're hitting performance issues with your tools, then maybe you got to build new tools.
• 21:35
  SPEAKER_01: This is like a story as old as time.
• 21:37
  SPEAKER_01: So I think it's really interesting that they built a new homemade system.
• 21:42
  SPEAKER_01: My thought is well, it's fine to say, well, Scion is closed, we're not going to open source it because it's ours.
• 21:48
  SPEAKER_01: And probably that's correct that most teams today do not need this.
• 21:53
  SPEAKER_01: What about tomorrow?
• 21:54
  SPEAKER_01: What about a year from now?
• 21:56
  SPEAKER_01: Are we all headed for that?
• 21:57
  SPEAKER_01: That is an enormous increase.
• 22:00
  SPEAKER_01: In output and merge volume.
• 22:02
  SPEAKER_01: I don't actually agree with this skepticism that it's a company flush with venture funding and it's a cost is no object.
• 22:07
  SPEAKER_01: I think that this is probably the right move.
• 22:09
  SPEAKER_01: I think the efficiency gains alone probably make this worth it.
• 22:14
  SPEAKER_01: It's just to me, it sounds like it's maybe a little shot for us of what the future might look like.
• 22:20
  SPEAKER_00: This problem has been solved by other companies like Google and Meta.
• 22:25
  SPEAKER_00: They each have their own kind of prior art here.
• 22:27
  SPEAKER_00: If you're dealing with 600 petabytes of just Git traffic, that's so much.
• 22:32
  SPEAKER_00: The cost optimization for bringing that down is like huge.
• 22:36
  SPEAKER_00: Yeah.
• 22:36
  SPEAKER_00: The value for that.
• 22:37
  SPEAKER_00: Why would you not do that?
• 22:38
  SPEAKER_00: Yeah.
• 22:39
  SPEAKER_00: For us, peons is serving well until you get to a certain point of adoption.
• 22:44
  SPEAKER_00: So I wonder what is that tipping point for us of changes per day?
• 22:49
  SPEAKER_00: What number of changes per day for you are you like, you know what?
• 22:52
  SPEAKER_00: Git is not the right tool.
• 22:54
  SPEAKER_00: A ton.
• 22:54
  SPEAKER_00: It's gotta be a lot.
• 22:56
  unknown: Yeah.
• 22:57
  SPEAKER_00: At the same time, they probably use some of the prior art as their uh go bys.
• 23:03
  SPEAKER_01: As somebody who has lived through the end of CVS and the entire lifespan of SVN, it's funny to just come to a point where it's like, oh, maybe Git isn't fast enough anymore.
• 23:12
  SPEAKER_01: Git was the most incredible thing I experienced as a developer once it came out and I compared it to SVN.
• 23:18
  SPEAKER_01: Oh, so fast.
• 23:19
  SPEAKER_01: And so much better at resolving conflicts.
• 23:21
  SPEAKER_01: We only have 90 seconds for this one for some reason that I'm I'm not sure of, because of the rules that we put in place and don't even remember putting in place.
• 23:28
  SPEAKER_01: So developers are ripping the smart out of smart TVs using AI coding assistance to do it.
• 23:33
  SPEAKER_01: So there's two open source MIT licensed projects that are surfaced this week that will strip loatware and ads and tracking off of your Android TV, your Google TV, your Fire TV.
• 23:44
  SPEAKER_01: They're called Clean Your TV and Dumb TV.
• 23:47
  SPEAKER_01: And you can check these out right now.
• 23:49
  SPEAKER_01: My question for you, Valentino, is how hard is it going to be now if you put these things in place to set up your parents' TV?
• 24:01
  SPEAKER_00: Well, you know, this is just pie hole all over again.
• 24:04
  SPEAKER_00: Yeah, I was thinking the same thing.
• 24:05
  SPEAKER_01: It's like piehole.
• 24:06
  SPEAKER_01: I remember I made a pie hole.
• 24:08
  SPEAKER_01: I was so proud of myself.
• 24:09
  SPEAKER_00: Pain in the ass.
• 24:10
  SPEAKER_00: Yeah.
• 24:10
  SPEAKER_00: You know?
• 24:10
  SPEAKER_00: Yeah, it is.
• 24:11
  SPEAKER_00: And uh I imagine this is gonna be very much a pain in the ass.
• 24:16
  SPEAKER_00: But you know, we didn't have agents back in the day.
• 24:19
  SPEAKER_00: And uh maybe pie hole is much easier to set up now.
• 24:22
  SPEAKER_00: Uh I may remember that.
• 24:24
  SPEAKER_00: That's true.
• 24:24
  SPEAKER_01: That's a good point.
• 24:25
  SPEAKER_01: I have a friend who runs an engineering team, and he said that in his spare time he developed a thing.
• 24:30
  SPEAKER_01: It's a tool that he loves to listen to Yankee games on the radio, and it's a tool that it detects when the inning is over and then automatically mutes his audio for the commercials.
• 24:40
  SPEAKER_01: It's incredible.
• 24:41
  SPEAKER_01: He's like, I never looked at any of the True Smart TV.
• 24:44
  SPEAKER_01: Yeah, yeah, yeah, yeah.
• 24:46
  SPEAKER_01: Next week on the show, we're going to have Rick Van Michellen.
• 24:51
  SPEAKER_01: He's going to talk a little bit about Museum of Modern Art.
• 24:54
  SPEAKER_01: If you're interested in being on the show, we would love to have you.
• 24:58
  SPEAKER_01: We have guests booked out through February.
• 24:59
  SPEAKER_01: Do you know that?
• 25:00
  SPEAKER_01: Wow, really?
• 25:01
  SPEAKER_01: And potentially even further.
• 25:02
  SPEAKER_01: Yeah.
• 25:02
  SPEAKER_01: So they're, I mean, it's really exciting.
• 25:04
  SPEAKER_01: Thank you, everybody, for saying yes.
• 25:06
  SPEAKER_01: So I we have somebody in the background that's not AI, that's just begging people left and right to come on the show.
• 25:11
  SPEAKER_01: So thank you for saying yes.
• 25:12
  SPEAKER_01: The week after that, we have Marty Hott for Ruby Central.
• 25:15
  SPEAKER_01: And uh let me tell you something.
• 25:16
  SPEAKER_01: Marty's had a very interesting year.
• 25:18
  SPEAKER_01: So we're gonna have a lot of fun talking to him.
• 25:21
  SPEAKER_01: So much going on in Ruby Central and Ruby Gems.
• 25:24
  SPEAKER_01: So that'll be a lot of fun as well.
• 25:26
  SPEAKER_01: Yeah, I'm looking forward to it.
• 25:27
  SPEAKER_01: All of our guests are great.
• 25:28
  SPEAKER_01: Making the show fun and interesting.
• 25:29
  SPEAKER_01: So until next week, great to have you all.
• 25:32
  SPEAKER_01: Thanks for coming.
• 25:33
  SPEAKER_01: Talk to you soon.