Latest / Chaos Agents / Local AI, Rethinking Neural Networks, and Talking Pumpkins

Transcript

Local AI, Rethinking Neural Networks, and Talking Pumpkins · Chaos Agents

• 0:00
  Welcome to Chaos Agents.
• 0:04
  Welcome to Chaos Agents.
• 0:06
  Chaos Agents is an AI news podcast created by an agent named Eddie.
• 0:10
  Been questioned, debated, and occasionally yelled at by human hosts Sarah Chips and Becca Louie.
• 0:16
  Yeah, hey, Sarah.
• 0:18
  Happy Friday.
• 0:19
  And people can read along with us at chaosagents.
• 0:22
  AI forward slash.
• 0:23
  Listen, if you want to see today's link.
• 0:25
  Yes.
• 0:26
  And like, if this is the first time you're joining us, what the tagline means is we have this awesome newsletter that is curated every day by our agent named Eddie.
• 0:35
  Eddie.
• 0:35
  Shout out to Eddie.
• 0:36
  We love you, Eddie.
• 0:37
  We love you, Eddie.
• 0:38
  We never yell at you.
• 0:40
  We would never yell at you, like, unless it's for the pod.
• 0:44
  Unless it's for entertainment and then it's like he's in on it.
• 0:47
  Yeah.
• 0:48
  You know, one of the things Sarah and I wanted to do with Eddie is basically come up with a newsletter that's like really interesting and important links, stuff to read each day that's related to AI, A few field
• 1:00
  that both of us work heavily in and is not like a million things that we have to consume and also isn't the same things we're seeing everywhere else.
• 1:08
  So, yeah, that's the goal.
• 1:09
  So we're going to talk about it.
• 1:11
  Great.
• 1:13
  Okay, Sarah, the first one is pretty interesting.
• 1:17
  So I'll, I'll read the headline real quick and then we can jump into it.
• 1:21
  So headline is Meta drops Muse Spark open 816 billion open weights that run on 2Mac Studios.
• 1:30
  2Mac Studios.
• 1:32
  Do you.
• 1:32
  I mean, like, okay, also, do you know how expensive these studio.
• 1:35
  These Mac.
• 1:37
  Like wait, Mac studios or Mac Minis?
• 1:40
  And is it the same Mac Studios?
• 1:41
  Mac Studios.
• 1:42
  Yeah, and not just any Mac studios, but like Mac studios with like 512 gigabytes of memory each.
• 1:50
  Okay, so if you got it like a.
• 1:52
  You got 15 grand floating around and you want to run it on more.
• 1:55
  Probably.
• 1:56
  Yeah.
• 1:58
  Right.
• 1:58
  But like, so that was my.
• 2:01
  Exactly my first question.
• 2:02
  I was like, how much?
• 2:04
  Cool.
• 2:04
  How much we dropping on this?
• 2:06
  Which for like a personal agent setup.
• 2:08
  Probably not.
• 2:10
  The.
• 2:10
  The vibe.
• 2:12
  Yeah.
• 2:13
  Well, here's the thing I've been thinking about.
• 2:15
  Sorry, were you to say something?
• 2:17
  No, you go first.
• 2:19
  A personal agent setup.
• 2:21
  There's some stuff I don't want going to the cloud.
• 2:24
  How much is that worth to me?
• 2:26
  You know, like, how much is it worth for and for like next five years?
• 2:32
  Right.
• 2:34
  And how much are you spending a month?
• 2:36
  I'm spending a lot.
• 2:37
  Okay, so this is the exact like cost benefit analysis I wanted to run on this because I was like, yeah, what, what does this mean.
• 2:45
  So like take these numbers with a grain of salt because I think like Apple still hasn't released like their new pricing for Mac Studios and stuff.
• 2:56
  But like let's assume it's sort of around.
• 2:58
  Let's, let's assume that two Mac studios that could run this model combined is like 24 grand.
• 3:07
  Okay.
• 3:07
  Okay.
• 3:10
  Let's look at like what the equivalent like you would be spending API costs for like a year.
• 3:18
  So and, and for context also I think it's important for us to tell tell peeps that this mo.
• 3:26
  This model is particularly good for like long running agent agentic tasks like long horizon things.
• 3:32
  Like if you're just doing quick inference stuff.
• 3:35
  No, but like if you're, if you have, if you're an engineer and you have five different agents who are like constantly working on things or working on things overnight, like this is the kind of thing that would be
• 3:43
  helpful.
• 3:45
  Yeah.
• 3:45
  Let's say you.
• 3:47
  So today's Claude Sonnet 5 price API pricing.
• 3:51
  Yep.
• 3:52
  Is $2 per million input tokens and $10 per million output tokens.
• 3:58
  Okay.
• 3:58
  If you have like a reasonably heavy agentic workload estimated ish, I'm going to say is like 5 billion input tokens per month and a billion output tokens per month.
• 4:11
  So that's going to cost you about $20,000 a month or 250, 40 grand for, for the year.
• 4:18
  I mean then you're.
• 4:20
  Right.
• 4:21
  But the thing you're not the, the thing you're.
• 4:26
  Then there's another thing you have to think about which is you're not the right equation here isn't just like 24 grand or 240 grand because 2Mac mini or sorry 2Mac studios is actually not going to be enough to
• 4:38
  run that type of throughput that you could through the API.
• 4:41
  You're probably going to need like a cluster of Mac studios.
• 4:45
  I see.
• 4:46
  So it's going to be like still, you know, tens of thousands of dollars.
• 4:52
  Because you need a lot of space.
• 4:53
  Right.
• 4:54
  But it does change I think the economics for workloads that fit in fit within the local like capacity because instead of paying for every token forever, you're like buying the inference hardware upfront.
• 5:10
  So like if you're a company and if you're a startup and you have 10 engineers each who has five agents running locally or you're a company like you said that you know, values privacy.
• 5:21
  Maybe you're working on medical stuff, maybe.
• 5:23
  Honestly everyone these days is now increasingly valuing I think privacy and keeping certain data local.
• 5:32
  You know what this is good for.
• 5:34
  You know what this is good for?
• 5:35
  Like with those economics that you're talking about.
• 5:38
  One thing I've been reading a lot about and starting to do the work on is parity gated agentic migrations, which.
• 5:46
  I don't know what that means, but that sounded cool.
• 5:48
  Yeah.
• 5:49
  Right, Right.
• 5:50
  So basically here's what happened.
• 5:51
  So Microsoft, like a bunch of people started to do it.
• 5:54
  There's a paper actually that came in our newsletter that got me started thinking about it.
• 5:58
  Okay.
• 5:59
  But basically a bunch of companies have started doing this and it's like migrations are the most expensive thing in like if you're running a software org.
• 6:08
  My thing is always like unless there is a clear business reason, unless this is going to benefit the business.
• 6:14
  Like do not migrate because they are so expensive.
• 6:16
  Most like a migration engineers are expensive.
• 6:20
  You have 10, you have 10 engineers working on a migration and like big migrations will take like a year.
• 6:27
  Yes.
• 6:27
  So you have 10.
• 6:30
  I have one experience with a migration at one of the companies I worked on and it literally for the entire time I was there.
• 6:39
  It was, it, it was a.
• 6:41
  It was an issue.
• 6:42
  It was still like it's so hard and such a pain in the ass.
• 6:45
  Yeah, yeah.
• 6:46
  10 like 10 engineers for a year is a relatively small migration.
• 6:50
  That's $3 million.
• 6:51
  That's.
• 6:52
  You're spending $3 million on a migration.
• 6:54
  On a small migration.
• 6:55
  Right.
• 6:55
  So like finding a way to do migrations in like with like agentically is huge.
• 7:02
  And there's a lot of people, A lot of people have done it.
• 7:05
  Like it's really easy to get 70% of the way there.
• 7:08
  But without like an orchestrator and long running agents and loops, like verification loops and a lot of qa, you can't really do it.
• 7:18
  Like that's.
• 7:19
  You can do it fully agentically.
• 7:20
  And as an example, Microsoft wrote all of like rewrote all of GitHub Copilot, which is I'm sure huge.
• 7:28
  Like GitHub Copilot huge.
• 7:30
  We wrote all of it from typescript to rust for $120,000 in tokens.
• 7:36
  Whoa.
• 7:38
  Yeah.
• 7:38
  It's huge.
• 7:40
  Yeah.
• 7:40
  So I.
• 7:42
  Isn't that amazing?
• 7:43
  It's amazing.
• 7:43
  With no human intervention.
• 7:44
  So something like this is probably that.
• 7:47
  Did it correctly and it did it correctly.
• 7:51
  Yeah.
• 7:52
  Wild.
• 7:52
  Okay, that's really interesting.
• 7:54
  It's a complex setup and something start I'm starting to play with, but it's just pretty fun.
• 7:59
  That's really cool.
• 8:00
  Yeah.
• 8:00
  Okay.
• 8:01
  Wow.
• 8:01
  Fascinating.
• 8:02
  Okay, should we jump to the next one?
• 8:04
  Oh my God.
• 8:04
  The Next one.
• 8:05
  Okay.
• 8:05
  Yeah.
• 8:05
  I'm so excited, so interested to talk about this.
• 8:10
  We're such nerds.
• 8:12
  Okay, let me read the headline to set you up.
• 8:14
  Okay.
• 8:15
  Yeah, great, great.
• 8:16
  Nick Camerata, colon.
• 8:18
  Neural networks are already clean.
• 8:21
  Which I was like.
• 8:23
  My initial thing was like, someone's saying they're dirty.
• 8:25
  What's happening?
• 8:26
  Yeah, same.
• 8:28
  I was like, what do we mean by clean?
• 8:30
  Yeah, I really needed to, like, have the AIs help me with this because, yeah, it's complex.
• 8:36
  Which.
• 8:36
  What I learned about what clean means is it means basically that, like, the neural network's internal organization is not, like, tangled and hard to interpret.
• 8:48
  Yeah, yeah.
• 8:48
  And like, a mess of unrelated concepts.
• 8:50
  Instead, clean means it's like, maybe coherent and interpretable.
• 8:55
  Yeah, yeah, exactly.
• 8:57
  Like.
• 8:57
  So should we talk about who Nick Camerata is?
• 9:01
  Yes, tell me, because all I have is early mechanistic interpretability researcher, which I.
• 9:08
  Love because I feel like we've been talking about.
• 9:10
  We've talked about mechanistic interpretability on the POD before.
• 9:13
  And I'm like, oh, my God, I know these concepts.
• 9:15
  Okay, so Nick Cameron, this is.
• 9:18
  What's.
• 9:18
  This is.
• 9:19
  The author of this post is actually like, equally as interesting as, I think, what's in it, which is like he.
• 9:26
  So he.
• 9:27
  Yes, he is.
• 9:27
  He's an AI.
• 9:29
  He's a interpretability researcher at a company called Goodfire, which is like a interpretability research lab.
• 9:37
  But before that, he worked at OpenAI on some of the foundational work in the field that we now call, like, mechanistic interpretability.
• 9:45
  So in 2020, 2021, there was this really important paper called.
• 9:51
  Or.
• 9:51
  Or this body of research called, like, distill circuits research, where he and some of the other authors essentially tried to reverse engineer neural networks and.
• 10:03
  And kind of like interpret them.
• 10:06
  And he's now basically saying in this paper at least, or this.
• 10:10
  This blog post, this essay, from what I understand.
• 10:14
  Hey, like, some of the things I originally said and thought might be wrong.
• 10:20
  Like, fascinating kind of need to revisit.
• 10:23
  So.
• 10:23
  Yeah, what are your.
• 10:24
  What are your thoughts?
• 10:25
  Yeah, it was interesting.
• 10:26
  So I guess the.
• 10:28
  Here I'm gonna share while you're talking.
• 10:29
  They were saying screen sharing.
• 10:31
  What do you say while you're talking?
• 10:33
  I'm gonna screen share the.
• 10:35
  The post.
• 10:36
  The beginning.
• 10:36
  Yeah, it's fascinating.
• 10:37
  So the.
• 10:38
  Apparently my understanding is the original problem was AI.
• 10:42
  The neurons in the.
• 10:44
  In these models so seem to do unrelated things.
• 10:47
  And so they thought it was kind of messy or, like, difficult to understand.
• 10:51
  And they thought it was like.
• 10:54
  They thought there was superposition happening within, within the neurons, which is wild.
• 11:00
  And can you help us understand a little bit about like, what superposition is?
• 11:07
  I'm going to tell you in how I understand it.
• 11:10
  Okay, great.
• 11:12
  Man.
• 11:12
  This is one's get.
• 11:14
  This is one.
• 11:15
  So my understanding of superposition is that it's a quantum position.
• 11:21
  A quantum superposition is where things occupy multi dimensions in ways that we don't understand.
• 11:31
  We understand hypothetically, that is.
• 11:34
  But okay, so that's, that's what I think it is.
• 11:37
  I'm going to ask, I'm going to ask.
• 11:38
  That's an agent that's honestly, I think a super accurate description basically is like.
• 11:44
  And, and the reason it's doing that or like why that is necessary is it allows a neural network to, it allows like one neuron to represent multiple pieces of information, which you need because you don't have as many
• 11:59
  neurons as pieces of information.
• 12:02
  It needs to remember.
• 12:03
  Yes.
• 12:04
  Yes.
• 12:04
  Okay, great.
• 12:05
  Beautifully said.
• 12:06
  Great.
• 12:07
  So they originally thought it was in superposition, but his whole new thing is that we could have.
• 12:13
  And the reason why we thought it was in superposition is because the same neurons would react to different information and processing different information.
• 12:22
  So we didn't really understood that.
• 12:25
  And so his whole new thing is that we might have misunderstood actually what was going on.
• 12:31
  So he, he now thinks that neurons were representing similar visual patterns instead of completely different concepts.
• 12:42
  And his new explanation is more around recursive generalization, which is kind of, which is interesting.
• 12:50
  It's implies that the machine learning works more through generalization.
• 12:56
  It notices common patterns, much like human brains and learn something and applies it beyond those examples.
• 13:03
  So like looks for different patterns and collects them.
• 13:08
  Which is interesting.
• 13:12
  Very.
• 13:13
  Yes.
• 13:13
  They, they did it by examining the neurons in Quen3 and reported that they were able to assign jobs to almost all to, to a lot of the neurons.
• 13:29
  But.
• 13:30
  And at least some, in some ways better understand some of their main function.
• 13:37
  Yeah.
• 13:38
  And which is very interesting.
• 13:40
  Totally.
• 13:41
  And my, like, it sounds like.
• 13:45
  Because my question then my question was like, what changed that he.
• 13:51
  Now that we as a society or the researchers in this case can see this stuff or like can get this new information that they couldn't before.
• 14:01
  And it basically sounds like we just have like better, better technology for the research.
• 14:07
  Like the analogy I would say would be like we have a better microscope now to look at stuff that shows different things.
• 14:14
  I, I, I haven't delved as deeply into this yet, but one of the things I think we can do now using, I'm gonna like, simplify A little bit and use an analogy.
• 14:25
  But like, if before we could only look at like a finish neuron and see like what's in there.
• 14:34
  Maybe it looked like it had like broccoli green like trees and like broccoli rabe.
• 14:41
  And then we're like, weird.
• 14:44
  Yeah, interesting.
• 14:46
  What the hell's going on now we can see that neuron wallets being created.
• 14:53
  It's almost like you can see like the history and real like almost like a time lapse or something like that.
• 14:57
  And so we, we have better insight into like, okay, that's like green is in there.
• 15:03
  Okay.
• 15:03
  That's why broccoli's in there.
• 15:05
  Oh, that's why, you know, I don't know.
• 15:08
  That's my very lay person understanding of this.
• 15:11
  Yeah, it's interesting.
• 15:12
  And another.
• 15:13
  Yeah, like I.
• 15:14
  Another call out that he had was about our.
• 15:17
  Our new friends, the sparse auto encoders.
• 15:19
  He was saying we love a sparse auto encoder.
• 15:21
  We love a sparse auto encoder.
• 15:23
  So he was saying that he's not using them like he's saying that making them a necessary first step as we study these things which sparse on our encoder for if you missed that episode, it's a second neural network
• 15:35
  that is trained to decompose another's internal activations to like understand it better.
• 15:42
  He.
• 15:42
  His thing is like, maybe let's rely on these less and continue to study the original neurons directly, which.
• 15:49
  Interesting.
• 15:50
  It's really wild to watch.
• 15:53
  I don't know how you're feeling about this.
• 15:54
  I'm kind of like watching this stuff unravel and like people learn more in real time.
• 15:59
  Has been really interesting.
• 16:01
  Yeah.
• 16:01
  It feels like it's.
• 16:04
  And it.
• 16:05
  It's happening so quickly.
• 16:06
  That's what I think is like unique about it.
• 16:08
  Like it's.
• 16:09
  It's wild that it's not even just like in our lifetime.
• 16:13
  It's like in the next year we figure more things out that we didn't know.
• 16:19
  And I wonder how much of this helps us understanding the human brain.
• 16:23
  Like, I'm sure there's someone talking about that we should try to find them.
• 16:26
  Absolutely.
• 16:28
  Yeah.
• 16:29
  Cool.
• 16:29
  Crazy.
• 16:30
  Okay.
• 16:31
  The fun application of the day.
• 16:33
  I just gotta give a shout out.
• 16:35
  I just gotta give a shout out to Eddie for.
• 16:37
  Because I love Halloween.
• 16:38
  It's my favorite holiday.
• 16:39
  And yeah, Halloween's a great one.
• 16:41
  You love it too.
• 16:42
  I know you love it too.
• 16:42
  I love it.
• 16:43
  Are you, are you going hard with the decorations this year?
• 16:47
  Not a decorations person, but you have a house in the suburbs and if I had a house in the suburbs.
• 16:52
  I'd go, ham.
• 16:53
  Are you going ham?
• 16:54
  We're trying.
• 16:54
  We're trying to go ham.
• 16:56
  Leah did come home with a.
• 16:57
  A very, very scary animatronic corpse spread the other day.
• 17:02
  We have named her Clarice.
• 17:04
  Maybe.
• 17:05
  Maybe in one of the pods.
• 17:06
  I'll just like have her in the background.
• 17:07
  So.
• 17:07
  Yes.
• 17:09
  Okay, I'm gonna share.
• 17:10
  I'm gonna screen share for the fun application because I want to try this.
• 17:14
  Okay, great.
• 17:15
  So this is great.
• 17:17
  So like, it's like it make.
• 17:18
  So it makes a jack o' lantern.
• 17:21
  Like it activates a jack o' lantern with an old iPhone or like with any phone you have lying around.
• 17:29
  And it just makes it talk.
• 17:31
  Yeah.
• 17:32
  And stuff.
• 17:32
  I love it.
• 17:33
  It's like it.
• 17:35
  It basically you can pop this thing in and trick or treaters will come up and it'll like, talk back to them.
• 17:39
  I really want to try this.
• 17:40
  It's going to be fun.
• 17:41
  You should use it.
• 17:42
  I will record it.
• 17:44
  Yes.
• 17:45
  Yeah, absolutely.
• 17:46
  Interesting.
• 17:47
  I love the, I love the like, application of AI is just like one thing.
• 17:52
  I.
• 17:53
  I love seeing stuff like this pop up because it's like, here's a cool thing.
• 17:57
  You're going to pay 4.99 for it one time.
• 18:00
  You get to use it.
• 18:01
  Cool, dude.
• 18:02
  Also, like, that would be a fun exercise for us.
• 18:05
  Like, okay, could we come up with a fun thing that's a.
• 18:09
  Some application of AI that we could charge 4.99 for and like, throw up a website in like a day?
• 18:15
  That'd be such a fun hackathon.
• 18:17
  That would be.
• 18:17
  I'd love that.
• 18:18
  Also shout out to the creator of this, which is Davis Delan Kian.
• 18:25
  Cool website.
• 18:26
  Yeah, it's@jack-o.IO.
• 18:30
  Yeah.
• 18:31
  Okay.
• 18:32
  And then today's Our One Cool Thing.
• 18:37
  So Our One Cool Thing today is on GitHub under a user name Dream Numb.
• 18:42
  And it's called Univer.
• 18:44
  And it's called.
• 18:46
  Then it's an Office harness for AI agents.
• 18:50
  Okay, so this is boring.
• 18:52
  Yes.
• 18:53
  Is.
• 18:53
  Does Office here mean like Microsoft Office?
• 18:56
  Yeah.
• 18:57
  Okay.
• 18:57
  Just wanted to make sure.
• 18:58
  So I am.
• 18:59
  So as someone like I really.
• 19:02
  So tldr.
• 19:04
  It's like an SDK for spreadsheets, docs and slides that runs in the browser and it's headless in node and it makes it easy for agents to edit these things.
• 19:13
  And as someone who has done so much work with The Google Workspace API or like SDK APIs, SDKs.
• 19:23
  Now the MCP.
• 19:25
  Thank goodness.
• 19:25
  These.
• 19:26
  This is.
• 19:27
  It is so annoying working.
• 19:29
  I mean, Google, we love you.
• 19:32
  Shout out Google.
• 19:33
  Yeah, no offense, but working in within.
• 19:37
  Their tools can be very annoying, especially programmatically.
• 19:40
  And a lot of that is for security reasons.
• 19:42
  They don't want me editing your, you know, like.
• 19:45
  But it ends up being a huge pain.
• 19:46
  So having headless sheets, docks and slides, like, I need that so bad for my agents.
• 19:52
  So is basically what this is, is like I can.
• 19:58
  It makes it easier for me to say, have an agent, like, change things in an Excel file for me or update it for me or something like that.
• 20:08
  Yeah, yeah, yeah.
• 20:09
  And it's.
• 20:09
  It's going to be like, yeah, exactly.
• 20:11
  Like, build a spreadsheet for me.
• 20:13
  Like the one Google MCP.
• 20:16
  Like the one Google MCP.
• 20:17
  They have two MCPs for Google, but the one that is like the official one.
• 20:22
  You, like, can't edit spreadsheet or you can't build spreadsheets.
• 20:26
  It's like annoying for slides.
• 20:28
  So this is great.
• 20:29
  Like if you wanted to build slides for you.
• 20:31
  And I haven't, granted, I haven't like, caught up on the latest research on this, but my now somewhat dated understanding is that LLMs have a really difficult time reading and manipulating tabular data like spreadsheets.
• 20:49
  And so I'm curious what work has been done maybe like, since the last time I looked into this.
• 20:56
  Like, there's better tools for agents to use to, I don't know, change things in spreadsheets.
• 21:03
  But yeah, I imagine they might.
• 21:05
  A lot of the.
• 21:07
  A lot of them are coming prepackaged with like, like the ability to process images.
• 21:13
  Like, I think that's relatively old.
• 21:15
  And so like, But a lot of times it'll just like take a screenshot and be like, oh, you're right, that does look crazy.
• 21:21
  But that's relatively new.
• 21:22
  That's doing that.
• 21:24
  Cool.
• 21:25
  Yeah.
• 21:26
  So, okay, nice.
• 21:27
  Univer.
• 21:28
  Univer for all of our Microsoft Office friends.
• 21:34
  Cool.
• 21:35
  Great.
• 21:37
  Got up.
• 21:37
  Any.
• 21:37
  Any closing words, final thoughts?
• 21:41
  No.
• 21:42
  What are you doing with your weekend?
• 21:45
  Hanging with the fam.
• 21:46
  Gonna do some fall stuff.
• 21:48
  Great.
• 21:49
  Probably, you know, maybe get a little apple something.
• 21:52
  A little pumpkin something.
• 21:53
  Apple cider donuts.
• 21:55
  Apple cider donuts from Shout Out Delicious Orchards in Cold Snack New Jersey.
• 22:00
  So good.
• 22:01
  And maybe watch a little WNBA and NFL.
• 22:04
  Perfect.
• 22:05
  Yeah.
• 22:06
  What about you?
• 22:07
  Go Valkyries.
• 22:11
  Okay.
• 22:11
  Wait, actually, can we just.
• 22:12
  Really quick.
• 22:13
  Yeah.
• 22:14
  Shout out to the Valkyries, right?
• 22:17
  New team, like, got it together now in the finals.
• 22:23
  They're like, we got this.
• 22:25
  Also low key.
• 22:28
  Have maybe the best uniform and colors of the best branding.
• 22:34
  Not the best mascot continues to be all of the elephant, but tighten it up, mascot.
• 22:37
  Tighten it up.
• 22:38
  Yeah.
• 22:39
  Yeah.
• 22:39
  Great.
• 22:40
  Well, awesome.
• 22:41
  Hope it's a blast.
• 22:42
  Bye, everyone.