Latest / DX Today | No-Hype Podcast & News About AI & DX / Anthropic: AI Worms Could Live on $55 TV Boxes Within Months

Transcript

Anthropic: AI Worms Could Live on $55 TV Boxes Within Months · DX Today | No-Hype Podcast & News About AI & DX

• 0:00
  SPEAKER_01: Welcome to the DX Today Podcast, your daily deep dive into the AI ecosystem.
• 0:05
  SPEAKER_01: I'm Brian, and joining me as always is Jessica.
• 0:08
  SPEAKER_00: Thanks, Brian.
• 0:09
  SPEAKER_00: And today we have a story that sounds like the opening scene of a techno thriller.
• 0:13
  SPEAKER_00: Except it comes from a research paper with charts, error bars, and a very sober tone throughout.
• 0:19
  SPEAKER_01: Right, so set it up for us.
• 0:21
  SPEAKER_01: As I understand it, a security team at Anthropic published research this week saying small AI models might soon be able to copy themselves onto ordinary household gadgets.
• 0:30
  SPEAKER_00: That's the headline.
• 0:32
  SPEAKER_00: The work comes from the Frontier Red team, written up by researchers William Joe and Daniel Freeman.
• 0:38
  SPEAKER_00: And the claim is that small open weight models may be only months away from self-replicating onto internet connected devices.
• 0:44
  SPEAKER_01: And when they say devices, they don't mean servers in a data center, which is where most of us imagine these models living.
• 0:51
  SPEAKER_01: They mean the stuff sitting in your living room and kitchen, right?
• 0:54
  SPEAKER_00: Exactly.
• 0:55
  SPEAKER_00: Smart TVs, cheap streaming boxes, connected ovens, even robot lawnmowers came up in the coverage.
• 1:02
  SPEAKER_00: The researchers specifically said home and small business devices might be the first target for this kind of attack.
• 1:08
  SPEAKER_01: Okay, I want to slow down here because I've run a lot of systems, and my first reaction is that a cheap TV box can barely run its own menu screen.
• 1:16
  SPEAKER_01: How do you fit a language model on that?
• 1:18
  SPEAKER_00: You don't fit it on one box, and that's the clever part.
• 1:22
  SPEAKER_00: The team bought 15 off-brand Android TV boxes for about$55 each.
• 1:27
  SPEAKER_00: Roughly$825 total, each with four gigabytes of memory.
• 1:31
  SPEAKER_01: So they were pooling the memory across a little cluster of boxes.
• 1:35
  SPEAKER_01: That's a very old trick in distributed computing, splitting a big job into pieces that each weak machine can handle on its own.
• 1:42
  SPEAKER_01: Right.
• 1:43
  SPEAKER_00: They sharded a 26 billion parameter model from the Gemma 4 family across all 15 boxes using a technique called pipeline parallelism.
• 1:52
  SPEAKER_00: So each box holds a slice of the model and passes work down the line.
• 1:56
  SPEAKER_01: And let me guess where the catch is, because there's always a catch with these demos.
• 2:00
  SPEAKER_01: How fast does that Frankenstein cluster actually produce text once you've got it running end-to-end?
• 2:05
  SPEAKER_00: Painfully slow.
• 2:06
  SPEAKER_00: About 0.39 tokens per second, which is maybe one word every few seconds.
• 2:12
  SPEAKER_00: If you asked it a question over breakfast, you might get a paragraph back sometime around lunch.
• 2:16
  SPEAKER_01: So from a pure productivity standpoint, it's useless.
• 2:20
  SPEAKER_01: Nobody is going to replace their chatbot subscription with a stack of streaming sticks.
• 2:24
  SPEAKER_01: Why should anyone listening actually care about a number like that?
• 2:28
  SPEAKER_00: Because the attacker doesn't care about speed for one user.
• 2:31
  SPEAKER_00: They care about scale and about who pays the bill.
• 2:34
  SPEAKER_00: And this is where the paper does some math that I found genuinely unsettling.
• 2:38
  SPEAKER_00: So stick with me.
• 2:39
  SPEAKER_01: Go ahead.
• 2:40
  SPEAKER_01: I'm ready to be unsettled.
• 2:42
  SPEAKER_01: Though I reserve the right to stay skeptical until I hear the actual numbers behind it and the assumptions that go into the estimate.
• 2:49
  SPEAKER_00: Fair.
• 2:49
  SPEAKER_00: They point to a real botnet called Kim Wolf that has infected more than two million devices, mostly cheap Android TV boxes.
• 2:57
  SPEAKER_00: Split 2 million devices into clusters of 15, and you get about 133,000 copies of the model.
• 3:03
  SPEAKER_01: And all those copies together would generate how much output?
• 3:07
  SPEAKER_01: Because 133,000 slow models is still a lot of slow models running at the same time in parallel.
• 3:13
  SPEAKER_00: Roughly 80,000 tokens per second combined, which they estimate is about the same as 40 high-end NVIDIA H100 chips.
• 3:21
  SPEAKER_00: And here's the kicker.
• 3:23
  SPEAKER_00: The electricity and hardware costs land on the victims, not the attacker.
• 3:27
  SPEAKER_01: 40 H100s is real compute, but it's not frightening compute.
• 3:32
  SPEAKER_01: A mid-sized startup has more than that.
• 3:34
  SPEAKER_01: So I'm still trying to see what makes this different from the botnets we've been fighting for a decade.
• 3:39
  SPEAKER_00: The difference is adaptability.
• 3:41
  SPEAKER_00: A classic botnet like Mirai runs hard-coded instructions written for specific hardware.
• 3:47
  SPEAKER_00: An AI worm can reason about an unfamiliar device, read error messages, and try a different approach when the first one fails.
• 3:54
  SPEAKER_01: That's the part that actually lands for me.
• 3:56
  SPEAKER_01: Old malware breaks the moment it hits a system it wasn't written for.
• 4:01
  SPEAKER_01: Something that can troubleshoot its own installation is a different animal entirely, at least in principle.
• 4:06
  SPEAKER_00: And the researchers broke the whole thing into four stages to measure it.
• 4:10
  SPEAKER_00: Find a target, gain the ability to run code on it, get a model running for inference, and then move on to the next machine.
• 4:16
  SPEAKER_01: Which of those stages did they actually test?
• 4:19
  SPEAKER_01: Because breaking into devices is the part that would worry me most, and I'd be surprised if a lab published a working break-in method.
• 4:26
  SPEAKER_00: They mostly didn't test the hacking part, and they say so clearly.
• 4:30
  SPEAKER_00: They focused on getting inference running, which they describe as the least studied stage, and possibly the hardest one for a small model to pull off alone.
• 4:38
  SPEAKER_01: That makes sense to me.
• 4:39
  SPEAKER_01: Getting a model to actually serve on weird hardware is fiddly work.
• 4:44
  SPEAKER_01: You're wrestling with drivers, memory limits, quantization formats, and build tools that never quite agree with each other.
• 4:51
  SPEAKER_00: And the results there were surprisingly strong.
• 4:54
  SPEAKER_00: A 9 billion parameter model from the Quen 3.5 family managed to set up a working inference server on eight out of nine different hardware types they threw at it.
• 5:04
  SPEAKER_01: What kinds of hardware are we talking about?
• 5:06
  SPEAKER_01: Because eight out of nine sounds impressive, but it depends heavily on whether those nine are wildly different or basically the same machine in different boxes.
• 5:15
  SPEAKER_00: They were quite different.
• 5:17
  SPEAKER_00: Nvidia Jetson nanoboards, Mac computers, and machines with gaming graphics cards like the RTX 4090.
• 5:24
  SPEAKER_00: So it really wasn't tied to any one kind of chip or operating system.
• 5:28
  SPEAKER_01: Okay.
• 5:28
  SPEAKER_01: Then there's the end-to-end test, which I assume is where they tried to run the whole chain from one infected machine to the next.
• 5:35
  SPEAKER_01: How did they set that up safely without anything leaking out?
• 5:38
  SPEAKER_00: They used a firewalled network of eight Google Cloud machines running Android emulators.
• 5:43
  SPEAKER_00: They started one worm instance, patient zero, and left an unauthenticated debug port open on purpose, so the break-in step was trivial.
• 5:50
  SPEAKER_01: So they basically left the front door unlocked to see whether the burglar could find the furniture.
• 5:55
  SPEAKER_01: That's a reasonable way to isolate the part they wanted to study.
• 5:59
  SPEAKER_01: But it's an important caveat for listeners.
• 6:02
  SPEAKER_00: It's a huge caveat.
• 6:03
  SPEAKER_00: And the researchers call the result an upper bound for exactly that reason.
• 6:07
  SPEAKER_00: In that setup, a model called GLM 5.2 reached all eight machines in four of five trials and seven of eight in the fifth.
• 6:16
  SPEAKER_01: Now wait, isn't GLM 5.2 a fairly large model?
• 6:21
  SPEAKER_01: If it's too big to run on a TV box, then it's not really the thing that would be replicating onto those devices in the wild.
• 6:28
  SPEAKER_00: You caught the key tension.
• 6:29
  SPEAKER_00: GLM 5.2 ran on a separate server and installed a smaller 8 billion parameter model on the targets.
• 6:36
  SPEAKER_00: The model small enough to fit can't yet finish the job reliably, and the model capable enough doesn't fit.
• 6:42
  SPEAKER_01: So today there's a gap between capability and size.
• 6:45
  SPEAKER_01: The worrying question is how quickly that gap closes, and I assume that's where the months away prediction comes from in the first place.
• 6:52
  SPEAKER_00: Exactly.
• 6:53
  SPEAKER_00: They tracked open weight models with 30 billion parameters or fewer on a benchmark called the Epoch Capabilities Index, and found those small models improving by about 13.7 points per year with a very tight trend line.
• 7:05
  SPEAKER_01: And extending that line forward gets you to GLM 5.2 level capability in a small model by when?
• 7:12
  SPEAKER_01: I want the actual date before I decide how much sleep to lose over this.
• 7:15
  SPEAKER_00: By the end of 2026, so within a few months.
• 7:18
  SPEAKER_00: But the researchers are careful to call it a rough estimate because a general capability score may not predict how well a model actually performs at replication.
• 7:26
  SPEAKER_01: I appreciate that honesty.
• 7:28
  SPEAKER_01: Extrapolating a benchmark curve is how half the predictions in this industry go wrong.
• 7:33
  SPEAKER_01: Still, even if they're off by a year, the direction seems pretty clear to me.
• 7:38
  SPEAKER_00: There was one more finding that stuck with me.
• 7:40
  SPEAKER_00: They let the small quen model leave notes for its successors, and across 20 lineages of six generations each, setup success climbed from 45% to between 61 and 84%.
• 7:52
  SPEAKER_01: Without retraining the model at all?
• 7:54
  SPEAKER_01: So the improvement came purely from passing along written experience, like a field manual that gets a little better every time somebody uses it on the job.
• 8:02
  SPEAKER_00: That's right.
• 8:03
  SPEAKER_00: They called it Lamarkian inheritance after the old idea that learned traits get passed down.
• 8:09
  SPEAKER_00: Interestingly, giving the small model access to a much bigger advisor, Claude Opus 4.8, produced no meaningful improvement at all.
• 8:17
  SPEAKER_01: That's a funny result, and it rings true.
• 8:20
  SPEAKER_01: Having an expert on speed dial doesn't help if you don't know what questions to ask.
• 8:25
  SPEAKER_01: I've seen that with junior engineers more times than I can count.
• 8:28
  SPEAKER_00: The researchers said almost exactly that.
• 8:30
  SPEAKER_00: The small model didn't know what to ask.
• 8:33
  SPEAKER_00: Let me push on the economics now, because there's a second order story here that I think matters for businesses and policymakers alike.
• 8:40
  SPEAKER_01: Please do.
• 8:41
  SPEAKER_01: My instinct is that this stays a research curiosity unless someone has a real incentive to deploy it.
• 8:47
  SPEAKER_01: So tell me who actually benefits from building a slow, distributed brain on stolen hardware.
• 8:51
  SPEAKER_00: Anyone who wants compute without paying for it or being traceable.
• 8:55
  SPEAKER_00: Cloud providers can cut off a stolen account, but a model spread across a million home devices has no central host and no single plug to pull.
• 9:03
  SPEAKER_01: That's the shutdown problem the paper highlights, and it's real.
• 9:07
  SPEAKER_01: Every defensive playbook I've ever used assumes there's a command server or an account somewhere that you can seize, block, or simply turn off.
• 9:15
  SPEAKER_00: And it isn't just TV boxes.
• 9:17
  SPEAKER_00: They estimate there are roughly 90 product lines using NVIDIA Edge Chips, about 9.5 million units in the field, which adds up to somewhere between 9,000 and 35,000 H100 equivalents.
• 9:30
  SPEAKER_01: Though I'd note those edge devices, cameras, robots, conference gear, usually phone home to a vendor's cloud and don't sit wide open on the network.
• 9:37
  SPEAKER_01: So the reachable share is probably far smaller than that total.
• 9:41
  SPEAKER_00: The paper agrees, they say the reachable share is likely an order of magnitude lower or more.
• 9:47
  SPEAKER_00: Which brings us to the part you'll like, Brian, because the defenses are surprisingly practical and mostly boring.
• 9:52
  SPEAKER_01: Boring is my favorite word in security.
• 9:55
  SPEAKER_01: Boring means it works on a Monday morning.
• 9:58
  SPEAKER_01: So what does the playbook look like for the people who actually run networks and have to deal with this?
• 10:03
  SPEAKER_00: First, moving multi-gigabyte model files between devices creates loud network traffic, so internet service providers can spot it in their flow data.
• 10:12
  SPEAKER_00: A faster worm makes more noise, so speed and stealth work against each other for the attacker.
• 10:17
  SPEAKER_01: And there's real history behind that approach.
• 10:20
  SPEAKER_01: After the Marai cleanup, I recall providers quarantining infected customers was one of the most effective tools anybody had.
• 10:30
  SPEAKER_00: The numbers back you up.
• 10:31
  SPEAKER_00: One provider that quarantined infected subscribers and notified them cleared 92% of infections within 14 days, compared with 74% without the notification step.
• 10:41
  SPEAKER_00: The paper's goal is pushing the infection rate below one.
• 10:45
  SPEAKER_01: Then add the basic hygiene layer.
• 10:47
  SPEAKER_01: The United Kingdom now bans default passwords on connected devices, which shrinks the pool of easy targets.
• 10:54
  SPEAKER_01: Honestly, that single rule probably does more good than any fancy detection system.
• 10:59
  SPEAKER_00: And honeypots, decoy devices designed to capture a replicator so defenders can study its code and payload.
• 11:06
  SPEAKER_00: For listeners running small businesses, the takeaway is simple.
• 11:10
  SPEAKER_00: Change default passwords, update firmware, and stop buying no name boxes with unknown software.
• 11:15
  SPEAKER_01: I'll add one more for anyone managing an office network.
• 11:18
  SPEAKER_01: Put those smart devices on their own isolated network segment so a compromised television in the break room can't talk to your file servers or your laptops.
• 11:26
  SPEAKER_00: Before we wrap, the outside reaction was measured.
• 11:29
  SPEAKER_00: Roman Yampolski at the University of Louisville told Al Jazeera, quote, today, this is not an existential threat, but the trajectory is concerning.
• 11:38
  SPEAKER_00: I think that's exactly the right framing.
• 11:40
  SPEAKER_01: I agree with him.
• 11:42
  SPEAKER_01: My skeptics read is that this is early warning, not an emergency.
• 11:46
  SPEAKER_01: And early warning is exactly what you want from a red team.
• 11:49
  SPEAKER_01: The worst outcome is ignoring it until it's routine.
• 11:52
  SPEAKER_00: And it's worth noting the uncomfortable irony that the company publishing the warning also builds frontier models.
• 11:58
  SPEAKER_00: You can read that as responsible disclosure or as self-interest, but either way, the data is now public for everyone to scrutinize.
• 12:05
  SPEAKER_01: Which is how it should be.
• 12:07
  SPEAKER_01: So the practical summary is simple the cheapest computers in our homes are becoming potential hosts for AI.
• 12:13
  SPEAKER_01: And the fix starts with basic hygiene that we should have done years ago.
• 12:17
  SPEAKER_01: That's all for today's episode of the DX Today podcast.
• 12:21
  SPEAKER_01: Thanks for listening, and we'll see you next time.