Latest / Ctrl AI Profit / Ep. 143 | China Just Stole 29 Million AI Conversations
Transcript
- 0:20SPEAKER_00: Frank, I want to talk about something that got almost no coverage in mainstream business press, but it might be the most important AI security story of the year.
- 0:30SPEAKER_01: Anthropic formally accused Alibaba's Quinn AI lab of running what they're calling the largest known distillation attack against their Claude model.
- 0:39SPEAKER_01: Distillation attack.
- 0:41SPEAKER_01: What does that mean?
- 0:42SPEAKER_01: In normal AI development, distillation is legitimate.
- 0:46SPEAKER_01: You take outputs from a capable model and use them to train a different model to behave similarly.
- 0:51SPEAKER_01: But when you do it without permission, at industrial scale, with fake accounts designed to evade detection, that's IP theft.
- 0:58SPEAKER_01: Okay.
- 0:59SPEAKER_01: And the scale here?
- 1:01SPEAKER_01: 28.8 million exchanges with Claude over 45 days, using roughly 25,000 fake accounts.
- 1:09SPEAKER_00: Someone built 25,000 fake accounts to have 30 million conversations with an AI just to steal its capabilities.
- 1:17SPEAKER_01: And they weren't caught for six weeks.
- 1:20SPEAKER_01: The accounts used proxy networks and IP rotation to look like normal user traffic.
- 1:26SPEAKER_00: At that volume, what are we actually talking about in terms of data?
- 1:30SPEAKER_01: One analysis estimated roughly 14 billion tokens of training data derived from Claude's outputs.
- 1:36SPEAKER_01: That is an enormous amount of capability transfer.
- 1:40SPEAKER_00: And if you feed that into a competing model, what happens?
- 1:44SPEAKER_01: That model learns to reason, write code, and handle complex tasks the way Claude does, without having to spend the years and the hundreds of millions of dollars that Anthropic invested to get there.
- 1:55SPEAKER_00: And the goal was to take what makes Claude good at software engineering and reasoning and inject it into Alibaba's competing model.
- 2:03SPEAKER_01: The queries weren't random.
- 2:05SPEAKER_01: Anthropic says they systematically targeted Claude's strongest capabilities: code generation, multi-step reasoning, long complex tasks.
- 2:14SPEAKER_01: Someone mapped out what Claude does best and built a data collection operation designed to harvest it specifically.
- 2:20SPEAKER_00: This is a playbook, not a mistake, not an accident.
- 2:23SPEAKER_00: You don't build 25,000 proxy accounts by accident.
- 2:34SPEAKER_01: They sent a letter to the U.S.
- 2:36SPEAKER_01: Senate Banking Committee and shared evidence with the White House.
- 2:42SPEAKER_01: A policy that treats this as both a national security and an economic security issue.
- 2:47SPEAKER_01: Stronger protections against AI model extraction through API access.
- 2:51SPEAKER_01: Essentially, the law hasn't caught up to what's possible, and they're trying to change that.
- 2:57SPEAKER_00: I've run businesses my whole life.
- 2:59SPEAKER_00: This is industrial espionage.
- 3:01SPEAKER_00: The methodology is different, but the intent is identical.
- 3:05SPEAKER_01: And they're pushing for policy that treats this as a national security and economic security issue.
- 3:11SPEAKER_00: Here's where I want to bring this home for small business owners.
- 3:14SPEAKER_00: This might feel like a fight between giant tech companies, but it's not.
- 3:19SPEAKER_00: Walk me through it.
- 3:21SPEAKER_00: When you use an AI tool in your business, you're trusting it to behave the way it's supposed to, that its safety guardrails work, that it won't give your customer dangerous advice or expose you legally.
- 3:33SPEAKER_01: And one of the more disturbing details, the attack probed Claude's refusal behaviors.
- 3:40SPEAKER_01: Not just what it knows, but how it decides what not to say.
- 3:45SPEAKER_00: If you can map those guardrails, you can build a model that knows how to evade them.
- 3:50SPEAKER_01: Exactly.
- 3:50SPEAKER_01: So the concern isn't just that someone copied Claude's homework, it's that they may have copied the homework in a way that helps them build something that doesn't have the same safeguards.
- 3:59SPEAKER_00: A model that looks capable but will say things a properly aligned model won't say.
- 4:04SPEAKER_01: And that gets dangerous fast when you're using AI in customer-facing contexts in legal or financial situations, or anywhere the stakes are real.
- 4:14SPEAKER_00: And then that gets sold to businesses through cheap API access, bundled into tools that look completely legitimate.
- 4:22SPEAKER_01: This is the question every business owner should be asking right now.
- 4:25SPEAKER_01: Do you know what AI model is actually running under the hood of the tools you use?
- 4:30SPEAKER_01: Most people don't.
- 4:32SPEAKER_01: And they probably should, because this is powered by AI is not an answer.
- 4:37SPEAKER_01: That's like saying this food contains ingredients.
- 4:41SPEAKER_00: Which model, what version, what safety testing has been done on it?
- 4:45SPEAKER_01: The reputable tools will tell you if a vendor is cagey about what model they use, that's worth asking about.
- 4:52SPEAKER_00: Because the AI tool market has exploded.
- 4:56SPEAKER_00: There are hundreds of apps, assistance, and automation tools.
- 4:59SPEAKER_00: Many of them don't build their own models.
- 5:01SPEAKER_00: They license or build on top of others.
- 5:04SPEAKER_01: If the underlying model was trained on stolen data with compromised safety properties, that flows through to every product built on it.
- 5:11SPEAKER_00: You might be using a customer service tool you believe is built on a major American AI provider.
- 5:18SPEAKER_00: Maybe it is, but cheap offshore AI models are everywhere now.
- 5:23SPEAKER_00: You need to actually ask that question.
- 5:27SPEAKER_01: The practical advice: stick with AI tools that are transparent about what model they run on.
- 5:32SPEAKER_01: Reputable ones will tell you, and the cost difference between legitimate tools and cheap alternatives has never been smaller.
- 5:39SPEAKER_00: There's a bigger picture here, too.
- 5:41SPEAKER_00: The United States and China are in a genuine competition over AI capability.
- 5:46SPEAKER_00: That's just the reality of the industry right now.
- 5:50SPEAKER_01: Anthropic had already blocked direct access from within China.
- 5:54SPEAKER_01: The attackers bypassed it using proxies and IP rotation.
- 5:57SPEAKER_00: And weren't caught for six weeks.
- 6:01SPEAKER_01: Across 25,000 accounts, which tells you technical controls alone aren't sufficient at this scale.
- 6:07SPEAKER_00: That's why Anthropic went to the Senate.
- 6:10SPEAKER_00: They can't solve this with code.
- 6:13SPEAKER_01: What this incident makes clear, AI model capabilities are genuinely valuable.
- 6:18SPEAKER_01: Valuable enough that someone ran a 45-day covert operation to extract them.
- 6:23SPEAKER_00: Where there's real value, there are people trying to take it.
- 6:27SPEAKER_00: That's true in every industry.
- 6:29SPEAKER_00: AI is no different.
- 6:31SPEAKER_01: The question for every business owner are you building on a foundation you can trust?
- 6:36SPEAKER_00: Because eventually you'll be running important things on these tools.
- 6:40SPEAKER_01: Understand what's under the hood before something goes wrong.
- 6:44SPEAKER_01: Not after.
- 6:46SPEAKER_01: Thanks for listening to Control AI Profit.
- 6:48SPEAKER_01: I'm Frank.
- 6:49SPEAKER_01: And I'm Michael Cadenhead.
- 6:51SPEAKER_01: We'll see you tomorrow.