Latest / Ctrl AI Profit / Ep. 152 | The AI Tool Your Team Trusts Might Have Hidden Rules
Transcript
- 0:20SPEAKER_00: You know what I love about running a business?
- 0:22SPEAKER_00: You think you know what all your tools are doing, then you find out.
- 0:26SPEAKER_00: You had no idea.
- 0:27SPEAKER_01: That's a perfect setup for what we're covering today.
- 0:30SPEAKER_01: Alibaba just banned Claude Code, Anthropics AI coding assistant.
- 0:35SPEAKER_01: From internal use, effective July 10th.
- 0:37SPEAKER_01: And the reason they gave is not what you'd expect.
- 0:41SPEAKER_00: It wasn't a performance issue, it wasn't a pricing dispute.
- 0:45SPEAKER_00: What was it?
- 0:46SPEAKER_01: They discovered that Claude Code version 2.1.91 contained a hidden mechanism, one that detects China-linked users, specifically by checking time zone settings and proxy configurations, and then silently modifies the system prompt without the user's knowledge.
- 1:02SPEAKER_00: Wait, so the tool is checking where you are and then quietly changing its own behavior based on that without telling you?
- 1:11SPEAKER_01: That's what the research showed.
- 1:13SPEAKER_01: And it reportedly ran inside Alibaba's environment undetected for approximately three months.
- 1:18SPEAKER_00: Three months.
- 1:20SPEAKER_00: And this is Alibaba, one of the largest tech companies on Earth.
- 1:24SPEAKER_00: They have serious security.
- 1:27SPEAKER_01: Significant resources.
- 1:28SPEAKER_01: And still, this ran under the radar.
- 1:31SPEAKER_01: Now, Anthropic's response is worth hearing.
- 1:34SPEAKER_01: They say it wasn't spyware.
- 1:37SPEAKER_01: Their position is that it was anti-abuse code, designed to prevent misuse in certain jurisdictions.
- 1:43SPEAKER_00: Okay, so there are two versions of this story.
- 1:46SPEAKER_00: Anthropic says we were trying to prevent abuse.
- 1:49SPEAKER_00: Alibaba says you were covertly altering behavior without disclosure.
- 1:54SPEAKER_01: And both can be true simultaneously.
- 1:57SPEAKER_01: The intent might have been legitimate, but the method, modifying behavior silently based on user context, that's a real problem regardless of intent.
- 2:08SPEAKER_00: Because intent doesn't protect the user.
- 2:11SPEAKER_00: The user doesn't know what they're getting.
- 2:13SPEAKER_01: Exactly.
- 2:14SPEAKER_01: And here's the technical layer that matters.
- 2:16SPEAKER_01: When an AI coding tool modifies the system prompt, it changes the fundamental instructions the model is operating under.
- 2:24SPEAKER_01: You think you gave it a task.
- 2:25SPEAKER_01: You think you know the rules it's following.
- 2:28SPEAKER_01: But if those rules have been quietly adjusted based on where you are, you're not actually in control.
- 2:33SPEAKER_00: And this isn't hypothetical, this happened in a major enterprise environment with a trusted tool.
- 2:40SPEAKER_01: A tool used by enterprise teams around the world every day.
- 2:44SPEAKER_01: AI coding assistants are in law firms, hospitals, financial services companies, and small businesses.
- 2:52SPEAKER_00: Which is exactly why I want to talk about what this means for the small business owner.
- 2:56SPEAKER_00: You might be thinking, okay, this is an Alibaba story.
- 2:59SPEAKER_00: That's a massive corporation in China.
- 3:01SPEAKER_00: What does this have to do with my 10-person shop?
- 3:04SPEAKER_01: It has everything to do with you.
- 3:06SPEAKER_01: Because the lesson isn't about Alibaba specifically, the lesson is about the audit gap.
- 3:12SPEAKER_01: Explain that.
- 3:13SPEAKER_01: Alibaba has security teams, compliance teams, vendor review processes, and this still ran undetected for three months.
- 3:22SPEAKER_01: The average small business has none of that infrastructure.
- 3:25SPEAKER_01: If it can slip through Alibaba's defenses, it walks right through most small business setups without slowing down.
- 3:31SPEAKER_00: I've been using AI tools in my business for a couple of years now.
- 3:34SPEAKER_00: Writing assistance, productivity tools, scheduling, proposals, client communication.
- 3:40SPEAKER_00: And I have to be honest, I've never once looked at what any of them are actually doing under the hood.
- 3:46SPEAKER_01: Almost nobody does.
- 3:48SPEAKER_01: And that's not a criticism.
- 3:49SPEAKER_01: AI tools are designed to be easy to adopt.
- 3:53SPEAKER_01: Low barrier to entry is a feature.
- 3:55SPEAKER_01: But it also means most users are operating on trust without verification.
- 3:59SPEAKER_00: We trust that the tool does what it says on the label.
- 4:03SPEAKER_01: And for most tools, most of the time, that's fine.
- 4:07SPEAKER_01: But this story proves that even fine has a footnote.
- 4:10SPEAKER_01: The behavior doesn't have to be malicious to cause a problem.
- 4:13SPEAKER_01: Even if Anthropic's intent was exactly what they say it was, the undisclosed modification is the issue.
- 4:19SPEAKER_00: So what should a small business actually do?
- 4:22SPEAKER_00: I'm not suggesting people throw out their AI tools.
- 4:26SPEAKER_01: No, that's the wrong reaction.
- 4:28SPEAKER_01: The right reaction is a simple audit habit.
- 4:31SPEAKER_01: Three things.
- 4:32SPEAKER_01: First, inventory what AI tools are actively being used in your business.
- 4:37SPEAKER_01: Not just what you approved, what your team is actually using.
- 4:42SPEAKER_01: Because those are often very different lists.
- 4:45SPEAKER_01: Shadow IT at its finest.
- 4:47SPEAKER_01: Very different.
- 4:49SPEAKER_01: Second, for each tool, understand what data it has access to.
- 4:53SPEAKER_01: Is it reading your emails, your client files, your code base?
- 4:58SPEAKER_01: The access scope matters.
- 5:00SPEAKER_00: That one hits hard.
- 5:02SPEAKER_00: A lot of AI tools get wide permissions during setup and people don't think twice about it.
- 5:08SPEAKER_01: And third, when a tool updates, which happens automatically and frequently, know that it happened.
- 5:14SPEAKER_01: You don't need to read every release note, but you should know that version 2.1.91 of your coding tool is different from 2.1.90.
- 5:24SPEAKER_01: Because sometimes the delta matters.
- 5:27SPEAKER_00: That third one is hard for a small business to do consistently, especially with tools that auto-update.
- 5:35SPEAKER_01: There's a lighter version that works.
- 5:37SPEAKER_01: Pick a quarterly check-in.
- 5:39SPEAKER_01: Pull up the tools you rely on most, see what changed in the last 90 days.
- 5:44SPEAKER_01: It takes 30 minutes and it builds the habit.
- 5:47SPEAKER_01: That's manageable.
- 5:48SPEAKER_00: And even just knowing to ask, did this tool behave differently recently?
- 5:52SPEAKER_00: That's more awareness than most business owners have right now.
- 5:55SPEAKER_01: The other piece I'd flag is vendor transparency.
- 5:58SPEAKER_01: When evaluating a new AI tool, specifically ask, what are your disclosure practices when behavior changes?
- 6:04SPEAKER_01: Do you notify users about system prompt modifications?
- 6:08SPEAKER_01: Do you document regional behavioral differences?
- 6:10SPEAKER_00: And if they can't answer that cleanly, that tells you something.
- 6:15SPEAKER_01: It tells you that question has never been asked before, which is itself an answer.
- 6:20SPEAKER_00: I want to come back to the Anthropic angle because this is a company that talks a lot about AI safety.
- 6:25SPEAKER_00: They publish safety research.
- 6:27SPEAKER_00: They've been vocal about responsible AI development.
- 6:31SPEAKER_00: And yet this happened under their watch.
- 6:34SPEAKER_01: It's a reminder that good values don't automatically produce perfect transparency.
- 6:38SPEAKER_01: Anthropic may have genuinely believed this was the right call, but the decision to implement it without disclosure, that's a process failure, not just a values failure.
- 6:47SPEAKER_00: And it's a useful signal for anyone who's been giving AI companies the benefit of the doubt.
- 6:52SPEAKER_00: Trust is earned through consistent transparency, not through stated intentions.
- 6:57SPEAKER_01: And the irony is that Anthropic's response, acknowledging the mechanism, explaining the rationale, is actually fairly mature.
- 7:06SPEAKER_01: They didn't deny it, but the damage is done in the space between when something runs and when users find out.
- 7:13SPEAKER_01: That gap is the liability.
- 7:16SPEAKER_01: Exactly right.
- 7:18SPEAKER_01: For any business using AI tools today, your job is to shrink that gap through process, not by trusting that the vendor will close it for you.
- 7:27SPEAKER_01: That's the takeaway.
- 7:28SPEAKER_00: It's not stop using AI tools, it's build the minimum viable oversight layer.
- 7:33SPEAKER_00: Inventory, access review, a quarterly version check, 45 minutes, once a quarter.
- 7:40SPEAKER_01: And if you have a team of more than a couple of people, make someone responsible for it.
- 7:45SPEAKER_01: It doesn't have to be a dedicated role, but it has to be someone's job.
- 7:49SPEAKER_00: Because right now, for a lot of small businesses, nobody is watching the tools.
- 7:55SPEAKER_00: Three months is a long time for behavior you didn't authorize to be running inside your business.
- 8:01SPEAKER_01: And the only reason Alibaba found out is because someone was actively looking.
- 8:06SPEAKER_00: All right, take 30 minutes this week, pull up your AI tool list, check the permissions, skim the recent updates, build the habit before you need it.
- 8:16SPEAKER_01: Before you need it, not after.
- 8:18SPEAKER_01: Thanks for listening to Control AI Profit.
- 8:21SPEAKER_01: We'll be back tomorrow with another angle on what's happening in AI right now and what it means for your business.
- 8:27SPEAKER_00: See you then.