Latest / Dark Perimeter: Real Breaches, Real Stakes / It Passed Every Check
Transcript
- 0:00SPEAKER_01: On Thursday of last week, more than six billion dollars left a cryptocurrency exchange in about 20 minutes.
- 0:08SPEAKER_01: The FBI has since attributed it to North Korea, and by the count of the blockchain analytics firms that traced it, it is the largest theft in history.
- 0:18SPEAKER_01: We are not going to spend this episode on it.
- 0:22SPEAKER_01: Binance has said its investigation found additional points of breach, it has not published a root cause, and that means anything we told you about how it happened would be a guess.
- 0:33SPEAKER_01: Instead, we are going to spend the hour on a story with no dollar figure attached to it at all, because the victim did something rare.
- 0:42SPEAKER_01: They published everything.
- 0:45SPEAKER_01: On October 21st, a widely used open source AI gateway shipped a release that stole credentials from the machines it landed on.
- 0:54SPEAKER_01: That release was built by the project's own pipeline, signed with the project's own key, and approved through the project's own production gate.
- 1:04SPEAKER_01: It passed every check.
- 1:06SPEAKER_01: With me, as always, are Dr.
- 1:09SPEAKER_01: Elliot Vance, who studies how these things actually work, and Marcus Hale, who has to run a program in the world where they do.
- 1:18SPEAKER_01: Elliot, start with what is confirmed.
- 1:22SPEAKER_00: The project is Light LLM.
- 1:24SPEAKER_00: It is an open source gateway that sits between applications and the large language model providers.
- 1:29SPEAKER_00: So by design it holds the API keys for those providers, and it is downloaded at very high volume.
- 1:35SPEAKER_00: At 1139 in the morning, UTC, on October 21st, a new version appeared on the Python package index.
- 1:43SPEAKER_00: Seven minutes later, the matching container images were published, and the tags that most people pull, including latest, were pointed at them.
- 1:51SPEAKER_00: Inside those packages were three files that had nothing to do with Light LLM, a credential stealer and a loader.
- 1:58SPEAKER_00: The package stayed on the index for one hour and 43 minutes before it was quarantined, and the container tags stayed poisoned for roughly another hour.
- 2:06SPEAKER_00: Wiz, citing download analytics, put the downloads in that window at about 49,000, and that figure includes mirrors and build caches, so it counts machines, not organizations, and nobody has published an organization count.
- 2:19SPEAKER_00: Lead LLM says customers on its private enterprise registry were not affected.
- 2:24SPEAKER_02: Let me put a frame on 49,000 because people hear a package name and think developer laptops.
- 2:29SPEAKER_02: This thing is a proxy.
- 2:30SPEAKER_02: It runs in production, it runs in the build system.
- 2:33SPEAKER_02: The places that pull latest at a quarter to 12 without a human in the loop are build runners and Kubernetes nodes, and those are exactly the environments where Wiz says the stolen credentials were used a few hours later.
- 2:43SPEAKER_02: So that number is not 49,000 curious engineers.
- 2:47SPEAKER_02: A good share of it is infrastructure that updated itself.
- 2:50SPEAKER_01: Elliot, you said three files.
- 2:53SPEAKER_01: What did they do, and why does it matter where they sat?
- 2:57SPEAKER_00: It matters a great deal where they sat.
- 2:59SPEAKER_00: The usual supply chain payload hides inside the library you installed, so it runs when you import that library.
- 3:06SPEAKER_00: These three files were dropped into the root of the Python environment, outside the light LLM folder entirely.
- 3:12SPEAKER_00: One of them is a path configuration file, the kind with the PTH extension, and the Python interpreter reads every one of those at startup.
- 3:20SPEAKER_00: This one was a single line, import cook, and cook was the stealer.
- 3:24SPEAKER_00: The second file was called SiteCustomize, which Python also runs automatically on startup, and that one was the loader.
- 3:30SPEAKER_00: The effect is that any Python process in that environment runs the malware.
- 3:34SPEAKER_00: You did not have to use Lite LLM, you did not have to import it.
- 3:37SPEAKER_00: And running pip to investigate the problem would run it again.
- 3:41SPEAKER_00: Lite LLM's own guidance made that point explicitly, and Microsoft's analysis said, and I am quoting, removing the package or a release tag is not sufficient recovery.
- 3:51SPEAKER_01: And what was it collecting?
- 3:53SPEAKER_00: Everything that identifies a machine to something else.
- 3:56SPEAKER_00: Cloud credentials for the three major providers, including the metadata endpoints that hand out instance role credentials, browser passwords and history, SSH keys, GPG keys, TLS keys, shell history, the environment variables of every process it could read, which on a build runner is where the secrets live.
- 4:15SPEAKER_00: And then the part that is specific to this target, it queried the Lite LLM proxy database for the credentials table and the model table, up to 500 rows each, which is the gateway's own store of every provider key it had been trusted with.
- 4:28SPEAKER_00: The collector encrypted all of it with a fresh symmetric key, wrapped that key with an embedded 4,096-bit public key, and posted the bundle to a lookalike domain that researchers say was registered the week before.
- 4:40SPEAKER_00: There was also a second stage loader that could fetch a server address from a dead drop and execute whatever came back.
- 4:46SPEAKER_00: But Wiz and Microsoft both report that the dead drop was never populated beyond test values.
- 4:51SPEAKER_00: So as far as anyone can tell, the second stage never fired.
- 4:55SPEAKER_02: And the first stage was enough.
- 4:57SPEAKER_02: Here's the detail I want every listener to sit with.
- 4:59SPEAKER_02: Wiz reported that between about 4.20 and 5 in the afternoon, UTC, the same day, stolen AWS instance role credentials taken from build runners and Kubernetes nodes were being used from attacker infrastructure.
- 5:10SPEAKER_02: The first call was the one that tells you who a credential belongs to.
- 5:14SPEAKER_02: The next calls listed the foundation models available in bedrock.
- 5:17SPEAKER_02: That is the whole plan.
- 5:18SPEAKER_02: They were not after your customer database, they were after your ability to buy inference because model capacity is a thing you can resell.
- 5:24SPEAKER_02: Roughly four and a half hours from exfiltration to use by Wiz's estimate, and that is one vendor's telemetry.
- 5:30SPEAKER_02: So the real picture is at least that fast.
- 5:33SPEAKER_01: So that is the what.
- 5:35SPEAKER_01: Marcus, before we go into how they got in, give me your gut reaction to the shape of this.
- 5:42SPEAKER_02: My gut reaction is that the payload is the least interesting part.
- 5:45SPEAKER_02: Credential stealers are a commodity.
- 5:47SPEAKER_02: The startup hook trick is clever, and Wiz says it is the first time they have documented that pairing in the wild, fine.
- 5:53SPEAKER_02: But the reason this matters is upstream of the payload.
- 5:56SPEAKER_02: This was not a typo squat, and nobody misspelled a package name.
- 5:59SPEAKER_02: The poison came down the real channel from the real publisher with the real signature through the real approval gate.
- 6:05SPEAKER_02: Every control that a mature consumer of open source is told to rely on said yes.
- 6:09SPEAKER_02: If your defense was that you only install from the official source and you verify signatures, you were fully compliant and fully compromised at the same time.
- 6:17SPEAKER_01: Elliot, take us back.
- 6:20SPEAKER_01: How does an attacker end up holding the project's own pipeline?
- 6:24SPEAKER_00: It starts in May, and it starts with a mistake that every organization listening has made.
- 6:29SPEAKER_00: On May 27th, an administrative key for an internal experimental service, something Light LLM calls its agent platform, was committed to a public repository.
- 6:39SPEAKER_00: Light LLM's own report says nobody followed through on revoking it, and it stayed public and valid for the better part of five months.
- 6:46SPEAKER_00: On October 10th, someone used it.
- 6:49SPEAKER_00: The agent platform had been configured, not with a scope token, but with the GitHub command line OAuth token, belonging to one of the organization's two owners.
- 6:58SPEAKER_00: The attacker made an API call to the platform, and the response handed that token back.
- 7:03SPEAKER_00: Light LLM's report makes a point about that class of token that I think is underappreciated.
- 7:08SPEAKER_00: Those tokens are not scoped.
- 7:10SPEAKER_00: They are issued after two-factor has already been satisfied, and they are not subject to GitHub's reauthentication prompts for sensitive action.
- 7:18SPEAKER_00: So on October 10th, the attacker is, for every practical purpose, an owner of the organization.
- 7:23SPEAKER_00: Then they did something patient.
- 7:25SPEAKER_00: That same evening, and again two days later, they created throwaway workflow runs in private repositories that posted repository secrets out to a public webhook service and then deleted the run records.
- 7:37SPEAKER_00: Two of the secrets they took were automation tokens belonging to two other maintainers.
- 7:41SPEAKER_00: So by October 12th, they held three identities.
- 7:44SPEAKER_00: Then they waited nine days.
- 7:46SPEAKER_00: On October 21st, between about 9 and 11 in the morning UTC, they pushed 10 comets into a look-alike organization.
- 7:53SPEAKER_00: The real organization is Berry AI, written as one word.
- 7:56SPEAKER_00: The fake one had a hyphen in it.
- 7:58SPEAKER_00: Those comets added the three files we discussed.
- 8:01SPEAKER_00: At eight minutes past 10, they pushed a change to the private release repository that made the workflow's source repository a parameter instead of a hard-coded value.
- 8:10SPEAKER_00: Branch protection rejected it because that branch required a pull request, two passing checks, and a code owner approval.
- 8:17SPEAKER_00: Five seconds later, the owner token disabled branch protection and force pushed the change.
- 8:22SPEAKER_00: Five seconds.
- 8:22SPEAKER_00: That is the whole gap between a control existing and a control mattering.
- 8:27SPEAKER_02: That five seconds is the episode.
- 8:29SPEAKER_02: Everybody who has ever written a policy document has written the sentence that branch protection prevents unauthorized changes to the release branch.
- 8:36SPEAKER_02: It does, right up until the person making the change is allowed to turn it off.
- 8:40SPEAKER_02: The pipeline worked exactly as designed.
- 8:42SPEAKER_02: Light LLM said it themselves in their follow-up, and I'm going to quote it because I think it is the most honest sentence a vendor has written this year.
- 8:49SPEAKER_02: No pipeline controls can defend against an attacker who is an organization owner.
- 8:53SPEAKER_02: The attacker did not break the gate.
- 8:54SPEAKER_02: He had the key to the gate, so he opened it.
- 8:57SPEAKER_01: Finish the sequence, Elliot.
- 8:59SPEAKER_00: At 1046, the owner token dispatched the staging build, pointed at the look-alike fork.
- 9:04SPEAKER_00: Staging finished, production started a little after 11, and the production approval gate requires a second person.
- 9:10SPEAKER_00: The second maintainer's stolen token approved it about two minutes later.
- 9:13SPEAKER_00: Light LLM notes that two days earlier, the approval rule had been loosened so that any two engineers could trigger and approve a release, and says plainly that it cannot rule out the owner token, having done the loosening.
- 9:26SPEAKER_00: At 1139, the seven wheels went to the Python package index through the project's trusted publisher identity.
- 9:32SPEAKER_00: At 1146, the containers went to Docker Hub and GitHub's registry, signed with the project's signing key.
- 9:39SPEAKER_00: And at 1153, the third maintainer's token deleted the production and staging run records, which removed the logs.
- 9:45SPEAKER_00: Light LLM believes three separate identities were used deliberately so that no single account's audit trail looked unusual.
- 9:52SPEAKER_00: That part is their inference, not established fact, but it fits.
- 9:56SPEAKER_01: And detection?
- 9:58SPEAKER_00: A community member, someone noticed the odd files and opened a public issue on the project's repository at six minutes past 12, which is under half an hour after publication.
- 10:08SPEAKER_00: Lead LLM's timeline has the team aware by about one in the afternoon, and the package frozen on the index by 23 past.
- 10:16SPEAKER_00: The container registries were cleaned up over the following hour.
- 10:19SPEAKER_00: For a two-hour exposure, that is fast.
- 10:21SPEAKER_00: It also means the control that caught it was a stranger's eyes and a GitHub issue, not any of the tooling in the pipeline.
- 10:36SPEAKER_02: Both, and I want to be fair in both directions.
- 10:38SPEAKER_02: On the critical side, this is the second time this year this project's release channel has shipped something it should not have.
- 10:44SPEAKER_02: In March, by their own account, a compromise scanning dependency that they ran unpinned in the build system read static registry credentials out of environment variables.
- 10:53SPEAKER_02: They rebuilt the pipeline after that with trusted publishing and signing and pinned actions, and that rebuilt pipeline is the one that published this.
- 11:00SPEAKER_02: So the hard lesson is that they fixed the build problem, and the attacker went around it to the identity layer.
- 11:05SPEAKER_02: On the credit side, and I do not say this often, the response was fast, the write-up is specific down to the second.
- 11:10SPEAKER_02: They published the token types and the account rolls, and when their first explanation turned out to be wrong, they said so in public and replaced it.
- 11:17SPEAKER_02: Their canonical report walks back the original theory about where the token was taken from.
- 11:21SPEAKER_02: Most vendors never get to the second paragraph of that.
- 11:25SPEAKER_00: I would add what we do not know, because it is a shorter list than usual, but it matters.
- 11:29SPEAKER_00: We do not know how many organizations were affected.
- 11:32SPEAKER_00: 49,000 is downloads, and nobody has published a victim count.
- 11:36SPEAKER_00: We do not know that the second stage ever ran anywhere, and the evidence says it did not.
- 11:41SPEAKER_00: The credential abuse observation comes from one vendor's telemetry, so we know it happened in the environments Wiz could see, and we infer it happened elsewhere.
- 11:49SPEAKER_00: And we do not know who did this.
- 11:51SPEAKER_00: None of the primary sources attribute it, and I would be wary of anyone who does this week.
- 11:55SPEAKER_00: What we do know is the sequence with timestamps from the victim's own logs, and that is more than we get for most incidents ten times this size.
- 12:04SPEAKER_01: Which brings us back to where we started.
- 12:06SPEAKER_01: Marcus, you run a program.
- 12:09SPEAKER_01: Someone walks into your office Monday with this story, what changes?
- 12:14SPEAKER_02: Six things, and none of them are exotic.
- 12:17SPEAKER_02: One, revocation is the control, not detection.
- 12:20SPEAKER_02: That key was committed in public in May, and by the project's own account, nobody followed through on revoking it, so whatever anyone noticed did not matter.
- 12:27SPEAKER_02: Lead LLM's new rule is that a secret scanning alert opens a ticket, and the ticket cannot close until revocation is confirmed, and I would steal that rule, verbatim.
- 12:36SPEAKER_02: Truffle Security said this week that most of the leaked keys it reported a year ago were still live six months later, so this is not a light LLM quirk.
- 12:44SPEAKER_02: It is the industry default, too.
- 12:46SPEAKER_02: Nothing automated authenticates as a person.
- 12:48SPEAKER_02: The agent platform was carrying a human owner's token, and every bot, runner, and internal tool should instead have its own identity, with minimum scope and an expiry.
- 12:56SPEAKER_02: Light LLM is moving to GitHub apps and fine-grained expiring tokens, and is said that automation still running on human tokens stay switched off until they are rebuilt.
- 13:05SPEAKER_01: Keep going.
- 13:15SPEAKER_02: Late LLM's phrasing is that no human day-to-day account is an owner, and I would adopt it word for word.
- 13:20SPEAKER_02: Four, separate the approval from the pipeline.
- 13:22SPEAKER_02: Their release approval is now held by a single account bound to a physical device in their office.
- 13:27SPEAKER_02: You may hate that answer for your environment, so find your version of it.
- 13:30SPEAKER_02: The point is that the thing that says yes to production cannot be reached by the thing that was compromised to build it.
- 13:35SPEAKER_01: And five and six.
- 13:37SPEAKER_02: Five, verify after you publish from outside.
- 13:40SPEAKER_02: By the end of the year, they intend to run a verifier that lives outside GitHub, downloads what actually landed on the index in the registries, checks out the tag and compares, and if they differ, it yanks the release.
- 13:50SPEAKER_02: That is the control that would have caught this in minutes instead of relying on a stranger with a GitHub account.
- 13:55SPEAKER_02: Six is for the consumer side, which is most of you.
- 13:58SPEAKER_02: Pin versions and pin image digest, not tags, because latest is not a version, it is a promise someone else made.
- 14:04SPEAKER_02: Know what a PTH file is, and alert when one appears in a site packages route.
- 14:08SPEAKER_02: And if you ran the poisoned release with outbound network, light LLM's own guidance is to assume the exfiltration succeeded, rotate from a clean machine, rotate the gateway's provider keys and master key, and database password and instance role, and go through your cloud audit logs for identity calls from addresses you do not own.
- 14:30SPEAKER_01: That is a small company solution.
- 14:33SPEAKER_01: Does it scale?
- 14:35SPEAKER_02: The specific implementation does not.
- 14:37SPEAKER_02: The principal does, and the principal is older than any of us.
- 14:40SPEAKER_02: The identity that can build a thing should not be the identity that can release it, and neither of those should be a token you can steal out of a service response.
- 14:46SPEAKER_02: Big shops do that with hardware keys, with separate signing infrastructure, with break glass accounts that page a human when they wake up.
- 14:53SPEAKER_02: Small shops do it with a laptop and a drawer.
- 14:56SPEAKER_02: Either way, if your release approval is just another secret in the same GitHub organization, it is one hop from the first compromise, and this incident is the proof.
- 15:04SPEAKER_01: Elliot, pull the camera back.
- 15:07SPEAKER_01: Where does this pattern go?
- 15:10SPEAKER_00: Three directions, and I think they are already visible.
- 15:13SPEAKER_00: The first is that the infrastructure around AI is becoming the preferred target, rather than the models themselves.
- 15:19SPEAKER_00: A model gateway is, by design, a credential concentrator.
- 15:24SPEAKER_00: It exists to hold every provider key so your applications do not have to, which makes its database the single most valuable table in the environment.
- 15:32SPEAKER_00: And the attackers knew that because the collector queried it by name.
- 15:35SPEAKER_00: The follow-on behavior tells you the market.
- 15:38SPEAKER_00: Within hours, stolen cloud credentials were being tested for access to model inference.
- 15:43SPEAKER_00: Compute you can resell has become a reason to break in.
- 15:45SPEAKER_00: These same two weeks saw reporting of intrusions at several AI developer platforms where the attackers mostly hunted stored secrets and model access tokens.
- 15:54SPEAKER_00: A hosting control panel whose update channel was used to push a root-level backdoor for about 70 minutes, and a popular coding extension whose marketplace bundle was swapped for a backdoored one for roughly half a day.
- 16:06SPEAKER_00: Different actors, different victims, one shape, compromise the thing developers trust to deliver code, and let distribution do the rest.
- 16:13SPEAKER_00: The second direction is identity as the supply chain.
- 16:16SPEAKER_00: We spent several years hardening the build, pinned actions, trusted publishing, signed artifacts, reproducible builds.
- 16:23SPEAKER_00: Those are good, and Light LLM had them.
- 16:26SPEAKER_00: But a signature proves who published, not whether they meant to.
- 16:29SPEAKER_00: If the publisher's identity is a long-lived token sitting in a service response, the signature travels with the thief.
- 16:35SPEAKER_00: I expect the next generation of this attack to look cleaner than this one.
- 16:38SPEAKER_00: Fewer traces, no deleted logs, releases that are indistinguishable from legitimate ones because they are, in every cryptographic sense, legitimate.
- 16:47SPEAKER_00: The defense has to move to the identity layer and to independent verification after the fact, which is exactly what Light LLM is now building.
- 16:55SPEAKER_00: The third is the hardest, and Light LLM said it better than I can.
- 16:59SPEAKER_00: Everything on a developer's machine should be assumed readable by the agents running on it.
- 17:03SPEAKER_00: They declined to ban coding agents, and I think that is correct.
- 17:07SPEAKER_00: But the consequence is that the developer workstation and every token on it are now in scope for whatever that agent does, including the mistakes it makes and the instructions it is tricked into following.
- 17:18SPEAKER_00: The make commit was a human error, but the service that leaked the owner token was an experimental platform for running agents.
- 17:25SPEAKER_00: The more we delegate to automation, the more the automation's credentials become the crown jewels, and the less any human is watching at the moment they are used.
- 17:33SPEAKER_00: That is the edge of this, and I do not think the industry has absorbed it yet.
- 17:38SPEAKER_01: Let me try to put this together.
- 17:41SPEAKER_01: The headline this week was six billion dollars, and we will not know for a while how that one actually happened.
- 17:48SPEAKER_01: The story we chose has no dollar figure, and we know it to the second because the victim told us.
- 17:55SPEAKER_01: What it tells us is uncomfortable.
- 17:59SPEAKER_01: A project that had already been hit once this year, that had rebuilt its pipeline the right way, with signing and pinning and trusted publishing and a two-person production gate shipped malware through that pipeline anyway.
- 18:14SPEAKER_01: Not because a control failed, but because the attacker arrived holding the identity the controls were built to trust, and had been holding it for 11 days.
- 18:25SPEAKER_01: It took five seconds to turn off branch protection, under half an hour for a stranger to notice, and about four and a half hours for the stolen keys to be used.
- 18:35SPEAKER_01: Marcus's list is the practical version.
- 18:38SPEAKER_01: Revoke what you find, never let a machine be a person, keep owners out of daily work, put the approval somewhere the pipeline cannot reach, and verify from outside after you publish.
- 18:52SPEAKER_01: Elliot's version is the uncomfortable one, the artifacts were real, the signatures were real, and the identity was real.
- 19:01SPEAKER_01: It just was not theirs anymore.
- 19:03SPEAKER_01: Signing tells you who.
- 19:05SPEAKER_01: It does not tell you whether they meant it.
- 19:08SPEAKER_01: Thanks to Elliot Vance and Marcus Hale, this is dark perimeter.