Latest / In The Loop / Why Anthropic are too scared to release their new model, Mythos

Transcript

Why Anthropic are too scared to release their new model, Mythos · In The Loop

• 0:00
  So on Wednesday, the US Treasury Secretary and the chair of the
• 0:03
  Federal Reserve called an emergency meeting with CE OS of
• 0:07
  America's biggest banks. The subject wasn't interest
• 0:10
  rates or inflation or, you know, pending World War Three.
• 0:13
  It was an AI model. Anthropic has now built
• 0:16
  something that could find and exploit security flaws in
• 0:20
  virtually any software, it was pointed out, and the financial
• 0:23
  system wanted to know how exposed it really was.
• 0:26
  Anthropic have decided not to sell this.
• 0:29
  Instead they've called all the biggest companies from Apple,
• 0:32
  Amazon, Google, banks, governments to form a coalition
• 0:35
  called Project Glass Way. And they've kept the model
• 0:38
  behind closed doors, only giving access to those companies.
• 0:42
  Today I want to walk through what this model actually does
• 0:45
  and what it means for us in practice, Why the sceptics have
• 0:49
  some really interesting points here, and why the way this was
• 0:53
  all handled tells us more about but where AI governance is truly
• 0:58
  at and what risks we really face.
• 1:01
  This is In the Loop with Jack Horton.
• 1:03
  Hope you enjoy the show. So let's start with what Mythos
• 1:18
  actually did. So Mythos is Anthropic's newest
• 1:21
  AI model. And what makes it quite unusual
• 1:24
  is that it turned out to be extremely good at, you know, one
• 1:27
  thing, you know, instead of being the best in the world at
• 1:30
  writing, and maybe it was really good at writing.
• 1:32
  It turned out it was really good at finding security flaws in
• 1:36
  software, not small business, not small theoretical problems
• 1:40
  that real flaws in real applications that are out there
• 1:43
  right now and used by basically everyone in the world.
• 1:46
  And these bugs have been missed by human experts for years.
• 1:51
  A little bit of context here. Every bit of software we use,
• 1:54
  you know, our browsers, our phones, the servers that handle
• 1:57
  online banking, there's just loads of bugs in it.
• 2:00
  And most of these are absolutely harmless, but some are what
• 2:04
  security researcher would describe as zero days.
• 2:07
  So flaws that nobody knew about but could let an attacker take
• 2:12
  control of an entire local machine.
• 2:14
  So you take control of your laptop just because you clicked
• 2:16
  on a website. If you heard about a company or
• 2:19
  a hospital or a government being hacked, it's usually because
• 2:22
  someone found and exploited one of these zero day problems.
• 2:26
  And you know, only a small group of elite researchers typically
• 2:29
  take on the job of finding these, whereas Mythos has just
• 2:33
  found them in minutes, you know, across many different systems
• 2:36
  all at once, and then without any human guidance or prompting,
• 2:39
  worked out exactly how to exploit them.
• 2:41
  And there are a few examples to talk through because, you know,
• 2:45
  it really tells us what this would mean in practice at scale.
• 2:49
  You know, it found a bug in Free BSD.
• 2:52
  So it's an operating system that basically runs a lot of Internet
• 2:55
  infrastructure and servers, networking equipment, basically
• 2:58
  the really boring backbone stuff that most people just never see.
• 3:02
  And that bug has been in the code since 2009.
• 3:04
  Ultimate Testing Tools had run through it over 5,000,000 times
• 3:08
  and it never caught this bug, whereas Mythos found it and
• 3:11
  actually built a multi step attack across different network
• 3:14
  requests that were given an attacker full control of
• 3:18
  everything. So in practical terms, that
• 3:20
  means anyone that could reach their server could read every
• 3:22
  file that exists on it, install whatever they wanted, and use it
• 3:26
  as basically a launchpad to attack any of the machines on
• 3:29
  that same network. He also found a 27 year old
• 3:32
  networking bugging Open BSD, which is another operating
• 3:36
  system known for being one of the most security countries in
• 3:39
  the world, and the the bug had been there since 1998.
• 3:43
  It found vulnerabilities in every major web browser and it
• 3:46
  found a chain of four separate bugs and flaws that if you
• 3:49
  string it together and attack, meant that if you visited a
• 3:52
  particular website and then Tucker could literally take
• 3:56
  control of the browser, the security, it can break out of
• 3:59
  the operating security and take control of your laptop.
• 4:02
  You wouldn't need to download anything or click anything.
• 4:05
  It would literally take control of your PC.
• 4:07
  Mythos has found thousands of these zero day vulnerabilities
• 4:11
  and few of them, 1% have actually been patched so far.
• 4:14
  And Anthropic have published cryptographic proof that these
• 4:18
  discoveries exist by basically tamper proof time stamps showing
• 4:21
  that these things really do exist.
• 4:23
  And they found these flaws before anyone else did.
• 4:26
  And you know, none of this was the goal.
• 4:28
  You know, Anthropic actually set out to build the best coding
• 4:30
  model in the world. And it turns out that also makes
• 4:33
  it really good at attacking software.
• 4:51
  Now it's worth having a conversation about the skeptics
• 4:54
  because there are a lot of skeptics out there and write
• 4:57
  this out. For example, the headline number
• 4:59
  that Anthropic have given out is thousands of 0 day flaws and
• 5:04
  books and Anthropic as Human reviewers check 190 of these
• 5:09
  reports and reviewers agreed with the models assessment about
• 5:12
  89% of the time, but that means there's still thousands that are
• 5:15
  unchecked. So there's a lot of
• 5:16
  extrapolation there to say that there are thousands of 0 day
• 5:21
  bugs and flaws. Another example is the security
• 5:24
  research. Different labs for example, have
• 5:26
  tested some of the vulnerabilities on cheaper
• 5:29
  smaller models and they ran 8 experiments and found eight of
• 5:33
  the problems. So it's 100% matching Mythos.
• 5:39
  Now there's there's, again, lots of nuance and extrapolation that
• 5:41
  that was only 8 problems. And it was also focused on the
• 5:45
  things that Mythos had already found.
• 5:47
  And on top of that, what those models definitely didn't do is
• 5:50
  then build an entire attack plan by itself.
• 5:53
  And another thing that I think is most compelling for me is
• 5:57
  that there's just this timing aspect.
• 5:58
  So Anthropic is about to go towards a stock market listing
• 6:02
  and it'll be valued at about $380 billion, apparently, you
• 6:05
  know, in a model that's too dangerous to release is
• 6:08
  brilliant PR in the world for everyone getting excited and
• 6:11
  putting lots of money in them. And there are many arguments out
• 6:14
  there saying that Anthropic are essentially withholding the best
• 6:17
  models so they can charge big enterprise huge amounts of money
• 6:23
  and essentially keep everyone else a generation behind.
• 6:26
  And, and there's good reasoning for that, you know, those
• 6:29
  listening, most likely me, our team, we're power users and
• 6:33
  power users cost model providers money.
• 6:36
  You know, we might be paying. You know, I've got a couple of
• 6:38
  accounts at $200 per account. We definitely use more than $200
• 6:43
  worth of credits every month. Again, intelligence isn't cheap,
• 6:47
  but you know, the long tail of you, let's say hundreds of
• 6:50
  millions of users out there that use it, fraction of the amount
• 6:54
  that maybe I do, That's where they make their money.
• 6:57
  So then suddenly finding a really good model, charging
• 7:01
  absolute fortunes to big enterprise, who wants a
• 7:03
  competitive advantage like that and will it be able to pay those
• 7:06
  huge sums of money is a very clever strategy.
• 7:10
  And there's also an aspect which is that we have been here
• 7:12
  before. You know, in 2019, Open AI
• 7:14
  announced a model called GPT 2, and they refused to release it.
• 7:18
  You know, they said it was too dangerous.
• 7:20
  It could write so convincingly that it's going to fuel
• 7:22
  disinformation and propaganda. And so they staged it out over
• 7:26
  nine months, releasing progressively larger versions.
• 7:29
  And, you know, by the time the model had come out, they had big
• 7:31
  test groups and nobody found any risk in the end.
• 7:35
  But obviously, it was brilliant PR for them to raise more money
• 7:39
  and attract more exciting people to come work for them, come
• 7:42
  invest in them. Though what I would add on to
• 7:45
  that is a little bit of a nuance, which is, you know, a
• 7:47
  mythos is a bit different. You know, Anthropic isn't
• 7:49
  withholding anything. They're in fact literally
• 7:52
  publishing all the evidence, all the reports, and allowing
• 7:57
  anybody to test them themselves. And arguably you can't really
• 8:00
  have a staged release process when the model is able to just
• 8:03
  attack software at will and take down the entire Internet.
• 8:07
  It's a bit too dangerous to go and risk it.
• 8:25
  So, yeah, is this real or is this marketing?
• 8:27
  And it's a good question. I think it's both and I think
• 8:31
  it's OK to have both. You know, two things can be true
• 8:33
  at once. It probably is quite dangerous
• 8:37
  or they're worried that it's dangerous and it's a wonderful
• 8:39
  PR opportunity. But personally, I think the
• 8:42
  bigger and more important question to ask of instead of is
• 8:45
  this PR marketing? Because, you know, of all the
• 8:48
  companies, I think Anthropic is probably the most ethical.
• 8:51
  You know, we covered the episode where they fought back against
• 8:53
  the US government and the Department of War.
• 8:56
  But the bigger question really is who decides that something is
• 8:59
  or is not safe? At what point does this become
• 9:02
  safe enough to release now? So like Anthropic is a good
• 9:05
  example, looked at what they built, concludes that gosh, this
• 9:07
  was too dangerous for the public and decided which specific
• 9:11
  companies could get to use this. And they called it Project Glass
• 9:15
  Wing. You know, Amazon, Apple, Google,
• 9:18
  Microsoft, 40 other organizations get access.
• 9:21
  And yes, Anthropica committed $100 million in computer credits
• 9:25
  to fine and patch any vulnerabilities.
• 9:27
  But essentially they are still a private company making a private
• 9:30
  assessment from the private coalition, arranged a private
• 9:33
  solution with no, say, treaty inspectors, independent
• 9:37
  oversight, no democratic input. Yet when it comes to nuclear
• 9:41
  weapons or industries, you have international oversight,
• 9:45
  independent governance bodies who are deciding who might get
• 9:48
  access, whether they're actually breaking the rules, whether
• 9:51
  they're being safe. Now, I'm not trying to be all
• 9:53
  too European on this in terms of trying to create huge regulation
• 9:58
  that makes it really difficult for these organizations to
• 10:00
  operate because that's definitely not me.
• 10:03
  However, I am often quite balanced when it comes to
• 10:07
  wanting to pair risk and reward. So maybe they are the right
• 10:12
  organization to take the right steps and make the right
• 10:15
  choices, but equally at all organizations, the right
• 10:19
  organization. I don't trust open AI as far as
• 10:21
  I could throw them. See, I think this tells us where
• 10:23
  we are in the maturity of AI and governance generally, that we
• 10:28
  still rely on private companies and we still haven't got the
• 10:31
  systems in place which Anthropic of openly acknowledged to keep
• 10:35
  society safe as these things are rolled out.
• 10:54
  Anyway, what I think to conclude, what sticks for me
• 10:56
  isn't where the mythos is, you know, super powerful and, and I
• 10:59
  actually really hope it is because it's quite exciting.
• 11:01
  It's going to help me at work, it's going to help me at life.
• 11:04
  And to be honest, it probably is, or at least close to it.
• 11:06
  But what sticks with me is that this is the first time the AI
• 11:09
  industry as a whole has agreed, really, that this capabilities
• 11:13
  are too dangerous to release based on real results rather
• 11:17
  than, you know, theoretical risk.
• 11:19
  And the best way we could manage this risk was obviously a group
• 11:22
  of private companies deciding amongst themselves what to do
• 11:25
  about it. So yeah, I think the question on
• 11:27
  my mind right now is who decides what happens next or when the
• 11:31
  next one appears, or on whose authority and accountability do
• 11:36
  these things get released? Anyway, thanks for listening and
• 11:40
  I'll see you next week.