Latest / AI Fire Daily / 🎙 EP 380: Anthropic to Ban High-Risk Models & Inception Launches Mercury 3 Diffusion Model

Transcript

🎙 EP 380: Anthropic to Ban High-Risk Models & Inception Launches Mercury 3 Diffusion Model · AI Fire Daily

• 0:00
  What if an AI could foreseeably cause 50 deaths
• 0:02
  or a billion dollars in damage? Well, Anthropic
• 0:06
  actually thinks it's possible. And they are,
• 0:08
  you know, drafting a law to stop it. Welcome
• 0:12
  to the Deep Dive. If you've been watching this
• 0:14
  space, you know it's moving insanely fast. No,
• 0:17
  absolutely breakneck. Right. But today we are
• 0:20
  looking at a massive collision. We have raw,
• 0:23
  decentralized technological speed crashing straight
• 0:26
  into regulators. Regulators who are literally
• 0:28
  trying to legislate a death threshold. Yeah,
• 0:32
  before this tech even hits your devices. Okay,
• 0:35
  let's unpack this. We'll look at Anthropic's
• 0:37
  radical new safety bill. We'll cover major geopolitical
• 0:40
  security shakeups. Which are huge right now.
• 0:42
  Very. We'll explore the massive explosion of
• 0:45
  new consumer tools, too. And finally, we'll break
• 0:48
  down a massive speed breakthrough from Inception's
• 0:51
  Mercury 3. That speed breakthrough is wild, by
• 0:53
  the way. I can't wait to get to it. But let's
• 0:56
  start with the highest stakes imaginable. Right.
• 0:59
  We are talking about the effort to regulate these
• 1:01
  frontier models and to do it before they even
• 1:03
  hit the public. Anthropic is fundamentally arguing
• 1:06
  that voluntary safety promises are no longer
• 1:08
  enough, right? Exactly. They want real, enforceable,
• 1:11
  legal teeth. They actually drafted a proposed
• 1:15
  bill. And the specifics of this bill are, frankly,
• 1:19
  intense. Yeah, they really are. It basically
• 1:21
  allows a Washington, D .C. judge to step in.
• 1:24
  They can legally block a frontier model's release
• 1:27
  entirely. But only under very specific conditions.
• 1:30
  I mean, independent evaluators have to believe
• 1:32
  releasing it creates a foreseeable imminent risk.
• 1:36
  And they put hard numbers on that risk threshold.
• 1:39
  Which is incredibly rare in tech or legislation.
• 1:42
  It is completely unprecedented. Yeah. They define
• 1:46
  the risk as 50 or more deaths. or $1 billion
• 1:49
  in property damage. It's a fascinating attempt
• 1:52
  to quantify catastrophe before it happens. It
• 1:55
  makes you wonder how you even measure that. Well,
• 1:57
  they specifically outline scenarios that would
• 1:59
  trigger this alarm. We are talking about models
• 2:02
  that could help create a biological weapon. Right.
• 2:05
  Or a model that could enable a major cyber attack.
• 2:09
  Exactly. They also include a very science fiction
• 2:12
  scenario. They worry about a model simply evading
• 2:15
  its developers' controls. Like an AI acting autonomously
• 2:19
  in ways creators cannot stop. Yeah. If the evaluators
• 2:23
  detect that level of risk, a clock starts ticking.
• 2:26
  They'd have exactly 72 hours to alert the Commerce
• 2:29
  Department. And then the Commerce Department
• 2:31
  takes that intelligence straight to a DC judge.
• 2:33
  Right. They legally stop the model from being
• 2:36
  deployed. It is essentially an emergency break
• 2:38
  for artificial intelligence. But I have to push
• 2:41
  back on the mechanics here. The entire system
• 2:43
  relies on these independent evaluators. True.
• 2:47
  But the proposed rules say something very interesting.
• 2:49
  Those evaluators can still be paid by the very
• 2:52
  AI labs they evaluate. That is the caveat. As
• 2:56
  long as the payment is not explicitly tied to
• 2:58
  the outcome. Let's pause and think about human
• 3:01
  nature. It's kind of like paying a building inspector
• 3:03
  who relies on the builder for their paycheck.
• 3:06
  You can say the payment isn't tied to the inspection
• 3:08
  passing, but financial incentives usually blur
• 3:11
  those ethical lines. That is exactly the primary
• 3:14
  critique of the bill. It creates a massive potential
• 3:17
  conflict of interest. Definitely. And it highlights
• 3:20
  a massive philosophical divide in the industry
• 3:22
  right now. You have Anthropic pushing for strict,
• 3:25
  preemptive legal frameworks, and then you have
• 3:27
  other massive players taking the exact opposite
• 3:29
  stance. Look at NVIDIA's CEO, for instance. He
• 3:33
  recently argued the industry does not need new
• 3:35
  AI laws at all. Really? Yeah. He believes the
• 3:38
  current trajectory of innovation should not be
• 3:40
  artificially bottlenecked. So he just trusts
• 3:42
  the engineering process. He does. What's fascinating
• 3:46
  here is the ideological split. One side sees
• 3:49
  an existential threat requiring judicial oversight.
• 3:52
  And the other side? The other side sees an engineering
• 3:55
  challenge, a challenge that self -correction
• 3:57
  and iterative design will eventually solve. Wait,
• 4:00
  what stops labs from just buying favorable safety
• 4:03
  reviews? Legally, just the rule decoupling payment
• 4:06
  from the evaluation results. Practically, it
• 4:09
  relies entirely on the ethical boundaries of
• 4:11
  the auditing firms. So testers are independent,
• 4:14
  but still paid by the labs. Tricky. Extremely
• 4:18
  tricky. And if we connect this to the bigger
• 4:20
  picture, this fear makes sense. The fear of hypothetical
• 4:25
  future AI. Right. It is valid, but it's happening
• 4:29
  right alongside what humans are already doing
• 4:31
  with it today. So Anthropic is trying to legislate
• 4:35
  against future catastrophes. But we are seeing
• 4:37
  real security breaches in the ecosystem right
• 4:40
  now. Oh, absolutely. Let's talk about the economics
• 4:42
  of cyber attacks for a second. The scale of these
• 4:44
  recent breaches is what really stands out. We
• 4:47
  just saw a report about a single hacker. They
• 4:49
  successfully broke into more than 100 companies.
• 4:52
  And these include some very recognizable corporate
• 4:55
  names. The shocking part isn't just the breach
• 4:57
  itself, though. No, it is the underlying cost.
• 5:00
  They pulled off these widespread attacks for
• 5:02
  roughly $50 a hit. $50? You can barely buy dinner
• 5:06
  for two with that. Yet it's funding corporate
• 5:09
  infiltration at scale. Because the automation
• 5:12
  of these tools is driving the cost down. It absolutely
• 5:16
  democratizes hacking. You no longer need a massive
• 5:19
  state -sponsored budget. Exactly. You don't need
• 5:21
  millions to cause serious damage anymore. A teenager
• 5:24
  with 50 bucks can rent automated tools to probe
• 5:26
  vulnerabilities everywhere. It's wild. And speaking
• 5:29
  of security concerns, the internal security at
• 5:31
  major AI labs is under a microscope. You're referring
• 5:34
  to the situation at OpenAI. This is straight
• 5:37
  out of a Cold War spy novel. It really is. An
• 5:40
  OpenAI staffer was actually accused of hiding
• 5:43
  a listening device. Inside the company. And the
• 5:46
  court filing suggests this wasn't just random
• 5:48
  corporate snooping. The staffer reportedly had
• 5:50
  frequent contact with a Saudi royal. Right. And
• 5:53
  then they made a one -way move back to Saudi
• 5:55
  Arabia. Wow. The geopolitical stakes here are
• 5:58
  massive. These foundation models are no longer
• 6:01
  just software products. They are essentially
• 6:04
  being treated as critical national security assets.
• 6:06
  Which brings us to the recent federal actions
• 6:08
  regarding DeepSeek. The global AI ecosystem is
• 6:11
  incredibly volatile right now. It is. DeepSeek
• 6:14
  just got pushed out of U .S. federal civilian
• 6:16
  systems completely. And we are reporting this
• 6:18
  purely on the facts of the policy. The federal
• 6:21
  ban is extensive. Very extensive. It goes much
• 6:24
  further than just deleting the DeepSeek app from
• 6:26
  government phones. It targets the underlying
• 6:28
  infrastructure. The mandate covers third -party
• 6:31
  products built on DeepSeek's models, too. Let's
• 6:34
  explain how deeply that cuts. If a domestic software
• 6:37
  vendor uses their model for back -end processing,
• 6:40
  what happens? That entire software package is
• 6:42
  banned from federal civilian use, even if the
• 6:45
  app itself isn't Chinese. That is incredibly
• 6:48
  difficult to enforce. It's like trying to ban
• 6:50
  a specific brand of flour, but you have to track
• 6:53
  it down inside thousands of different baked cakes.
• 6:56
  The model is deeply embedded in the software
• 6:59
  supply chain. This raises an important question
• 7:01
  about auditing. How will governments actually
• 7:04
  audit the software they buy? They have to trace
• 7:07
  the lineage of every algorithm operating inside
• 7:09
  their networks. How deep does that deep -seek
• 7:12
  federal ban actually go? It requires federal
• 7:15
  IT departments to rip out any tool, even from
• 7:18
  domestic vendors, if the underlying cognitive
• 7:20
  engine relies on deep -seek. It basically blocks
• 7:24
  any software quietly using their models behind
• 7:26
  the scenes. Precisely. So the federal government
• 7:29
  is frantically building walls around cloud -based
• 7:31
  AI. But the consumer market has realized something
• 7:33
  else. Oh, definitely. The only way to guarantee
• 7:36
  security is to pull AI out of the cloud entirely,
• 7:40
  which is why we're seeing a massive explosion
• 7:42
  in local consumer tools. The focus is shifting
• 7:45
  heavily toward privacy and local control. It
• 7:47
  is a complete paradigm shift. We are seeing AI
• 7:51
  become deeply personalized. We really are. I
• 7:54
  have to admit something here. Honestly, I still
• 7:56
  wrestle with prompt drift myself. Oh, yeah. That's
• 7:59
  frustrating. You know, when AI behavior subtly
• 8:02
  shifts over time, ruining your old commands.
• 8:05
  So the fact that we have all these new tools
• 8:07
  is a bit overwhelming. It is completely overwhelming.
• 8:11
  And that's exactly why companies are pivoting.
• 8:14
  They know users like you are experiencing friction
• 8:16
  and fatigue. Notice the trend in these consumer
• 8:19
  updates. They are trying to make it feel safe
• 8:22
  and permanent. ChatGPT finally introduced a hiding
• 8:26
  place for sensitive conversations. They call
• 8:28
  it locked chats. This is a big deal for privacy.
• 8:31
  You can now hide specific threads behind a pin,
• 8:34
  face ID, or a device unlock. It acknowledges
• 8:38
  how people actually use the tool. People are
• 8:41
  using AI for highly personal, sensitive brainstorming.
• 8:44
  They don't want those thoughts sitting open on
• 8:47
  their screen. Or floating unprotected on a server.
• 8:50
  Which leads to the massive push to take AI entirely
• 8:53
  offline. Perplexity Computer is now running locally
• 8:56
  on Mac. No internet required. No paid credits
• 9:00
  required either. It uses its own localized AI
• 9:03
  model for complex tasks. This is a huge shift
• 9:06
  in the relationship between user and machine.
• 9:09
  You are no longer renting brainpower from a distant
• 9:12
  cloud server. You actually own the cognitive
• 9:14
  process on your local hardware. And the adoption
• 9:17
  rates for these consumer tools are staggering.
• 9:20
  People are hungry for this. Look at Meta's new
• 9:22
  app, Muse. It just crossed 10 million US downloads.
• 9:26
  Wow. And it did that in only 40 days. To put
• 9:29
  that in perspective, Muse is now in an exclusive
• 9:31
  club. Only 12 apps in history have ever hit that
• 9:35
  10 million milestone that quickly. The appetite
• 9:38
  is voracious. And the audio space is evolving
• 9:40
  just as fast. Oh, absolutely. Mistral just launched
• 9:43
  VoxTroll 3. It brings AI -powered speech -to
• 9:47
  -text and natural voice generation. And it does
• 9:49
  this in over 50 languages. What's fascinating
• 9:52
  here is that Mistral also offers self -hosting
• 9:54
  options. Again, we see this recurring theme.
• 9:57
  Yeah, letting users and companies control their
• 9:59
  own data locally. The financial markets are certainly
• 10:01
  paying attention to this growth. Otter .ai just
• 10:04
  secured $320 million. They are pushing deeper
• 10:07
  into enterprise tools. They already passed $100
• 10:10
  million in annual recurring revenue. That's massive.
• 10:13
  And they've reached 40 million users. That is
• 10:15
  up from $25 million just last year. We are also
• 10:18
  seeing AI merge heavily with biotech. OpenAI
• 10:21
  just acquired Orchestra Bio. Right. They are
• 10:23
  an AI biotech startup building specialized agents
• 10:26
  for drug development. Their team and their technology
• 10:29
  are going to join Rosalind. The goal is to accelerate
• 10:32
  how fast we can move new treatments through the
• 10:34
  pipeline. And speaking of mass distribution of
• 10:37
  assets, there is a massive equity push happening.
• 10:41
  Yeah, this was interesting. Elon Musk announced
• 10:44
  an initiative to give one SpaceX share to every
• 10:47
  kid in America starting next year. And the rollout
• 10:49
  is potentially happening through Trump accounts.
• 10:51
  Just for context on these broad stock distributions,
• 10:55
  this is part of a wider trend. It is. This comes
• 10:58
  after Anthropic committed 5 % of its own shares
• 11:00
  to a similar public benefit model. We're seeing
• 11:03
  massive private wealth mechanisms intersect with
• 11:05
  public distribution. Why does running perplexity
• 11:08
  locally on a Mac matter so much? Because it completely
• 11:11
  severs the cord to the cloud. Your data never
• 11:14
  leaves your machine, eliminating the risk of
• 11:16
  corporate data harvesting entirely. Total privacy
• 11:19
  without relying on internet servers or paying
• 11:21
  for credits. Exactly. Mid -roll sponsor break,
• 11:25
  script provided separately. So we just talked
• 11:28
  about running complex AI locally on your device.
• 11:31
  We did. But to power all these local agents and
• 11:33
  voice apps, the underlying technology needs to
• 11:35
  change. It needs to be blindingly fast. Speed
• 11:37
  is the ultimate bottleneck for AI right now.
• 11:40
  If it takes too long to think, the illusion breaks.
• 11:43
  Here's where it gets really interesting. A company
• 11:45
  called Inceptions just launched something that
• 11:47
  might shatter that bottleneck. Enter Mercury
• 11:50
  3. Mercury 3 is completely changing the math
• 11:53
  on latency. It's a new language model. Inception
• 11:56
  claims it can match or beat models like GPT -6
• 11:59
  Luna and Claude Haukou 5 .5. But the real story
• 12:03
  isn't just the benchmark intelligence scores.
• 12:05
  It's the raw speed. Yeah. Mercury 3 runs four
• 12:09
  to eight times faster than its top competitors.
• 12:12
  And that speed doesn't come from just adding
• 12:14
  more computer chips. No, it comes from a fundamental
• 12:16
  shift in architecture. Mercury 3 is a diffusion
• 12:20
  -based language model. Okay, hold on. Traditional
• 12:23
  AI writes like I do, one word at a time. Right.
• 12:26
  If Mercury 3 isn't doing that, how on earth is
• 12:29
  it generating text? What exactly is a diffusion
• 12:31
  -based language model? It drafts the whole answer
• 12:34
  at once, not word by word. Let's expand on that.
• 12:38
  If sequential AI is a brick layer building a
• 12:40
  wall one brick at a time, what is Mercury 3?
• 12:43
  Mercury 3's diffusion model is like a sculptor.
• 12:46
  A sculptor? Yeah. It works on the whole statue
• 12:49
  at once. It smooths the head, arms, and legs
• 12:52
  simultaneously until the exact shape emerges.
• 12:55
  It is a completely different paradigm for generating
• 12:57
  text. And the results are staggering. Mercury
• 13:00
  3 is reaching 1 ,068 tokens per second on NVIDIA
• 13:04
  GPUs. Let's define tokens per second. It's how
• 13:07
  fast the AI generates fragments of words in real
• 13:10
  time. Over a thousand is essentially instant
• 13:13
  generation. Whoa. Imagine scaling to a billion
• 13:17
  queries at that speed. It completely changes
• 13:20
  what is possible in real -time computing. Let's
• 13:23
  look at the specs under the hood. It has a one
• 13:25
  million token context window. Meaning the amount
• 13:28
  of text an AI can hold in active memory. Right.
• 13:31
  It can hold massive amounts of information, like
• 13:34
  whole books or code bases. And the cost to run
• 13:37
  it is incredibly low. 20 cents per million input
• 13:41
  tokens, 70 cents per million output tokens. It's
• 13:44
  also 40 times larger than Mercury 1, yet it maintains
• 13:47
  that same blistering speed. Inception isn't the
• 13:50
  only one pushing efficiency, though. No, DeepSeek
• 13:53
  just open -sourced its AI inference engine as
• 13:55
  well. That move helps developers run AI much
• 13:58
  faster across the board. It cuts computing costs.
• 14:01
  And it uses significantly less energy to run
• 14:03
  the models. But going back to Mercury 3... The
• 14:06
  real test of speed is live environments. Inception
• 14:08
  tested it using mind -to -web browser tasks.
• 14:11
  These are tasks where the AI essentially acts
• 14:13
  as a human agent. It navigates a live web browser
• 14:16
  to accomplish a complex goal. Mercury 3 completed
• 14:19
  50 % of 50 live tasks. And in their internal
• 14:22
  agent tests, it outperformed GPT 6 .1 Sol. Wow.
• 14:26
  It used the same number of steps and a similar
• 14:29
  token volume, but the diffusion architecture
• 14:31
  gave it the decisive edge. Speed is absolutely
• 14:34
  becoming the new battleground. Think about it
• 14:36
  from a user perspective. For AI agents, digital
• 14:39
  tutors, or automated customer support, latency
• 14:42
  is death. If an agent has to think for three
• 14:46
  seconds before clicking every single button on
• 14:48
  a web page, a complex task takes forever. Every
• 14:52
  extra second of waiting ruins the illusion of
• 14:54
  intelligence. Mercury 3 is betting that diffusion
• 14:57
  models are the answer. They can deliver strong
• 14:59
  intelligence at a dramatically lower latency.
• 15:03
  Why is raw speed, like a thousand tokens a second,
• 15:06
  the ultimate deciding factor for AI agents? Because
• 15:09
  agents don't just answer questions. They execute
• 15:12
  multi -step plans. If each microstep takes seconds
• 15:16
  to process, the entire workflow grinds to a halt.
• 15:19
  Every safe second makes automated tasks feel
• 15:22
  instant and practically human. Right. And that
• 15:25
  brings us to the core friction of everything
• 15:27
  we've discussed today. We are looking at two
• 15:30
  massive, opposier forces in the world of technology
• 15:33
  right now. Yeah. So what does this all mean?
• 15:36
  If we look at the big picture, we see the first
• 15:38
  force. We have regulators and safety advocates
• 15:40
  trying desperately to throw the brakes on AI.
• 15:43
  Anthropic is trying to codify a literal 50 death
• 15:47
  threshold to preemptively stop dangerous models.
• 15:50
  They want judicial oversight before deployment.
• 15:52
  And governments are handing down massive federal
• 15:55
  bans. They want to lock... insecure models out
• 15:58
  of civilian infrastructure entirely. The geopolitical
• 16:01
  walls are going up around centralized AI. But
• 16:04
  then we look at the second force. The technology
• 16:06
  itself is accelerating at breakneck speeds. And
• 16:09
  it is actively decentralizing. It's moving out
• 16:12
  of their reach. Exactly. We have Perplexity running
• 16:15
  complex queries entirely locally on a Mac. No
• 16:18
  cloud required. We have Mercury 3 generating
• 16:20
  text at a thousand tokens a second. The true
• 16:23
  power is moving from the heavily regulated cloud
• 16:26
  down to the individual hardware device. Which
• 16:29
  leads us with a final thought to ponder. If a
• 16:32
  diffusion model can draft a highly intelligent
• 16:34
  response at over a thousand tokens a second.
• 16:37
  And it can do it entirely offline on your local
• 16:39
  computer. Who is left to evaluate it for a 50
• 16:42
  death risk? That is the real question. Can the
• 16:45
  slow gears of the law ever truly catch up to
• 16:48
  decentralized localized speed? Or are we building
• 16:52
  a rigid regulatory framework for a centralized
• 16:55
  world that is already disappearing? It is a structural
• 16:58
  tension that will define the next decade of technology.
• 17:01
  Thank you for joining us on this deep dive. Keep
• 17:03
  questioning the rapid changes around us because
• 17:05
  the pace is only going to increase. Until next
• 17:07
  time, keep exploring.