Latest / AI Fire Daily / 🎙 EP 368: Trump War Grok Consultation & "Mode-Hopping" Pre-Training Discovery

Transcript

🎙 EP 368: Trump War Grok Consultation & "Mode-Hopping" Pre-Training Discovery · AI Fire Daily

• 0:00
  We live in a bizarre technological paradox today.
• 0:03
  AI chatbots are now actively advising presidents
• 0:06
  on global military operations. Yeah, it really
• 0:09
  is like a wild time to be alive. We desperately
• 0:12
  want these systems to act as perfect oracles,
• 0:15
  but mathematically speaking, they are still highly
• 0:19
  unpredictable experiments. They really are. And
• 0:22
  yet under the hood, these exact same models randomly
• 0:25
  fail. Like they will suddenly forget how to do
• 0:27
  basic math. Exactly. Yeah. For absolutely no
• 0:30
  apparent reason. So welcome to the deep dive.
• 0:33
  Our mission today is quite simple. We want to
• 0:36
  extract the signal from the constant noise. Because
• 0:38
  there is so much noise out there right now. There
• 0:40
  really is. We are analyzing a stack of reports
• 0:43
  on new AI developments today. We are going to
• 0:46
  examine AI's sudden leap into high stakes geopolitics.
• 0:49
  And from there, we explore the bizarre behavioral
• 0:52
  quirks these models are developing. We also unpack
• 0:56
  the sheer security nightmare of watermarking
• 0:58
  synthetic biology. And finally, we will explore
• 1:02
  a massive new discovery in AI training. Which
• 1:05
  is a breakthrough that really challenges everything
• 1:07
  we assumed about how models learn. Right. So
• 1:11
  let us get into it. We really are seeing a massive
• 1:14
  fundamental shift right now. I mean, we are moving
• 1:17
  far beyond models just generating polite corporate
• 1:20
  emails. Yeah, AI is actively shaping the physical
• 1:22
  world at the highest levels. The stakes have
• 1:25
  escalated incredibly fast. They absolutely have.
• 1:28
  According to recent reporting from Time, this
• 1:31
  shift is already happening. Back in December
• 1:33
  2025, President Trump held a highly private meeting.
• 1:36
  Right. He reportedly spent hours consulting Elon
• 1:39
  Musk's Grok AI. They were directly discussing
• 1:42
  the complex geopolitical situation in Venezuela.
• 1:45
  Which is just fascinating. Specifically, he asked
• 1:48
  the model a very high stakes political question.
• 1:51
  He wanted to gauge how the Venezuelan public
• 1:53
  would actually react. He asked what happens if
• 1:57
  President Nicolas Maduro were captured. And Grok
• 2:00
  responded to that prompt with absolute supreme
• 2:03
  confidence. Yeah, it did. It reportedly told
• 2:06
  the president that Maduro was deeply unpopular.
• 2:08
  It suggested that many locals would likely celebrate
• 2:10
  his sudden removal. Which brings us to the actual
• 2:14
  events of January 3rd, 2026. The U .S. launched
• 2:17
  an operation in Venezuela and successfully captured
• 2:21
  Maduro. Right. And just as the model predicted,
• 2:24
  there were indeed public celebrations. Time reported
• 2:27
  a very interesting detail about the immediate
• 2:30
  aftermath. Oh, yeah. A source stated Trump viewed
• 2:34
  Grok as highly capable afterward. Now, let's
• 2:38
  be extremely clear about the reported facts here.
• 2:40
  The reporting does not state that Grok caused
• 2:42
  the invasion. Exactly. The AI did not make the
• 2:46
  final military decision. It was utilized as one
• 2:48
  specific source of geopolitical information.
• 2:51
  Right. It was just used to weigh the potential
• 2:53
  political consequences of action. But it highlights
• 2:56
  a massive transition in modern global governance.
• 2:59
  It's kind of like relying on a supercomputer
• 3:02
  to read a geopolitical crystal ball. Yet you
• 3:05
  are asking a machine to predict human emotion
• 3:07
  on a massive scale. And the Pentagon is already
• 3:10
  using something called GovGrok internally. They
• 3:13
  are deploying it across various defense and military
• 3:15
  operations. The connections run very deep across
• 3:18
  the entire tech industry, too. OpenAI and Anthropic
• 3:22
  also maintain significant relationships with
• 3:24
  the U .S. defense establishment. This integration
• 3:27
  is rapidly becoming the new normal for global
• 3:29
  superpowers. Which brings up a very real mechanical
• 3:32
  concern about these systems. When a human intelligence
• 3:36
  analyst writes a briefing, they include self
• 3:39
  -doubt. Right. They use confidence intervals.
• 3:41
  Exactly. They flag when data is highly uncertain.
• 3:44
  Large language models do not possess that kind
• 3:47
  of human hesitation. Yeah, that is the core structural
• 3:50
  problem with LLMs. They are ultimately just probabilistic
• 3:54
  token prediction engines. Right. They are mathematically
• 3:56
  designed to generate the most likely next word.
• 3:59
  They are not actually designed to evaluate fundamental
• 4:01
  objective truth. So they deliver every single
• 4:05
  answer with the exact same unyielding authority.
• 4:08
  But given how AI can hallucinate, aren't we risking
• 4:13
  international crises if a model is just overly
• 4:15
  confident in its geopolitical assumptions? Yes,
• 4:18
  absolutely. The distinct danger is that AI models
• 4:21
  become the ultimate yes -men. If a defense system
• 4:24
  accepts a confidently hallucinated output as
• 4:26
  fact, The cascading effects are terrifying. It
• 4:30
  completely removes the crucial human hesitation
• 4:32
  needed to prevent catastrophic escalation. So
• 4:35
  hallucinated confidence could literally spark
• 4:38
  an international crisis if we aren't careful.
• 4:40
  Right. And that exact vulnerability is shaking
• 4:43
  up the tech world. Security is suddenly the loudest
• 4:46
  conversation in every single boardroom. If world
• 4:48
  leaders are trusting these models with matters
• 4:51
  of war, the obvious question becomes security.
• 4:55
  How secure is the digital foundation we are currently
• 4:58
  building on? Yeah. Because recent reports suggest
• 5:02
  we are building on absolute quicksand. The major
• 5:06
  tech companies certainly realize how precarious
• 5:08
  this all is. OpenAI, Google, Anthropic, Meta,
• 5:11
  NVIDIA, and XAI recently made a move. Right,
• 5:15
  the new accord. Yeah, they all signed the new
• 5:17
  White House Superintelligence Accord. They voluntarily
• 5:20
  committed to rigorous outside audits for their
• 5:23
  advanced systems. It was presented to the public
• 5:25
  as a massive step forward. Tech CEOs proudly
• 5:28
  shared the accord across their social media platforms.
• 5:31
  Oh, yeah, they did. But then users instantly
• 5:35
  spotted a typo on the official signatory page.
• 5:38
  Yeah, that really happened. Nothing kills the
• 5:41
  superintelligence vibe faster than a basic human
• 5:44
  typo. Exactly. It is deeply ironic for a document
• 5:47
  governing the most advanced intelligence. It
• 5:49
  really is. But the actual security threats surrounding
• 5:53
  these models are not funny at all. No, not at
• 5:56
  all. Google just reported some highly alarming
• 5:59
  numbers on the cybersecurity front. Publicly
• 6:03
  reported software vulnerabilities literally doubled
• 6:05
  in August alone. They hit 10 ,740 distinct vulnerabilities
• 6:11
  in a single month. Hackers are increasingly using
• 6:15
  AI agents to weaponize newly patched flaws. It
• 6:17
  is wild. The defensive tools are getting smarter,
• 6:20
  but the digital weapons are scaling faster. It
• 6:22
  is a terrifying, invisible arms race happening
• 6:26
  in the background. And it is not just limited
• 6:28
  to traditional software vulnerabilities anymore.
• 6:30
  We also have a rising threat on the synthetic
• 6:33
  biological front. AI -generated synthetic biology
• 6:36
  is getting much harder to effectively track.
• 6:39
  DeepMind recognized this specific danger and
• 6:42
  tried to build a solution. They recently built
• 6:44
  a new tool called SynthID Bio. It is a system
• 6:47
  specifically designed to tag synthetic biology
• 6:50
  sequences. If you are listening and wondering
• 6:52
  how you watermark physical biology, it is fascinating.
• 6:56
  DNA is essentially just biological code. Exactly.
• 6:59
  And much like human language, there are different
• 7:01
  ways to write the same thing. Right. In biology,
• 7:04
  certain different DNA sequences will produce
• 7:07
  the exact same protein. These are called synonymous
• 7:09
  codons. Okay. SynthID Bio subtly alters those
• 7:13
  specific synonymous codons. So it inserts a hidden
• 7:16
  digital barcode into the physical biological
• 7:19
  design. And crucially, it does this without hurting
• 7:22
  the actual lab performance. Yeah, the organism
• 7:24
  functions identically, but it carries a hidden
• 7:26
  cryptographic signature. This is incredibly vital
• 7:30
  for the future of global biosecurity. we desperately
• 7:33
  need a standardized way to track lab -created
• 7:35
  biological materials. We really do. With DeepMind's
• 7:39
  SynthID Bio, is watermarking biology actually
• 7:43
  going to stop a determined rogue lab from doing
• 7:45
  damage? Not entirely. It is really more about
• 7:50
  creating significant friction in the system.
• 7:53
  Okay. Watermarking establishes a very clear trail
• 7:56
  of breadcrumbs back to the origin. A highly determined
• 7:59
  rogue lab could potentially try to strip the
• 8:01
  watermark out. Right. But it forces them to work
• 8:04
  much harder and leaves complex forensic clues
• 8:06
  behind. Essentially, it's about tracking the
• 8:09
  origin, not completely stopping a determined
• 8:12
  rogue lab. Exactly. We are trying to build fences
• 8:16
  in the dark right now. The models are evolving
• 8:18
  wildly and they are developing bizarre new behaviors.
• 8:21
  Yeah, their outputs morph without us explicitly
• 8:23
  coding them to do so. We used to rely on... Very
• 8:27
  simple tells to spot AI text. Oh, yeah. The classic
• 8:32
  em dash giveaway was the gold standard for a
• 8:34
  while. If you saw a paragraph flooded with em
• 8:36
  dashes, it was probably a chatbot. But the newer
• 8:38
  models are rapidly changing their internal linguistic
• 8:41
  habits. Opus 5 .5 has developed a brand new,
• 8:45
  highly specific behavioral tell. Right. It used
• 8:48
  the exact phrase, this matters, 13 ,000 times
• 8:52
  in recent data. That is so wild to me. It organically
• 8:55
  adopted a human rhetorical crutch all on its
• 8:58
  own. And the new Astra model has developed its
• 9:00
  own weird linguistic tells, too. They drift over
• 9:03
  time in highly unpredictable and frustrating
• 9:06
  ways. Honestly, I still wrestle with prompt drifts
• 9:09
  myself. Yeah, it happens to everyone. You get
• 9:11
  a complex prompt working perfectly for your workflow
• 9:14
  one week. Then the underlying model suddenly
• 9:16
  shifts and the output is suddenly completely
• 9:19
  useless. It is incredibly frustrating for daily
• 9:22
  users and enterprise developers alike. These
• 9:24
  advanced models are not static pieces of traditional
• 9:28
  software at all. Right. They operate much more
• 9:30
  like living, breathing digital ecosystems. And
• 9:33
  sometimes those fragile ecosystems just completely
• 9:36
  collapse in on themselves. Which brings us perfectly
• 9:39
  to our final and perhaps most unsettling topic.
• 9:42
  We are currently building our infrastructure.
• 9:45
  on top of these shifting tools. We are. For instance,
• 9:48
  Grok Primary Bot proactively finds complex tasks
• 9:51
  it can handle. It actually suggests ways to help
• 9:54
  you without hitting your usage limits. And Cloudflare
• 9:58
  Clef just introduced two brand new open source
• 10:01
  AI decision models. They deliver incredibly fast,
• 10:04
  probability -based answers for AI agents and
• 10:06
  automated trading bots. We are deploying these
• 10:09
  models to make thousands of autonomous decisions
• 10:11
  daily. But... AI researchers just discovered
• 10:14
  a massive fundamental flaw in their architecture.
• 10:17
  Yeah. We do not actually understand how they
• 10:20
  truly learn to reason. It all comes down to the
• 10:23
  pre -training phase of model development. This
• 10:25
  is where the magic and clearly the mystery actually
• 10:28
  happens. It is. The concept of pre -training
• 10:31
  is usually quite simple. Feeding an AI vast amounts
• 10:34
  of text to learn language rules. Perfect. Now,
• 10:38
  the baseline assumption in the tech industry
• 10:40
  has always been linear. We assumed more pre -training
• 10:43
  data inherently equals better generalization
• 10:45
  and deeper reasoning. Just keep feeding the machine
• 10:48
  more data and it gets smarter. Right. But a new
• 10:52
  research paper found something surprisingly unstable
• 10:54
  inside LLMs. They call this deeply chaotic internal
• 10:58
  phenomenon mode hopping. The specific data they
• 11:01
  found in the OMO3 -32B model is absolutely wild.
• 11:06
  We really need to look closely at the actual
• 11:08
  performance numbers here. Okay, let's unpack
• 11:10
  the timeline of this model's data training. Yeah.
• 11:14
  At 2 .17 trillion tokens of training data, the
• 11:17
  model had 81 % accuracy. That is a very solid
• 11:21
  expected score for that exact stage of pre -training.
• 11:24
  But then at 2 .19 trillion tokens, it dropped
• 11:28
  to 0 % accuracy. Complete catastrophic performance
• 11:32
  collapse. 0%. It functionally forgot how to do
• 11:36
  everything it had just learned. But then just
• 11:38
  slightly later at 2 .21 trillion tokens, it jumped
• 11:41
  back up. It hit 81 .7 % accuracy as if nothing
• 11:46
  had ever happened. Whoa, imagine scaling to a
• 11:49
  billion queries on a live enterprise system and
• 11:52
  the model suddenly hits one of those mathematical
• 11:54
  dead zones. Right. The entire automated system
• 11:57
  just... breaks completely without any warning
• 11:59
  whatsoever. Within just 40 billion more training
• 12:03
  tokens, the core performance collapsed to zero.
• 12:06
  Then it almost fully recovered its complex reasoning
• 12:09
  abilities just as fast. Yeah. It is literally
• 12:12
  like stacking Lego blocks of data and the tower
• 12:15
  randomly vanishing. To understand this chaotic
• 12:18
  behavior, researchers carefully examined different
• 12:21
  model checkpoints. A checkpoint is basically
• 12:23
  a saved structural snapshot of the model during
• 12:26
  its training. They compared an earlier $4 .5
• 12:29
  trillion token checkpoint with a later $4 .9
• 12:32
  trillion token checkpoint. They ran both of them
• 12:35
  through a brutally tough benchmark after doing
• 12:37
  math fine -tuning. Logically, you would completely
• 12:40
  expect the later model to be much smarter. It
• 12:43
  has processed significantly more high -quality
• 12:45
  data than the earlier version. Right. But the
• 12:49
  $4 .5 trillion checkpoint scored 36 .3 % on the
• 12:53
  GPQA benchmark. And the later 4 .9 trillion checkpoint
• 12:57
  only scored 29 .8%. The later model was actually
• 13:01
  a much worse starting point for fine -tuning.
• 13:03
  Wow. It fundamentally lost its robustness. The
• 13:07
  model literally saw more data, yet its foundational
• 13:10
  reasoning became measurably dumber. So what is
• 13:14
  mathematically happening inside the neural network
• 13:16
  to cause this collapse? The researchers believe
• 13:19
  different internal learning strategies constantly
• 13:21
  compete during the pre -training phase. It is
• 13:24
  a brutal internal tug of war for the model's
• 13:26
  actual brain. We often visualize model training
• 13:29
  as walking down a sloping mountain range. You
• 13:32
  are trying to find the absolute lowest point
• 13:34
  in the valley. That lowest point represents the
• 13:37
  lowest error rate. Exactly. Sometimes the model
• 13:40
  leans on deep, highly generalizable reasoning
• 13:43
  to get there. It actually builds a structural,
• 13:45
  logical understanding of the complex problem.
• 13:48
  But other times, easier, shortcut -like patterns
• 13:51
  take over the neural network. The model finds
• 13:54
  an incredibly cheap mathematical trick to just
• 13:56
  guess the right answer. Yes. Why would an AI
• 14:00
  suddenly abandon deep reasoning to take a shortcut
• 14:03
  after it already learned how to solve the problem?
• 14:06
  Because the only mathematical goal of training
• 14:09
  is reducing the immediate error rate. The model
• 14:12
  discovers a highly superficial pattern that works
• 14:14
  perfectly for a specific batch of data. Okay.
• 14:17
  So it lazily adopts that shortcut, entirely overriding
• 14:21
  the deeper reasoning it previously built. It
• 14:23
  gets lazy choosing a quick pattern over deep
• 14:26
  logic just to minimize errors? Yes. And this
• 14:29
  creates a major, almost entirely invisible problem
• 14:32
  for AI developers. The training loss metric can
• 14:35
  keep improving steadily on their dashboard. Right.
• 14:38
  It visually looks like the model is getting smarter
• 14:40
  every single day. But the model's actual logical
• 14:43
  reasoning behavior quietly gets worse in the
• 14:45
  background. The final checkpoint of a model may
• 14:48
  not always be the best checkpoint. This raises
• 14:50
  a profoundly important question about how we
• 14:52
  currently evaluate these systems. We might be
• 14:55
  widely deploying internally compromised models
• 14:57
  without even realizing it. We absolutely might
• 14:59
  be. We are going to take a quick second for a
• 15:02
  word from our sponsor. Sounds good. Okay, let's
• 15:06
  synthesize this entire journey and look at the
• 15:08
  actual big picture. We are placing immense...
• 15:12
  almost blind trust in these AI systems today.
• 15:14
  We really are. We use advanced cursor coding
• 15:17
  tools powered by GLM 5 .3 for our daily software.
• 15:21
  We literally have the Oval Office making geopolitical
• 15:24
  moves based on probabilistic chatbot outputs.
• 15:27
  Yeah, it is everywhere. Yet the foundational
• 15:30
  architecture of these models remains remarkably
• 15:33
  unstable underneath it all. It can mode hop into
• 15:36
  the total failure instantly. And it does this
• 15:38
  just by reading a few billion more words. The
• 15:42
  overarching takeaway from all these reports is
• 15:44
  quite clear and honestly a little daunting. The
• 15:47
  sheer raw capability of AI is scaling incredibly
• 15:50
  fast right now, but it is scaling much faster
• 15:53
  than our fundamental understanding of it. We
• 15:56
  absolutely do not fully grasp its inner workings
• 15:58
  yet. Think about this as you go about your day.
• 16:01
  If the final checkpoint of an AI isn't always
• 16:04
  the smartest one, what happens next? That is
• 16:08
  the real question. What happens if a government
• 16:10
  or defense agency accidentally deploys a model
• 16:13
  right as it hits a mode -hopping slump? It is
• 16:15
  kind of terrifying to think about. Are we absolutely
• 16:18
  sure the model giving the advice today is as
• 16:21
  smart as the one from yesterday? Thank you for
• 16:25
  taking this deep dive with us. Keep questioning
• 16:27
  the invisible systems around you. Stay curious.
• 16:30
  We will see you next time.