Latest / AGI Dreams – Open, Uncensored, & Local - AI News Digest / AGI Dreams Podcast – May 05, 2026

Transcript

AGI Dreams Podcast – May 05, 2026 · AGI Dreams – Open, Uncensored, & Local - AI News Digest

• 0:00
  A
• 0:00
  position
• 0:00
  paper
• 0:00
  from
• 0:00
  NVIDIA
• 0:01
  and
• 0:01
  Johns
• 0:01
  Hopkins
• 0:02
  researchers
• 0:03
  lays
• 0:03
  out
• 0:03
  an
• 0:04
  architectural
• 0:04
  blueprint
• 0:05
  yet
• 0:05
  for
• 0:05
  defending
• 0:05
  AI
• 0:06
  agents
• 0:06
  against
• 0:06
  indirect
• 0:07
  prompt
• 0:07
  injection
• 0:08
  —
• 0:09
  the
• 0:09
  attack
• 0:09
  class
• 0:10
  where
• 0:10
  malicious
• 0:10
  instructions
• 0:11
  hidden
• 0:11
  in
• 0:11
  emails,
• 0:12
  web
• 0:12
  pages,
• 0:13
  or
• 0:13
  tool
• 0:13
  outputs
• 0:14
  hijack
• 0:14
  agent
• 0:14
  behavior.
• 0:16
  The
• 0:16
  paper's
• 0:16
  central
• 0:16
  claim
• 0:16
  is
• 0:17
  uncomfortable
• 0:17
  but
• 0:17
  honest:
• 0:18
  static
• 0:18
  security
• 0:19
  policies
• 0:19
  are
• 0:19
  insufficient.
• 0:20
  When
• 0:20
  a
• 0:21
  coding
• 0:21
  agent
• 0:21
  hits
• 0:21
  a
• 0:21
  deprecated
• 0:22
  API
• 0:22
  and
• 0:22
  needs
• 0:23
  to
• 0:23
  replan,
• 0:24
  or
• 0:24
  a
• 0:24
  debugging
• 0:24
  agent
• 0:25
  discovers
• 0:25
  the
• 0:25
  crash
• 0:25
  evidence
• 0:26
  lives
• 0:26
  in
• 0:26
  a
• 0:26
  log
• 0:26
  file
• 0:27
  nobody
• 0:27
  anticipated,
• 0:28
  the
• 0:28
  security
• 0:28
  boundary
• 0:29
  has
• 0:29
  to
• 0:29
  flex.
• 0:30
  But
• 0:30
  the
• 0:30
  moment
• 0:31
  plans
• 0:31
  and
• 0:31
  policies
• 0:31
  become
• 0:32
  dynamic,
• 0:33
  untrusted
• 0:33
  environmental
• 0:34
  data
• 0:34
  can
• 0:34
  influence
• 0:35
  them
• 0:35
  —
• 0:35
  and
• 0:36
  that
• 0:36
  is
• 0:36
  the
• 0:36
  fundamental
• 0:36
  tension
• 0:37
  the
• 0:37
  paper
• 0:37
  tries
• 0:37
  to
• 0:37
  resolve.
• 0:38
  The
• 0:39
  proposed
• 0:39
  architecture
• 0:40
  splits
• 0:40
  agent
• 0:40
  execution
• 0:41
  into
• 0:41
  six
• 0:41
  components:
• 0:42
  an
• 0:42
  orchestrator
• 0:43
  that
• 0:43
  generates
• 0:43
  plans
• 0:44
  and
• 0:44
  policies,
• 0:45
  a
• 0:45
  plan/policy
• 0:46
  approver
• 0:46
  with
• 0:46
  optional
• 0:47
  human
• 0:47
  escalation,
• 0:48
  an
• 0:48
  executor,
• 0:49
  a
• 0:49
  policy
• 0:50
  enforcer,
• 0:51
  the
• 0:51
  environment,
• 0:52
  and
• 0:52
  a
• 0:52
  feedback
• 0:52
  loop
• 0:52
  enabling
• 0:53
  iterative
• 0:53
  updates.
• 0:55
  The
• 0:55
  key
• 0:55
  insight
• 0:55
  lives
• 0:56
  in
• 0:56
  Position
• 0:56
  2:
• 0:57
  when
• 0:57
  LLMs
• 0:57
  must
• 0:58
  participate
• 0:58
  in
• 0:58
  security
• 0:59
  decisions
• 1:00
  (and
• 1:00
  the
• 1:00
  authors
• 1:00
  argue
• 1:01
  they
• 1:01
  must,
• 1:02
  because
• 1:02
  the
• 1:02
  complexity
• 1:03
  exceeds
• 1:03
  what
• 1:03
  rule-based
• 1:04
  checks
• 1:04
  can
• 1:04
  express),
• 1:05
  the
• 1:06
  system
• 1:06
  should
• 1:06
  feed
• 1:06
  the
• 1:06
  model
• 1:07
  only
• 1:07
  narrowly
• 1:08
  scoped,
• 1:08
  structured
• 1:09
  artifacts
• 1:09
  —
• 1:10
  typed
• 1:10
  diffs,
• 1:11
  proposed
• 1:11
  plan
• 1:12
  changes,
• 1:13
  provenance
• 1:13
  traces
• 1:13
  —
• 1:14
  never
• 1:15
  raw
• 1:15
  attacker-controlled
• 1:16
  text.
• 1:17
  Two
• 1:17
  concrete
• 1:17
  proposals
• 1:18
  follow.
• 1:19
  First,
• 1:19
  decouple
• 1:19
  instruction
• 1:20
  recognition
• 1:20
  from
• 1:20
  instruction-following:
• 1:22
  make
• 1:22
  the
• 1:22
  model
• 1:22
  explicitly
• 1:23
  verbalize
• 1:23
  which
• 1:24
  instructions
• 1:24
  it
• 1:24
  intends
• 1:24
  to
• 1:24
  follow,
• 1:25
  then
• 1:25
  trace
• 1:26
  provenance
• 1:26
  and
• 1:26
  apply
• 1:26
  system-level
• 1:27
  policy.
• 1:28
  Second,
• 1:29
  use
• 1:29
  an
• 1:29
  LLM
• 1:29
  to
• 1:29
  synthesize
• 1:30
  step-specific
• 1:31
  programmatic
• 1:31
  validators
• 1:31
  for
• 1:32
  environment
• 1:32
  responses
• 1:33
  rather
• 1:33
  than
• 1:33
  trusting
• 1:34
  the
• 1:34
  executor
• 1:34
  to
• 1:34
  handle
• 1:34
  adversarial
• 1:35
  content
• 1:35
  directly.
• 1:37
  The
• 1:37
  paper's
• 1:37
  sharpest
• 1:37
  contribution
• 1:38
  may
• 1:38
  be
• 1:38
  its
• 1:39
  benchmark
• 1:39
  critique.
• 1:41
  AgentDojo,
• 1:41
  the
• 1:41
  most
• 1:42
  popular
• 1:42
  agent-security
• 1:43
  benchmark,
• 1:44
  contains
• 1:44
  only
• 1:44
  6
• 1:44
  of
• 1:45
  97
• 1:45
  tasks
• 1:45
  that
• 1:46
  require
• 1:46
  replanning.
• 1:47
  Every
• 1:47
  benchmark
• 1:48
  uses
• 1:48
  static,
• 1:49
  non-adaptive
• 1:49
  attack
• 1:50
  payloads.
• 1:51
  The
• 1:51
  authors
• 1:51
  argue
• 1:51
  this
• 1:52
  creates
• 1:52
  "a
• 1:52
  false
• 1:53
  sense
• 1:53
  of
• 1:53
  both
• 1:53
  utility
• 1:54
  and
• 1:54
  security"
• 1:55
  and
• 1:55
  call
• 1:55
  for
• 1:55
  pluggable
• 1:56
  adaptive
• 1:56
  attackers
• 1:56
  —
• 1:57
  RL-trained
• 1:58
  adversary
• 1:58
  models
• 1:59
  or
• 1:59
  genetic-algorithm-style
• 2:00
  payload
• 2:00
  refinement
• 2:01
  —
• 2:02
  as
• 2:02
  a
• 2:02
  minimum
• 2:02
  standard.
• 2:03
  On
• 2:03
  the
• 2:03
  tooling
• 2:04
  side,
• 2:04
  Vercel
• 2:05
  open-sourced
• 2:05
  DeepSec,
• 2:06
  a
• 2:06
  security
• 2:07
  harness
• 2:07
  that
• 2:07
  runs
• 2:08
  Opus
• 2:08
  4.7
• 2:09
  and
• 2:09
  GPT
• 2:09
  5.5
• 2:11
  at
• 2:11
  maximum
• 2:11
  effort
• 2:12
  against
• 2:12
  production
• 2:12
  codebases.
• 2:13
  The
• 2:13
  pipeline
• 2:14
  —
• 2:15
  static
• 2:15
  regex
• 2:15
  scan,
• 2:16
  agent
• 2:16
  investigation,
• 2:17
  validation
• 2:18
  pass
• 2:18
  to
• 2:19
  reduce
• 2:19
  false
• 2:19
  positives,
• 2:20
  git-metadata-based
• 2:21
  triage
• 2:21
  —
• 2:22
  scales
• 2:22
  to
• 2:23
  1,000+
• 2:24
  concurrent
• 2:25
  sandboxes
• 2:25
  on
• 2:25
  Vercel's
• 2:26
  own
• 2:26
  infrastructure.
• 2:27
  The
• 2:27
  false
• 2:28
  positive
• 2:28
  rate
• 2:28
  sits
• 2:28
  at
• 2:29
  10–20%,
• 2:30
  which
• 2:31
  Vercel
• 2:31
  considers
• 2:31
  acceptable
• 2:32
  given
• 2:32
  the
• 2:32
  severity
• 2:33
  of
• 2:33
  true
• 2:33
  positives.
• 2:34
  Dub.co's
• 2:35
  founder
• 2:35
  noted
• 2:36
  that
• 2:36
  DeepSec
• 2:36
  "surfaced
• 2:37
  the
• 2:37
  kind
• 2:37
  of
• 2:37
  issues
• 2:38
  we'd
• 2:38
  actually
• 2:38
  want
• 2:38
  a
• 2:38
  security
• 2:39
  engineer
• 2:39
  to
• 2:39
  flag"
• 2:40
  —
• 2:41
  a
• 2:41
  bar
• 2:41
  most
• 2:41
  automated
• 2:42
  scanners
• 2:42
  still
• 2:42
  fail
• 2:43
  to
• 2:43
  clear.
• 2:44
  The
• 2:44
  tool
• 2:44
  works
• 2:44
  with
• 2:44
  off-the-shelf
• 2:45
  models;
• 2:46
  the
• 2:46
  "cyber"
• 2:46
  fine-tunes
• 2:47
  from
• 2:47
  Anthropic
• 2:48
  and
• 2:48
  OpenAI
• 2:48
  are
• 2:49
  supported
• 2:49
  but
• 2:49
  not
• 2:49
  required,
• 2:50
  as
• 2:50
  DeepSec's
• 2:51
  built-in
• 2:51
  classifier
• 2:52
  checks
• 2:52
  for
• 2:52
  refusals
• 2:53
  after
• 2:53
  each
• 2:53
  research
• 2:54
  step
• 2:54
  and
• 2:55
  finds
• 2:55
  them
• 2:55
  to
• 2:55
  be
• 2:55
  a
• 2:55
  non-issue
• 2:56
  for
• 2:56
  both
• 2:56
  models.
• 2:59
  Anthropic
• 3:00
  shipped
• 3:00
  Claude
• 3:00
  for
• 3:00
  Creative
• 3:01
  Work
• 3:01
  with
• 3:01
  nine
• 3:01
  official
• 3:02
  Model
• 3:02
  Context
• 3:02
  Protocol
• 3:03
  (MCP)
• 3:03
  connectors,
• 3:04
  including
• 3:04
  a
• 3:04
  flagship
• 3:05
  Blender
• 3:05
  integration.
• 3:06
  These
• 3:06
  are
• 3:06
  not
• 3:06
  copy-paste
• 3:07
  plugins
• 3:07
  —
• 3:08
  each
• 3:08
  connector
• 3:08
  maintains
• 3:09
  persistent
• 3:09
  context
• 3:10
  within
• 3:10
  the
• 3:10
  host
• 3:10
  application's
• 3:11
  own
• 3:11
  data
• 3:11
  model,
• 3:12
  letting
• 3:12
  Claude
• 3:12
  read
• 3:13
  live
• 3:13
  project
• 3:13
  state
• 3:13
  and
• 3:13
  execute
• 3:14
  actions
• 3:14
  directly.
• 3:15
  The
• 3:15
  Blender
• 3:16
  connector
• 3:16
  can
• 3:16
  manipulate
• 3:17
  objects,
• 3:17
  materials,
• 3:18
  and
• 3:18
  render
• 3:18
  settings
• 3:19
  without
• 3:19
  leaving
• 3:19
  the
• 3:19
  3D
• 3:20
  viewport.
• 3:21
  This
• 3:21
  is
• 3:21
  one
• 3:22
  of
• 3:22
  the
• 3:22
  first
• 3:22
  production-scale
• 3:22
  deployments
• 3:23
  where
• 3:23
  an
• 3:23
  LLM
• 3:23
  maintains
• 3:24
  persistent
• 3:24
  context
• 3:25
  within
• 3:25
  a
• 3:25
  host
• 3:25
  application's
• 3:26
  native
• 3:26
  data
• 3:26
  structures,
• 3:27
  and
• 3:27
  if
• 3:27
  the
• 3:27
  pattern
• 3:27
  holds,
• 3:28
  it
• 3:28
  probably
• 3:28
  becomes
• 3:29
  the
• 3:29
  template
• 3:29
  for
• 3:29
  how
• 3:29
  agents
• 3:30
  integrate
• 3:30
  with
• 3:30
  domain-specific
• 3:31
  professional
• 3:31
  software
• 3:32
  going
• 3:32
  forward.
• 3:33
  Community
• 3:33
  response
• 3:34
  immediately
• 3:34
  surfaced
• 3:34
  the
• 3:35
  wish
• 3:35
  list:
• 3:35
  CAD
• 3:36
  tools
• 3:36
  (FreeCAD
• 3:36
  for
• 3:36
  Linux
• 3:37
  users
• 3:37
  locked
• 3:37
  out
• 3:37
  of
• 3:37
  Fusion),
• 3:38
  building
• 3:38
  plan
• 3:38
  generation,
• 3:39
  and
• 3:39
  video
• 3:39
  editing.
• 3:40
  The
• 3:40
  MCP
• 3:41
  ecosystem
• 3:41
  continues
• 3:42
  to
• 3:42
  grow
• 3:42
  in
• 3:42
  parallel.
• 3:43
  n8n
• 3:44
  released
• 3:44
  a
• 3:44
  new
• 3:44
  MCP
• 3:45
  server
• 3:45
  designed
• 3:45
  specifically
• 3:46
  for
• 3:46
  agentic
• 3:46
  coding
• 3:47
  tools
• 3:47
  like
• 3:47
  Claude
• 3:47
  Code
• 3:48
  and
• 3:48
  Codex.
• 3:49
  The
• 3:49
  key
• 3:49
  design
• 3:49
  choice:
• 3:50
  TypeScript-based
• 3:51
  validation
• 3:51
  instead
• 3:51
  of
• 3:51
  JSON.
• 3:52
  A
• 3:52
  prompt
• 3:53
  describing
• 3:53
  a
• 3:53
  desired
• 3:54
  automation
• 3:54
  gets
• 3:54
  parsed
• 3:54
  for
• 3:55
  intent,
• 3:55
  the
• 3:55
  MCP
• 3:56
  server
• 3:56
  resolves
• 3:57
  the
• 3:57
  required
• 3:57
  node
• 3:57
  types,
• 3:58
  the
• 3:58
  coding
• 3:58
  agent
• 3:59
  writes
• 3:59
  TypeScript
• 3:59
  that
• 3:59
  compiles
• 4:00
  and
• 4:00
  validates
• 4:00
  before
• 4:01
  being
• 4:01
  converted
• 4:01
  to
• 4:01
  JSON
• 4:02
  and
• 4:02
  deployed
• 4:02
  to
• 4:03
  a
• 4:03
  running
• 4:03
  n8n
• 4:03
  instance
• 4:04
  —
• 4:04
  cloud
• 4:05
  or
• 4:05
  self-hosted.
• 4:06
  For
• 4:07
  workflows
• 4:07
  that
• 4:07
  benefit
• 4:08
  from
• 4:08
  deterministic
• 4:08
  orchestration
• 4:09
  rather
• 4:09
  than
• 4:10
  end-to-end
• 4:10
  agent
• 4:10
  reasoning,
• 4:11
  this
• 4:11
  is
• 4:11
  a
• 4:11
  pragmatic
• 4:12
  middle
• 4:12
  ground.
• 4:14
  The
• 4:14
  governance
• 4:14
  question
• 4:14
  keeps
• 4:15
  surfacing
• 4:15
  alongside
• 4:16
  adoption.
• 4:17
  Architecture
• 4:17
  Decision
• 4:18
  Records
• 4:18
  (ADRs)
• 4:18
  —
• 4:19
  structured
• 4:20
  documents
• 4:20
  capturing
• 4:20
  what
• 4:21
  was
• 4:21
  decided,
• 4:21
  why,
• 4:22
  and
• 4:22
  what
• 4:22
  it
• 4:22
  supersedes
• 4:23
  —
• 4:24
  are
• 4:24
  finding
• 4:24
  new
• 4:24
  relevance
• 4:25
  in
• 4:25
  agentic
• 4:25
  engineering.
• 4:26
  With
• 4:26
  the
• 4:27
  Ruflo
• 4:27
  ADR
• 4:27
  plugin
• 4:28
  for
• 4:28
  Claude
• 4:28
  Code,
• 4:29
  each
• 4:29
  ADR
• 4:29
  is
• 4:30
  stored
• 4:30
  as
• 4:30
  a
• 4:30
  graph
• 4:30
  node
• 4:30
  in
• 4:31
  AgentDB,
• 4:32
  linked
• 4:32
  through
• 4:32
  relationships
• 4:33
  like
• 4:33
  "depends
• 4:33
  on,"
• 4:34
  "amends,"
• 4:34
  and
• 4:34
  "supersedes."
• 4:36
  The
• 4:36
  plugin
• 4:37
  scans
• 4:37
  diffs
• 4:37
  against
• 4:37
  accepted
• 4:38
  ADRs
• 4:38
  and
• 4:39
  flags
• 4:39
  mismatches,
• 4:40
  closing
• 4:40
  the
• 4:40
  drift
• 4:40
  gap
• 4:41
  between
• 4:41
  code
• 4:41
  and
• 4:41
  intent
• 4:42
  that
• 4:42
  compounds
• 4:42
  rapidly
• 4:43
  when
• 4:43
  autonomous
• 4:43
  agents
• 4:43
  are
• 4:44
  modifying
• 4:44
  a
• 4:44
  codebase.
• 4:47
  The
• 4:47
  Qwen
• 4:47
  team
• 4:47
  released
• 4:47
  Qwen-Scope,
• 4:48
  a
• 4:48
  collection
• 4:49
  of
• 4:49
  Sparse
• 4:49
  Autoencoders
• 4:50
  (SAEs)
• 4:50
  for
• 4:50
  the
• 4:51
  Qwen
• 4:51
  3.5
• 4:51
  family
• 4:52
  spanning
• 4:52
  2B
• 4:52
  to
• 4:53
  35B
• 4:53
  MoE.
• 4:54
  SAEs
• 4:54
  decompose
• 4:55
  a
• 4:55
  model's
• 4:55
  internal
• 4:56
  activations
• 4:56
  into
• 4:56
  interpretable
• 4:57
  features
• 4:57
  —
• 4:58
  think
• 4:58
  of
• 4:58
  them
• 4:58
  as
• 4:58
  a
• 4:58
  dictionary
• 4:59
  of
• 4:59
  the
• 4:59
  model's
• 4:59
  concepts.
• 5:00
  Instead
• 5:01
  of
• 5:01
  opaque
• 5:01
  floating-point
• 5:02
  vectors,
• 5:02
  you
• 5:03
  can
• 5:03
  identify
• 5:03
  specific
• 5:04
  feature
• 5:04
  IDs
• 5:05
  that
• 5:05
  correspond
• 5:05
  to
• 5:05
  things
• 5:05
  like
• 5:06
  "legal
• 5:06
  talk,"
• 5:06
  "Python
• 5:07
  code,"
• 5:07
  or
• 5:07
  "refusal."
• 5:09
  This
• 5:09
  is
• 5:09
  the
• 5:09
  largest
• 5:09
  official
• 5:10
  interpretability
• 5:10
  release
• 5:11
  for
• 5:11
  an
• 5:11
  open-weight
• 5:11
  model
• 5:12
  to
• 5:12
  date:
• 5:12
  GemmaScope
• 5:13
  from
• 5:13
  Google
• 5:13
  only
• 5:14
  covered
• 5:14
  models
• 5:14
  up
• 5:14
  to
• 5:14
  9B
• 5:15
  parameters,
• 5:16
  while
• 5:16
  Qwen-Scope
• 5:16
  includes
• 5:17
  a
• 5:17
  dense
• 5:17
  27B
• 5:18
  variant.
• 5:19
  The
• 5:19
  interactive
• 5:19
  Space
• 5:19
  demo
• 5:20
  shows
• 5:20
  what
• 5:20
  happens
• 5:20
  when
• 5:21
  a
• 5:21
  model
• 5:21
  unexpectedly
• 5:22
  switches
• 5:22
  from
• 5:22
  English
• 5:22
  to
• 5:23
  Chinese
• 5:23
  mid-response:
• 5:24
  feature
• 5:24
  comparison
• 5:25
  reveals
• 5:25
  exactly
• 5:25
  which
• 5:26
  feature
• 5:26
  ID
• 5:26
  spiked.
• 5:27
  With
• 5:27
  that
• 5:27
  ID
• 5:28
  in
• 5:28
  hand,
• 5:28
  you
• 5:28
  can
• 5:28
  steer
• 5:29
  the
• 5:29
  model
• 5:29
  by
• 5:29
  suppressing
• 5:30
  or
• 5:30
  amplifying
• 5:31
  specific
• 5:31
  features
• 5:31
  during
• 5:32
  generation
• 5:32
  —
• 5:33
  what
• 5:33
  practitioners
• 5:33
  call
• 5:34
  "surgical
• 5:34
  abliteration."
• 5:36
  This
• 5:36
  is
• 5:36
  considerably
• 5:37
  more
• 5:37
  precise
• 5:37
  than
• 5:37
  the
• 5:37
  brute-force
• 5:38
  "mean
• 5:38
  difference"
• 5:39
  approach
• 5:39
  that
• 5:39
  previous
• 5:40
  community
• 5:40
  efforts
• 5:40
  relied
• 5:41
  on.
• 5:42
  The
• 5:42
  Qwen
• 5:42
  team
• 5:42
  released
• 5:43
  everything
• 5:43
  under
• 5:43
  Apache
• 5:44
  2.0,
• 5:45
  though
• 5:45
  their
• 5:45
  caution
• 5:45
  statement
• 5:46
  explicitly
• 5:46
  prohibits
• 5:47
  using
• 5:47
  the
• 5:47
  tools
• 5:47
  to
• 5:48
  remove
• 5:48
  safety
• 5:48
  filters
• 5:48
  —
• 5:49
  a
• 5:49
  tension
• 5:50
  the
• 5:50
  community
• 5:50
  has
• 5:50
  already
• 5:50
  noticed.
• 5:51
  Google
• 5:52
  maintains
• 5:52
  its
• 5:52
  own
• 5:53
  parallel
• 5:53
  effort
• 5:53
  with
• 5:53
  Gemma
• 5:54
  Scope
• 5:54
  for
• 5:54
  Gemma
• 5:54
  2
• 5:54
  and
• 5:55
  3.
• 5:56
  Meanwhile,
• 5:56
  GuideLabs
• 5:57
  released
• 5:57
  steerling-8b,
• 5:58
  a
• 5:58
  purpose-built
• 5:59
  steerable
• 6:00
  8B
• 6:00
  model
• 6:00
  that
• 6:00
  represents
• 6:01
  a
• 6:01
  different
• 6:01
  approach
• 6:02
  to
• 6:02
  the
• 6:02
  same
• 6:02
  goal:
• 6:03
  baking
• 6:04
  steerability
• 6:04
  into
• 6:04
  the
• 6:04
  model
• 6:05
  weights
• 6:05
  rather
• 6:05
  than
• 6:05
  discovering
• 6:06
  it
• 6:06
  post-hoc
• 6:07
  through
• 6:07
  SAE
• 6:07
  analysis.
• 6:09
  On
• 6:09
  the
• 6:09
  quantization
• 6:09
  front,
• 6:10
  NVIDIA
• 6:10
  published
• 6:11
  Gemma-4-31B-IT-NVFP4,
• 6:16
  a
• 6:16
  4-bit
• 6:17
  floating-point
• 6:17
  quantized
• 6:18
  version
• 6:18
  of
• 6:18
  Google's
• 6:19
  latest
• 6:19
  31B
• 6:19
  model.
• 6:20
  FP4
• 6:21
  quantization
• 6:22
  represents
• 6:22
  a
• 6:22
  different
• 6:23
  tradeoff
• 6:23
  from
• 6:23
  the
• 6:23
  integer-based
• 6:24
  Q4
• 6:24
  formats
• 6:25
  common
• 6:25
  in
• 6:25
  the
• 6:25
  llama.cpp
• 6:27
  ecosystem:
• 6:28
  floating-point
• 6:28
  mantissa
• 6:29
  bits
• 6:29
  can
• 6:29
  preserve
• 6:30
  more
• 6:30
  dynamic
• 6:30
  range
• 6:31
  at
• 6:31
  the
• 6:31
  cost
• 6:31
  of
• 6:31
  reduced
• 6:32
  precision
• 6:32
  in
• 6:32
  the
• 6:32
  tails.
• 6:34
  Whether
• 6:34
  this
• 6:34
  translates
• 6:35
  to
• 6:35
  measurably
• 6:35
  better
• 6:36
  output
• 6:36
  quality
• 6:36
  at
• 6:36
  the
• 6:37
  31B
• 6:37
  scale
• 6:37
  —
• 6:38
  a
• 6:38
  sweet
• 6:39
  spot
• 6:39
  for
• 6:39
  consumer
• 6:39
  GPUs
• 6:40
  with
• 6:40
  24GB
• 6:40
  VRAM
• 6:41
  —
• 6:42
  is
• 6:42
  the
• 6:42
  question
• 6:43
  worth
• 6:43
  tracking.
• 6:44
  The
• 6:44
  Qwen
• 6:44
  3.6
• 6:45
  family
• 6:45
  continues
• 6:46
  to
• 6:46
  produce
• 6:46
  both
• 6:47
  impressive
• 6:47
  results
• 6:48
  and
• 6:48
  frustrating
• 6:48
  failure
• 6:49
  modes.
• 6:50
  One
• 6:50
  user
• 6:50
  running
• 6:50
  the
• 6:51
  27B
• 6:51
  model
• 6:52
  for
• 6:52
  tax
• 6:52
  accounting
• 6:52
  on
• 6:53
  a
• 6:53
  modest
• 6:53
  Ryzen
• 6:53
  9
• 6:53
  laptop
• 6:54
  with
• 6:54
  60GB
• 6:55
  RAM
• 6:55
  reports
• 6:56
  accuracy
• 6:56
  matching
• 6:57
  Claude
• 6:57
  on
• 6:57
  structured
• 6:57
  tasks
• 6:58
  like
• 6:58
  converting
• 6:58
  PDF
• 6:59
  tax
• 6:59
  forms
• 6:59
  to
• 7:00
  Excel
• 7:00
  —
• 7:01
  slower,
• 7:01
  but
• 7:02
  correct.
• 7:03
  The
• 7:03
  local
• 7:04
  models
• 7:05
  are
• 7:05
  entering
• 7:05
  the
• 7:05
  phase
• 7:06
  where
• 7:06
  they
• 7:06
  handle
• 7:06
  domain-specific
• 7:07
  professional
• 7:08
  workflows,
• 7:09
  not
• 7:09
  just
• 7:09
  coding
• 7:09
  tasks.
• 7:11
  But
• 7:11
  Qwen
• 7:11
  3.6
• 7:12
  27B
• 7:13
  also
• 7:13
  has
• 7:13
  a
• 7:13
  documented
• 7:14
  tendency
• 7:14
  to
• 7:15
  fall
• 7:15
  into
• 7:15
  self-affirming
• 7:16
  thinking
• 7:16
  loops
• 7:16
  —
• 7:17
  the
• 7:17
  model's
• 7:18
  internal
• 7:18
  reasoning
• 7:19
  degenerates
• 7:19
  into
• 7:19
  increasingly
• 7:20
  absurd
• 7:21
  self-reinforcement
• 7:22
  ("I
• 7:22
  will
• 7:22
  be
• 7:22
  concise.
• 7:24
  I
• 7:24
  will
• 7:24
  be
• 7:24
  precise.
• 7:25
  I
• 7:25
  will
• 7:26
  be
• 7:26
  accurate.
• 7:27
  I
• 7:27
  will
• 7:27
  be
• 7:27
  reliable...")
• 7:28
  that
• 7:28
  never
• 7:29
  terminates.
• 7:30
  This
• 7:30
  is
• 7:30
  not
• 7:30
  a
• 7:31
  new
• 7:31
  phenomenon
• 7:31
  at
• 7:31
  the
• 7:31
  27B
• 7:32
  scale:
• 7:33
  Qwen
• 7:34
  3.5
• 7:34
  27B
• 7:35
  exhibited
• 7:36
  similar
• 7:36
  degenerate
• 7:36
  behavior
• 7:37
  on
• 7:37
  coding
• 7:37
  benchmarks,
• 7:38
  declaring
• 7:39
  existing
• 7:39
  tests
• 7:40
  as
• 7:40
  passing
• 7:40
  and
• 7:40
  quitting
• 7:41
  without
• 7:41
  writing
• 7:41
  code.
• 7:43
  Community
• 7:43
  consensus
• 7:44
  points
• 7:44
  to
• 7:44
  setting
• 7:44
  presence
• 7:45
  penalty
• 7:45
  to
• 7:45
  1.5
• 7:46
  (per
• 7:46
  Qwen's
• 7:46
  own
• 7:47
  recommendation)
• 7:48
  and
• 7:48
  using
• 7:48
  fixed
• 7:48
  chat
• 7:49
  templates
• 7:49
  from
• 7:49
  the
• 7:49
  community
• 7:50
  as
• 7:50
  mitigations,
• 7:51
  though
• 7:51
  neither
• 7:51
  is
• 7:51
  a
• 7:51
  complete
• 7:52
  fix.
• 7:53
  For
• 7:53
  anyone
• 7:53
  deploying
• 7:54
  these
• 7:54
  models
• 7:54
  in
• 7:54
  production
• 7:55
  loops,
• 7:55
  loop
• 7:55
  detection
• 7:56
  remains
• 7:56
  a
• 7:56
  mandatory
• 7:57
  guardrail.
• 7:59
  NVIDIA's
• 8:00
  Nemotron
• 8:00
  3
• 8:01
  Nano
• 8:01
  Omni
• 8:02
  is
• 8:02
  a
• 8:02
  30B-A3B
• 8:04
  omni-modal
• 8:05
  understanding
• 8:05
  model
• 8:06
  built
• 8:06
  to
• 8:06
  reason
• 8:06
  jointly
• 8:07
  across
• 8:07
  text,
• 8:07
  images,
• 8:08
  video,
• 8:08
  and
• 8:09
  audio
• 8:09
  —
• 8:09
  and
• 8:09
  it
• 8:10
  is
• 8:10
  efficient
• 8:10
  enough
• 8:10
  to
• 8:10
  The
• 8:11
  architecture
• 8:12
  combines
• 8:12
  a
• 8:12
  hybrid
• 8:13
  Mamba-Transformer
• 8:14
  Mixture-of-Experts
• 8:15
  backbone
• 8:15
  (23
• 8:16
  Mamba
• 8:16
  state-space
• 8:17
  layers
• 8:17
  for
• 8:17
  long-context
• 8:18
  efficiency,
• 8:19
  MoE
• 8:19
  for
• 8:19
  conditional
• 8:20
  capacity,
• 8:20
  Transformer
• 8:21
  attention
• 8:22
  for
• 8:22
  global
• 8:22
  expressivity)
• 8:23
  with
• 8:23
  a
• 8:23
  C-RADIOv4-H
• 8:25
  vision
• 8:26
  encoder
• 8:26
  and
• 8:26
  Parakeet-TDT-0.6B-v2
• 8:29
  audio
• 8:29
  encoder.
• 8:30
  Each
• 8:30
  encoder
• 8:31
  connects
• 8:31
  through
• 8:31
  lightweight
• 8:32
  MLP
• 8:32
  projectors
• 8:33
  into
• 8:33
  a
• 8:33
  shared
• 8:33
  embedding
• 8:34
  space
• 8:34
  where
• 8:34
  all
• 8:34
  modality
• 8:35
  tokens
• 8:35
  are
• 8:35
  interleaved
• 8:36
  and
• 8:36
  processed
• 8:36
  jointly.
• 8:37
  The
• 8:37
  model
• 8:38
  targets
• 8:38
  five
• 8:38
  workload
• 8:39
  classes:
• 8:40
  complex
• 8:40
  multi-page
• 8:41
  document
• 8:41
  intelligence
• 8:42
  (100+
• 8:42
  pages),
• 8:43
  speech
• 8:43
  understanding,
• 8:44
  joint
• 8:45
  audio-video
• 8:45
  reasoning,
• 8:46
  agentic
• 8:46
  GUI
• 8:47
  computer
• 8:47
  use,
• 8:47
  and
• 8:48
  cross-modal
• 8:48
  multi-step
• 8:49
  reasoning.
• 8:50
  The
• 8:50
  design
• 8:50
  innovations
• 8:51
  target
• 8:51
  real-world
• 8:52
  bottlenecks.
• 8:53
  Dynamic
• 8:53
  resolution
• 8:54
  processing
• 8:54
  at
• 8:54
  native
• 8:55
  aspect
• 8:55
  ratio
• 8:55
  handles
• 8:56
  images
• 8:56
  from
• 8:56
  512×512
• 8:58
  up
• 8:58
  to
• 8:59
  1840×1840,
• 9:00
  critical
• 9:00
  for
• 9:01
  dense
• 9:01
  documents
• 9:01
  and
• 9:01
  screenshots.
• 9:03
  Conv3D
• 9:03
  temporal
• 9:04
  compression
• 9:04
  fuses
• 9:05
  consecutive
• 9:05
  frame
• 9:05
  pairs
• 9:06
  into
• 9:06
  "tubelets,"
• 9:07
  halving
• 9:07
  the
• 9:07
  vision
• 9:07
  token
• 9:08
  count
• 9:08
  for
• 9:08
  video.
• 9:09
  Efficient
• 9:10
  Video
• 9:10
  Sampling
• 9:10
  then
• 9:11
  drops
• 9:11
  redundant
• 9:11
  static
• 9:12
  tokens
• 9:12
  at
• 9:12
  inference,
• 9:13
  keeping
• 9:13
  only
• 9:14
  frames
• 9:14
  where
• 9:14
  something
• 9:14
  has
• 9:14
  actually
• 9:15
  changed.
• 9:16
  The
• 9:16
  result:
• 9:17
  9×
• 9:17
  higher
• 9:18
  throughput
• 9:18
  and
• 9:18
  7.4×
• 9:19
  higher
• 9:20
  system
• 9:20
  efficiency
• 9:21
  on
• 9:21
  multi-document
• 9:21
  tasks
• 9:22
  compared
• 9:22
  to
• 9:22
  other
• 9:23
  open
• 9:23
  omni
• 9:23
  models
• 9:24
  at
• 9:24
  a
• 9:24
  fixed
• 9:24
  per-user
• 9:25
  interactivity
• 9:25
  threshold.
• 9:26
  NVIDIA's
• 9:27
  training
• 9:27
  recipe
• 9:28
  includes
• 9:28
  multimodal
• 9:29
  RL
• 9:29
  with
• 9:29
  intentionally
• 9:30
  unanswerable
• 9:30
  questions
• 9:31
  —
• 9:32
  forcing
• 9:32
  the
• 9:32
  model
• 9:32
  to
• 9:32
  learn
• 9:33
  to
• 9:33
  abstain
• 9:33
  rather
• 9:34
  than
• 9:34
  hallucinate
• 9:34
  —
• 9:35
  and
• 9:35
  11.4M
• 9:36
  synthetic
• 9:36
  QA
• 9:37
  pairs
• 9:37
  from
• 9:37
  real
• 9:38
  PDFs
• 9:38
  that
• 9:39
  produced
• 9:39
  a
• 9:39
  2.19×
• 9:40
  accuracy
• 9:41
  gain
• 9:41
  on
• 9:41
  MMLongBench-Doc.
• 9:43
  The
• 9:43
  local
• 9:43
  voice
• 9:44
  stack
• 9:44
  gained
• 9:44
  another
• 9:44
  piece
• 9:45
  with
• 9:45
  vibevoice.cpp,
• 9:47
  a
• 9:48
  pure
• 9:48
  C++
• 9:49
  ggml
• 9:49
  port
• 9:49
  of
• 9:50
  Microsoft's
• 9:50
  VibeVoice
• 9:51
  speech-to-speech
• 9:51
  model
• 9:52
  from
• 9:52
  the
• 9:52
  LocalAI
• 9:52
  team.
• 9:53
  The
• 9:53
  entire
• 9:54
  pipeline
• 9:54
  —
• 9:55
  TTS
• 9:56
  with
• 9:56
  voice
• 9:56
  cloning
• 9:57
  from
• 9:57
  a
• 9:57
  30-second
• 9:57
  reference
• 9:58
  clip
• 9:58
  and
• 9:58
  7B-parameter
• 9:59
  ASR
• 10:00
  with
• 10:00
  speaker
• 10:01
  diarization
• 10:01
  —
• 10:02
  runs
• 10:02
  with
• 10:02
  zero
• 10:03
  Python
• 10:03
  at
• 10:03
  inference.
• 10:04
  On
• 10:04
  CUDA
• 10:05
  Q4
• 10:05
  K
• 10:05
  (GB10),
• 10:07
  a
• 10:07
  68-second
• 10:08
  sample
• 10:08
  processes
• 10:08
  in
• 10:09
  28
• 10:09
  seconds
• 10:10
  at
• 10:10
  a
• 10:10
  real-time
• 10:10
  factor
• 10:11
  of
• 10:11
  0.41;
• 10:12
  CPU-only
• 10:13
  on
• 10:13
  a
• 10:13
  Ryzen
• 10:14
  9
• 10:14
  pushes
• 10:14
  to
• 10:14
  2.2×
• 10:15
  real
• 10:15
  time.
• 10:16
  The
• 10:16
  17-minute
• 10:17
  audio
• 10:17
  test
• 10:18
  case
• 10:18
  still
• 10:18
  requires
• 10:19
  26GB
• 10:20
  peak
• 10:20
  RSS
• 10:20
  on
• 10:20
  CPU,
• 10:22
  and
• 10:22
  streaming
• 10:22
  output
• 10:23
  is
• 10:23
  not
• 10:23
  yet
• 10:23
  implemented,
• 10:24
  but
• 10:24
  backends
• 10:24
  for
• 10:25
  CUDA,
• 10:25
  Metal,
• 10:26
  Vulkan,
• 10:26
  and
• 10:26
  hipBLAS
• 10:27
  make
• 10:27
  deployment
• 10:27
  radically
• 10:28
  simpler
• 10:28
  than
• 10:28
  the
• 10:28
  upstream
• 10:29
  Transformers/vLLM
• 10:31
  stack.
• 10:32
  On
• 10:32
  the
• 10:32
  research
• 10:33
  side,
• 10:33
  a
• 10:33
  paper
• 10:34
  from
• 10:34
  National
• 10:34
  Tsing
• 10:34
  Hua
• 10:35
  University
• 10:35
  introduces
• 10:36
  VI-NBFNet,
• 10:40
  a
• 10:40
  neural
• 10:40
  beamforming
• 10:41
  network
• 10:41
  that
• 10:41
  uses
• 10:41
  lip
• 10:41
  movement
• 10:42
  features
• 10:42
  extracted
• 10:43
  from
• 10:43
  a
• 10:43
  pretrained
• 10:43
  visual
• 10:44
  speech
• 10:44
  recognition
• 10:44
  model
• 10:45
  to
• 10:45
  steer
• 10:45
  a
• 10:46
  four-microphone
• 10:46
  array.
• 10:48
  The
• 10:48
  system
• 10:48
  jointly
• 10:48
  learns
• 10:49
  audiovisual
• 10:49
  features
• 10:50
  and
• 10:50
  spatial
• 10:50
  covariance
• 10:51
  matrices
• 10:51
  end-to-end,
• 10:52
  achieving
• 10:52
  PESQ
• 10:53
  2.088
• 10:54
  and
• 10:54
  8%
• 10:55
  word
• 10:55
  error
• 10:55
  rate
• 10:55
  on
• 10:56
  Whisper-turbo
• 10:56
  for
• 10:56
  moving
• 10:56
  speakers
• 10:57
  —
• 10:58
  roughly
• 10:58
  half
• 10:58
  the
• 10:58
  error
• 10:59
  rate
• 10:59
  of
• 10:59
  the
• 10:59
  next-best
• 10:59
  baseline
• 11:00
  —
• 11:01
  while
• 11:01
  remaining
• 11:01
  notably
• 11:01
  compact
• 11:02
  at
• 11:02
  7.15M
• 11:03
  parameters.
• 11:07
  A
• 11:07
  startup
• 11:08
  founder
• 11:08
  burning
• 11:08
  $1,500
• 11:10
  to
• 11:10
  $4,000
• 11:10
  per
• 11:11
  day
• 11:11
  on
• 11:11
  Claude
• 11:11
  Opus
• 11:11
  4.7
• 11:12
  API
• 11:12
  credits
• 11:13
  posted
• 11:13
  an
• 11:13
  open
• 11:13
  question
• 11:14
  to
• 11:14
  the
• 11:14
  LocalLLaMA
• 11:14
  community:
• 11:15
  what
• 11:15
  does
• 11:16
  the
• 11:16
  optimal
• 11:16
  $100K
• 11:17
  self-hosted
• 11:17
  inference
• 11:18
  server
• 11:18
  look
• 11:18
  like
• 11:18
  for
• 11:18
  agentic
• 11:19
  coding?
• 11:20
  The
• 11:20
  thread
• 11:20
  crystallized
• 11:20
  the
• 11:20
  current
• 11:21
  build-vs-buy
• 11:21
  debate
• 11:22
  with
• 11:22
  unusual
• 11:22
  clarity.
• 11:23
  The
• 11:23
  EPYC
• 11:24
  +
• 11:24
  RTX
• 11:25
  6000
• 11:25
  Pro
• 11:25
  path
• 11:25
  dominated:
• 11:26
  one
• 11:27
  practitioner
• 11:27
  running
• 11:27
  4×
• 11:28
  6000
• 11:28
  Pros
• 11:28
  (768GB
• 11:30
  total
• 11:30
  VRAM)
• 11:31
  reports
• 11:31
  serving
• 11:31
  MiniMax
• 11:32
  M2.7
• 11:33
  FP8
• 11:34
  "all
• 11:34
  day
• 11:34
  long"
• 11:34
  at
• 11:34
  high
• 11:35
  speed,
• 11:35
  with
• 11:35
  Qwen3.5
• 11:37
  397B
• 11:38
  NVFP4
• 11:40
  hitting
• 11:40
  hundreds
• 11:40
  of
• 11:40
  tokens
• 11:41
  per
• 11:41
  second
• 11:41
  under
• 11:41
  concurrent
• 11:42
  load.
• 11:43
  "With
• 11:43
  Claude
• 11:43
  CLI
• 11:44
  hooked
• 11:44
  up
• 11:44
  to
• 11:44
  those
• 11:44
  bad
• 11:44
  boys
• 11:45
  it
• 11:45
  feels
• 11:45
  like
• 11:45
  SOTA
• 11:45
  in
• 11:45
  a
• 11:46
  box."
• 11:46
  Eight
• 11:47
  6000
• 11:47
  Pros
• 11:47
  fit
• 11:48
  within
• 11:48
  $100K
• 11:49
  if
• 11:49
  you
• 11:49
  pare
• 11:49
  back
• 11:49
  system
• 11:49
  RAM.
• 11:50
  The
• 11:50
  consistent
• 11:51
  warning:
• 11:51
  do
• 11:51
  not
• 11:51
  buy
• 11:52
  Macs
• 11:52
  for
• 11:52
  inference
• 11:52
  workloads
• 11:53
  —
• 11:53
  memory
• 11:54
  bandwidth
• 11:54
  makes
• 11:54
  them
• 11:55
  uncompetitive
• 11:55
  for
• 11:55
  production
• 11:56
  throughput
• 11:56
  —
• 11:56
  and
• 11:57
  open-weight
• 11:57
  models
• 11:57
  still
• 11:58
  trail
• 11:58
  Opus
• 11:58
  quality
• 11:59
  meaningfully
• 11:59
  enough
• 11:59
  that
• 11:59
  the
• 12:00
  investment
• 12:00
  thesis
• 12:00
  requires
• 12:01
  careful
• 12:01
  validation.
• 12:02
  Tenstorrent's
• 12:02
  TT-QuietBox
• 12:03
  2
• 12:03
  represents
• 12:03
  the
• 12:03
  ASIC
• 12:04
  alternative.
• 12:05
  The
• 12:05
  desktop
• 12:05
  workstation
• 12:06
  packs
• 12:06
  two
• 12:06
  liquid-cooled
• 12:07
  Blackhole
• 12:07
  cards,
• 12:08
  each
• 12:08
  carrying
• 12:08
  two
• 12:09
  Blackhole
• 12:09
  ASICs
• 12:09
  with
• 12:09
  240
• 12:10
  Tensix
• 12:10
  cores
• 12:11
  and
• 12:11
  64GB
• 12:12
  DDR6
• 12:13
  at
• 12:13
  16
• 12:13
  GT/s
• 12:14
  —
• 12:14
  1
• 12:15
  TB/s
• 12:15
  aggregate
• 12:16
  memory
• 12:16
  bandwidth
• 12:16
  across
• 12:17
  128GB
• 12:18
  of
• 12:18
  VRAM,
• 12:19
  plus
• 12:19
  256GB
• 12:20
  DDR5
• 12:21
  system
• 12:21
  memory,
• 12:22
  with
• 12:22
  800G
• 12:22
  Ethernet
• 12:23
  between
• 12:23
  ASICs
• 12:23
  internally.
• 12:24
  Community
• 12:25
  response
• 12:25
  is
• 12:25
  cautious:
• 12:26
  a
• 12:26
  single
• 12:26
  RTX
• 12:27
  Pro
• 12:27
  6000
• 12:27
  delivers
• 12:28
  1.79
• 12:29
  TB/s
• 12:30
  on
• 12:30
  its
• 12:30
  own,
• 12:30
  so
• 12:31
  the
• 12:31
  raw
• 12:31
  bandwidth
• 12:31
  competition
• 12:32
  is
• 12:32
  not
• 12:32
  yet
• 12:32
  won.
• 12:33
  But
• 12:33
  at
• 12:33
  roughly
• 12:34
  $10K
• 12:34
  for
• 12:34
  a
• 12:34
  complete
• 12:35
  system
• 12:35
  with
• 12:35
  experimental
• 12:36
  ASIC
• 12:36
  silicon,
• 12:37
  the
• 12:37
  price-per-exploration-dollar
• 12:38
  is
• 12:39
  compelling
• 12:39
  if
• 12:39
  Tenstorrent
• 12:40
  expands
• 12:40
  model
• 12:40
  support.
• 12:41
  For
• 12:41
  those
• 12:41
  unwilling
• 12:42
  to
• 12:42
  pick
• 12:42
  a
• 12:42
  single
• 12:42
  vendor,
• 12:43
  llama.cpp's
• 12:44
  -DGGML
• 12:45
  BACKEND
• 12:47
  DL=ON
• 12:48
  flag
• 12:49
  now
• 12:49
  enables
• 12:49
  simultaneous
• 12:50
  CUDA
• 12:50
  and
• 12:50
  ROCm
• 12:51
  backend
• 12:51
  loading
• 12:51
  —
• 12:52
  genuine
• 12:53
  mixed-vendor
• 12:53
  inference
• 12:54
  in
• 12:54
  a
• 12:54
  single
• 12:54
  process.
• 12:55
  One
• 12:55
  user
• 12:55
  reports
• 12:56
  running
• 12:56
  MiniMax
• 12:57
  M2.7
• 12:58
  Q4
• 12:59
  across
• 12:59
  an
• 12:59
  NVIDIA
• 12:59
  GPU
• 13:00
  (83.6
• 13:00
  GB)
• 13:01
  and
• 13:01
  an
• 13:01
  AMD
• 13:02
  GPU
• 13:02
  (40.3
• 13:03
  GB)
• 13:03
  simultaneously,
• 13:04
  with
• 13:05
  prefill
• 13:05
  speed
• 13:05
  as
• 13:05
  the
• 13:05
  primary
• 13:06
  advantage.
• 13:07
  The
• 13:07
  setup
• 13:07
  requires
• 13:07
  manual
• 13:08
  CMakeLists.txt
• 13:12
  edits
• 13:12
  and
• 13:12
  careful
• 13:12
  flag
• 13:13
  management,
• 13:13
  but
• 13:13
  the
• 13:13
  fact
• 13:14
  that
• 13:14
  it
• 13:14
  works
• 13:14
  at
• 13:14
  all
• 13:14
  collapses
• 13:15
  a
• 13:15
  boundary
• 13:15
  the
• 13:15
  community
• 13:16
  had
• 13:16
  accepted
• 13:16
  as
• 13:16
  permanent.
• 13:17
  A
• 13:17
  Blackwell
• 13:18
  +
• 13:18
  M3
• 13:18
  Ultra
• 13:19
  RDMA
• 13:19
  cluster
• 13:20
  with
• 13:20
  roughly
• 13:20
  2TB
• 13:21
  of
• 13:21
  aggregate
• 13:21
  RAM
• 13:21
  is
• 13:22
  being
• 13:22
  assembled
• 13:22
  for
• 13:22
  tinygrad
• 13:23
  driver
• 13:23
  testing,
• 13:24
  with
• 13:24
  community-requested
• 13:25
  benchmarks
• 13:25
  of
• 13:25
  DeepSeek
• 13:26
  V4
• 13:26
  and
• 13:26
  MiMo
• 13:26
  V2.5
• 13:28
  on
• 13:28
  the
• 13:28
  schedule
• 13:29
  .
• 13:29
  MiMo
• 13:30
  2.5
• 13:30
  itself
• 13:31
  drew
• 13:31
  pointed
• 13:31
  criticism
• 13:32
  for
• 13:32
  requiring
• 13:33
  GPUs
• 13:33
  in
• 13:33
  multiples
• 13:34
  of
• 13:34
  four
• 13:34
  —
• 13:35
  the
• 13:35
  checkpoint's
• 13:36
  TP=4-interleaved
• 13:38
  fused
• 13:38
  qkv
• 13:39
  proj
• 13:39
  architecture
• 13:40
  makes
• 13:40
  anything
• 13:41
  below
• 13:41
  four
• 13:41
  GPUs
• 13:42
  a
• 13:42
  non-starter
• 13:42
  on
• 13:43
  SGLang
• 13:43
  and
• 13:43
  vLLM,
• 13:44
  though
• 13:45
  GGUF
• 13:45
  via
• 13:45
  llama.cpp
• 13:47
  avoids
• 13:48
  the
• 13:48
  restriction
• 13:48
  entirely.
• 13:50
  Community
• 13:50
  commentary
• 13:51
  was
• 13:51
  blunt:
• 13:52
  the
• 13:52
  release
• 13:52
  looks
• 13:52
  like
• 13:53
  "a
• 13:53
  dump
• 13:53
  of
• 13:53
  the
• 13:53
  files
• 13:54
  they
• 13:54
  use
• 13:54
  for
• 13:54
  their
• 13:54
  internal
• 13:55
  proprietary
• 13:55
  serving
• 13:56
  stack,"
• 13:57
  with
• 13:57
  missing
• 13:57
  weights,
• 13:58
  garbage
• 13:58
  model
• 13:58
  index
• 13:59
  files,
• 13:59
  and
• 14:00
  nonstandard
• 14:00
  tensor
• 14:00
  padding.
• 14:04
  Stripe
• 14:04
  published
• 14:05
  the
• 14:05
  full
• 14:05
  story
• 14:05
  behind
• 14:05
  rubyfmt,
• 14:06
  the
• 14:06
  zero-configuration
• 14:07
  Ruby
• 14:07
  autoformatter
• 14:08
  that
• 14:08
  reformatted
• 14:09
  25
• 14:09
  million
• 14:09
  lines
• 14:09
  of
• 14:10
  code
• 14:10
  in
• 14:10
  a
• 14:10
  single
• 14:10
  Saturday
• 14:11
  morning
• 14:11
  commit.
• 14:12
  The
• 14:12
  project
• 14:12
  started
• 14:12
  as
• 14:12
  a
• 14:12
  bar
• 14:13
  conversation
• 14:13
  at
• 14:13
  RubyConf
• 14:14
  2018
• 14:15
  and
• 14:15
  spent
• 14:15
  six
• 14:15
  years
• 14:16
  evolving
• 14:16
  from
• 14:16
  a
• 14:16
  single-file
• 14:17
  Ruby
• 14:17
  script
• 14:17
  to
• 14:17
  a
• 14:17
  Rust
• 14:18
  binary
• 14:18
  that
• 14:18
  linked
• 14:18
  an
• 14:18
  entire
• 14:19
  Ruby
• 14:19
  VM
• 14:19
  just
• 14:19
  to
• 14:20
  walk
• 14:20
  parse
• 14:20
  trees
• 14:20
  in
• 14:20
  memory.
• 14:21
  The
• 14:22
  speed
• 14:22
  budget
• 14:22
  was
• 14:22
  strict:
• 14:23
  100ms
• 14:24
  to
• 14:24
  format
• 14:24
  all
• 14:25
  but
• 14:25
  the
• 14:25
  largest
• 14:25
  files,
• 14:26
  because
• 14:27
  Ruby's
• 14:27
  158ms
• 14:28
  startup
• 14:29
  time
• 14:29
  made
• 14:29
  invoking
• 14:29
  it
• 14:30
  normally
• 14:30
  a
• 14:30
  non-starter
• 14:31
  for
• 14:31
  save
• 14:31
  hooks.
• 14:32
  The
• 14:32
  Rust
• 14:33
  rewrite
• 14:33
  produced
• 14:33
  genuinely
• 14:34
  unusual
• 14:34
  engineering:
• 14:35
  Serde
• 14:36
  deserializing
• 14:36
  Ruby
• 14:37
  VALUE
• 14:37
  objects
• 14:37
  directly
• 14:38
  from
• 14:38
  memory
• 14:38
  by
• 14:38
  introspecting
• 14:39
  C
• 14:39
  types
• 14:40
  through
• 14:40
  Ruby's
• 14:40
  API,
• 14:41
  with
• 14:41
  the
• 14:41
  schema
• 14:42
  unchanged
• 14:42
  —
• 14:43
  only
• 14:43
  the
• 14:43
  deserializer
• 14:44
  swapped
• 14:44
  out.
• 14:44
  The
• 14:45
  payoff
• 14:45
  came
• 14:46
  in
• 14:46
  2024
• 14:46
  when
• 14:46
  Stripe
• 14:47
  flipped
• 14:47
  the
• 14:47
  opt-in
• 14:47
  model
• 14:48
  —
• 14:48
  only
• 14:49
  a
• 14:49
  tiny
• 14:49
  minority
• 14:50
  of
• 14:50
  files
• 14:50
  were
• 14:50
  held
• 14:50
  back
• 14:51
  —
• 14:51
  and
• 14:51
  merged
• 14:52
  a
• 14:52
  diff
• 14:52
  so
• 14:52
  large
• 14:53
  GitHub
• 14:53
  could
• 14:53
  not
• 14:53
  render
• 14:54
  it.
• 14:54
  Today,
• 14:55
  100%
• 14:55
  of
• 14:55
  Stripe's
• 14:56
  42
• 14:56
  million
• 14:57
  lines
• 14:57
  of
• 14:57
  Ruby
• 14:57
  run
• 14:57
  through
• 14:58
  rubyfmt.
• 14:58
  One
• 14:58
  engineer
• 14:59
  who
• 14:59
  arrived
• 14:59
  from
• 14:59
  Python
• 15:00
  captured
• 15:00
  the
• 15:00
  result:
• 15:01
  "I
• 15:01
  sometimes
• 15:02
  receive
• 15:02
  code
• 15:02
  format
• 15:03
  nits
• 15:03
  on
• 15:03
  code
• 15:03
  reviews...
• 15:04
  I
• 15:04
  haven't
• 15:05
  thought
• 15:05
  about
• 15:05
  this
• 15:05
  problem
• 15:05
  in
• 15:06
  years."
• 15:07
  In
• 15:07
  2025,
• 15:08
  rubyfmt
• 15:08
  migrated
• 15:09
  to
• 15:09
  Prism,
• 15:10
  the
• 15:10
  new
• 15:10
  official
• 15:10
  Ruby
• 15:10
  parser
• 15:11
  that
• 15:11
  instantiates
• 15:11
  parse
• 15:12
  trees
• 15:12
  without
• 15:12
  a
• 15:12
  Ruby
• 15:13
  VM
• 15:13
  at
• 15:13
  all,
• 15:14
  shrinking
• 15:14
  the
• 15:14
  binary
• 15:15
  by
• 15:15
  megabytes.
• 15:16
  The
• 15:16
  engineering
• 15:17
  lesson
• 15:17
  generalizes:
• 15:18
  the
• 15:18
  hardest
• 15:19
  part
• 15:19
  of
• 15:19
  building
• 15:19
  a
• 15:19
  formatter
• 15:19
  is
• 15:20
  not
• 15:20
  the
• 15:20
  formatting
• 15:20
  —
• 15:21
  it
• 15:21
  is
• 15:21
  parsing
• 15:21
  a
• 15:21
  language
• 15:22
  whose
• 15:22
  grammar
• 15:22
  was
• 15:22
  never
• 15:23
  designed
• 15:23
  for
• 15:23
  external
• 15:23
  tooling.
• 15:24
  Crabbox
• 15:25
  launched
• 15:25
  as
• 15:25
  an
• 15:25
  open-source
• 15:26
  remote
• 15:26
  testbox
• 15:27
  runner
• 15:27
  targeting
• 15:27
  maintainers
• 15:28
  and
• 15:28
  AI
• 15:28
  agents.
• 15:29
  The
• 15:29
  architecture
• 15:30
  —
• 15:31
  a
• 15:31
  Go
• 15:31
  CLI
• 15:31
  on
• 15:32
  the
• 15:32
  laptop,
• 15:32
  a
• 15:33
  Cloudflare
• 15:33
  Worker
• 15:33
  broker
• 15:33
  that
• 15:34
  owns
• 15:34
  provider
• 15:34
  credentials
• 15:35
  and
• 15:35
  serializes
• 15:35
  lease
• 15:36
  state,
• 15:36
  and
• 15:36
  managed
• 15:37
  runners
• 15:37
  on
• 15:37
  Hetzner
• 15:37
  or
• 15:38
  AWS
• 15:38
  —
• 15:39
  lets
• 15:39
  developers
• 15:40
  sync
• 15:40
  a
• 15:40
  dirty
• 15:40
  local
• 15:41
  checkout
• 15:41
  to
• 15:41
  cloud
• 15:41
  compute,
• 15:42
  run
• 15:42
  a
• 15:42
  command,
• 15:43
  and
• 15:43
  stream
• 15:43
  output
• 15:43
  back.
• 15:44
  Cost
• 15:45
  guardrails
• 15:45
  enforce
• 15:45
  per-lease
• 15:46
  and
• 15:46
  monthly
• 15:46
  spend
• 15:47
  caps
• 15:47
  with
• 15:47
  live
• 15:47
  EC2
• 15:48
  Spot
• 15:48
  pricing,
• 15:49
  and
• 15:49
  GitHub
• 15:49
  Actions
• 15:50
  hydration
• 15:50
  turns
• 15:51
  a
• 15:51
  leased
• 15:51
  box
• 15:51
  into
• 15:52
  an
• 15:52
  ephemeral
• 15:52
  Actions
• 15:52
  runner
• 15:53
  so
• 15:53
  the
• 15:53
  repo's
• 15:54
  own
• 15:54
  workflow
• 15:54
  handles
• 15:55
  runtime
• 15:55
  setup.
• 15:56
  The
• 15:56
  OpenClaw
• 15:57
  plugin
• 15:57
  exposes
• 15:58
  the
• 15:58
  full
• 15:58
  lifecycle
• 15:58
  as
• 15:58
  agent
• 15:59
  tools,
• 15:59
  making
• 16:00
  crabbox
• 16:00
  run
• 16:00
  and
• 16:00
  crabbox
• 16:01
  warmup
• 16:01
  native
• 16:01
  to
• 16:02
  AI-assisted
• 16:02
  development
• 16:03
  workflows.
• 16:04
  The
• 16:04
  Open
• 16:04
  WebUI
• 16:04
  ecosystem
• 16:05
  got
• 16:05
  a
• 16:05
  significant
• 16:06
  polish
• 16:06
  with
• 16:06
  Inline
• 16:07
  Visualizer
• 16:07
  v2.1.0,
• 16:10
  which
• 16:10
  adds
• 16:10
  pre-styled
• 16:11
  bare
• 16:11
  HTML
• 16:11
  tags,
• 16:12
  a
• 16:12
  9-color
• 16:13
  accent
• 16:13
  palette,
• 16:14
  and
• 16:14
  expanded
• 16:14
  chart
• 16:15
  types
• 16:15
  including
• 16:15
  stacked
• 16:16
  areas,
• 16:16
  radar,
• 16:17
  and
• 16:17
  KPI
• 16:17
  cards
• 16:18
  with
• 16:18
  sparklines.
• 16:19
  Every
• 16:20
  bug
• 16:20
  reported
• 16:20
  after
• 16:21
  the
• 16:21
  v2.0
• 16:22
  launch
• 16:23
  was
• 16:23
  fixed,
• 16:24
  including
• 16:24
  a
• 16:24
  particularly
• 16:25
  insidious
• 16:25
  issue
• 16:26
  where
• 16:26
  visualizations
• 16:27
  stayed
• 16:27
  blank
• 16:27
  when
• 16:27
  the
• 16:28
  model's
• 16:28
  "Thinking"
• 16:28
  section
• 16:29
  was
• 16:29
  expanded
• 16:29
  —
• 16:30
  a
• 16:30
  problem
• 16:31
  affecting
• 16:31
  Bedrock-hosted
• 16:32
  Haiku
• 16:33
  and
• 16:33
  any
• 16:33
  provider
• 16:33
  wrapping
• 16:34
  responses
• 16:34
  in
• 16:34
  reasoning
• 16:35
  blocks.
• 16:37
  Firgelli
• 16:38
  Automations
• 16:38
  published
• 16:39
  a
• 16:39
  comprehensive
• 16:39
  engineering
• 16:40
  guide
• 16:40
  on
• 16:40
  humanoid
• 16:40
  robot
• 16:41
  actuators
• 16:41
  that
• 16:42
  reads
• 16:42
  like
• 16:42
  a
• 16:42
  field
• 16:42
  manual
• 16:43
  for
• 16:43
  the
• 16:43
  mechanical
• 16:43
  problems
• 16:44
  everyone
• 16:44
  in
• 16:44
  robotics
• 16:45
  knows
• 16:45
  about
• 16:45
  but
• 16:45
  few
• 16:45
  document
• 16:46
  this
• 16:46
  thoroughly.
• 16:47
  The
• 16:47
  central
• 16:47
  constraint
• 16:47
  is
• 16:48
  brutal:
• 16:48
  a
• 16:48
  commercially
• 16:49
  viable
• 16:49
  humanoid
• 16:50
  takes
• 16:50
  roughly
• 16:50
  5,000
• 16:51
  steps
• 16:51
  per
• 16:51
  hour,
• 16:52
  each
• 16:52
  delivering
• 16:52
  2–3×
• 16:54
  body
• 16:54
  weight
• 16:54
  in
• 16:54
  impact
• 16:54
  force
• 16:55
  through
• 16:55
  the
• 16:55
  leg
• 16:55
  actuators.
• 16:56
  At
• 16:57
  a
• 16:57
  sustained
• 16:57
  84
• 16:58
  steps
• 16:58
  per
• 16:58
  minute,
• 16:59
  an
• 16:59
  8-hour
• 16:59
  shift
• 17:00
  accumulates
• 17:00
  over
• 17:00
  40,000
• 17:01
  impacts
• 17:01
  —
• 17:02
  compressing
• 17:03
  years
• 17:03
  of
• 17:03
  standard
• 17:03
  industrial
• 17:04
  wear
• 17:04
  into
• 17:04
  weeks.
• 17:05
  These
• 17:05
  impacts
• 17:06
  arrive
• 17:06
  in
• 17:06
  50–100
• 17:08
  milliseconds,
• 17:09
  faster
• 17:09
  than
• 17:09
  any
• 17:09
  sensor
• 17:10
  loop
• 17:10
  can
• 17:10
  react,
• 17:11
  which
• 17:11
  means
• 17:11
  the
• 17:11
  actuator
• 17:12
  must
• 17:12
  be
• 17:12
  mechanically
• 17:13
  back-drivable
• 17:13
  to
• 17:13
  absorb
• 17:14
  the
• 17:14
  energy.
• 17:15
  Self-locking
• 17:15
  designs
• 17:16
  like
• 17:16
  industrial
• 17:17
  lead
• 17:17
  screws
• 17:17
  force
• 17:17
  the
• 17:17
  gearbox
• 17:18
  to
• 17:18
  absorb
• 17:18
  100%
• 17:19
  of
• 17:19
  the
• 17:19
  shock,
• 17:20
  causing
• 17:20
  immediate
• 17:21
  shear
• 17:21
  failure.
• 17:22
  The
• 17:22
  guide
• 17:22
  documents
• 17:22
  what
• 17:23
  it
• 17:23
  calls
• 17:23
  the
• 17:23
  "mass
• 17:23
  penalty
• 17:24
  spiral."
• 17:25
  A
• 17:25
  200-gram
• 17:25
  error
• 17:26
  at
• 17:26
  the
• 17:26
  ankle
• 17:26
  —
• 17:27
  choosing
• 17:27
  a
• 17:27
  cheaper,
• 17:28
  heavier
• 17:28
  actuator
• 17:28
  —
• 17:29
  compounds
• 17:30
  exponentially
• 17:30
  upward
• 17:31
  through
• 17:31
  the
• 17:31
  kinematic
• 17:31
  chain.
• 17:32
  The
• 17:32
  knee
• 17:33
  must
• 17:33
  grow
• 17:33
  to
• 17:33
  handle
• 17:33
  the
• 17:34
  heavier
• 17:34
  foot,
• 17:35
  the
• 17:35
  hip
• 17:35
  must
• 17:35
  grow
• 17:35
  to
• 17:35
  handle
• 17:36
  the
• 17:36
  heavier
• 17:36
  knee,
• 17:37
  the
• 17:37
  battery
• 17:37
  must
• 17:38
  grow
• 17:38
  to
• 17:38
  power
• 17:38
  larger
• 17:39
  motors.
• 17:40
  A
• 17:40
  component-level
• 17:40
  mistake
• 17:41
  becomes
• 17:41
  a
• 17:41
  system-level
• 17:42
  catastrophe.
• 17:43
  The
• 17:43
  only
• 17:43
  escape
• 17:43
  is
• 17:44
  enforcing
• 17:44
  density
• 17:44
  targets
• 17:45
  from
• 17:45
  day
• 17:45
  one:
• 17:46
  greater
• 17:46
  than
• 17:47
  10
• 17:47
  Nm/kg
• 17:48
  specific
• 17:48
  torque
• 17:49
  for
• 17:49
  rotary
• 17:49
  actuators,
• 17:50
  greater
• 17:51
  than
• 17:51
  4,000
• 17:52
  N/kg
• 17:52
  specific
• 17:53
  force
• 17:53
  for
• 17:53
  linear
• 17:54
  ones.
• 17:54
  "There
• 17:55
  is
• 17:55
  no
• 17:55
  'upgrading
• 17:55
  later.'
• 17:57
  The
• 17:57
  mass
• 17:57
  budget
• 17:57
  is
• 17:57
  set
• 17:58
  by
• 17:58
  the
• 17:58
  actuators."
• 17:59
  Every
• 17:59
  major
• 18:00
  humanoid
• 18:00
  manufacturer
• 18:01
  —
• 18:01
  Tesla,
• 18:02
  Figure,
• 18:03
  Apptronik
• 18:03
  —
• 18:04
  independently
• 18:04
  converged
• 18:05
  on
• 18:05
  the
• 18:05
  same
• 18:05
  split
• 18:06
  architecture.
• 18:07
  Strain
• 18:07
  wave
• 18:08
  gears
• 18:08
  (harmonic
• 18:08
  drives)
• 18:09
  handle
• 18:09
  rotational
• 18:10
  joints
• 18:10
  like
• 18:10
  shoulders
• 18:11
  and
• 18:11
  wrists:
• 18:12
  zero
• 18:12
  backlash,
• 18:13
  50:1
• 18:13
  to
• 18:13
  100:1
• 18:14
  reduction
• 18:14
  in
• 18:14
  a
• 18:14
  single
• 18:15
  flat
• 18:15
  stage,
• 18:16
  though
• 18:16
  the
• 18:16
  flexing
• 18:17
  metal
• 18:17
  generates
• 18:17
  heat
• 18:18
  and
• 18:18
  fatigues.
• 18:19
  Planetary
• 18:20
  roller
• 18:20
  screws
• 18:20
  handle
• 18:20
  impact-bearing
• 18:21
  joints
• 18:21
  like
• 18:22
  knees
• 18:22
  and
• 18:22
  ankles:
• 18:23
  their
• 18:23
  threaded
• 18:24
  rollers
• 18:24
  make
• 18:24
  line
• 18:24
  contact
• 18:25
  with
• 18:25
  10–15×
• 18:27
  more
• 18:27
  surface
• 18:27
  area
• 18:27
  than
• 18:28
  ball
• 18:28
  screws,
• 18:28
  keeping
• 18:29
  peak
• 18:29
  Hertzian
• 18:29
  stress
• 18:30
  below
• 18:30
  yield
• 18:30
  threshold
• 18:31
  under
• 18:31
  repeated
• 18:32
  walking
• 18:32
  impacts.
• 18:33
  A
• 18:34
  ball
• 18:34
  screw
• 18:34
  rated
• 18:34
  for
• 18:35
  10
• 18:35
  million
• 18:35
  cycles
• 18:36
  might
• 18:36
  fail
• 18:36
  at
• 18:36
  100,000
• 18:37
  under
• 18:37
  walking
• 18:38
  loads
• 18:38
  —
• 18:39
  the
• 18:39
  rating
• 18:39
  assumes
• 18:39
  smooth,
• 18:40
  unidirectional
• 18:41
  force
• 18:41
  that
• 18:41
  never
• 18:42
  exists
• 18:42
  in
• 18:42
  a
• 18:42
  humanoid
• 18:43
  leg.
• 18:44
  The
• 18:44
  reflected
• 18:44
  inertia
• 18:45
  trap
• 18:45
  explains
• 18:45
  the
• 18:45
  gear
• 18:46
  ratio
• 18:46
  choices:
• 18:47
  a
• 18:47
  100:1
• 18:48
  gearbox
• 18:48
  multiplies
• 18:49
  motor
• 18:49
  inertia
• 18:49
  by
• 18:50
  10,000×
• 18:50
  at
• 18:50
  the
• 18:50
  output,
• 18:51
  making
• 18:51
  the
• 18:51
  leg
• 18:52
  feel
• 18:52
  like
• 18:52
  a
• 18:52
  brick
• 18:53
  rather
• 18:53
  than
• 18:53
  a
• 18:53
  spring.
• 18:54
  Thermal
• 18:55
  management
• 18:55
  is
• 18:55
  the
• 18:55
  final
• 18:56
  gatekeeper
• 18:56
  separating
• 18:57
  lab
• 18:57
  demonstrations
• 18:58
  from
• 18:58
  commercial
• 18:58
  products
• 18:58
  —
• 18:59
  continuous
• 19:00
  torque
• 19:00
  is
• 19:00
  typically
• 19:01
  only
• 19:01
  25–30%
• 19:03
  of
• 19:03
  peak
• 19:03
  in
• 19:03
  air-cooled
• 19:04
  actuators,
• 19:05
  and
• 19:05
  "peak
• 19:05
  torque
• 19:05
  impresses
• 19:06
  investors.
• 19:07
  Continuous
• 19:08
  torque
• 19:08
  —
• 19:09
  the
• 19:09
  ability
• 19:09
  to
• 19:09
  work
• 19:10
  all
• 19:10
  day
• 19:10
  without
• 19:10
  overheating
• 19:11
  —
• 19:11
  is
• 19:12
  what
• 19:12
  makes
• 19:12
  a