Latest / AGI Dreams – Open, Uncensored, & Local - AI News Digest / AGI Dreams Podcast – May 06, 2026

Transcript

AGI Dreams Podcast – May 06, 2026 · AGI Dreams – Open, Uncensored, & Local - AI News Digest

• 0:00
  The
• 0:00
  Cloud
• 0:00
  Security
• 0:00
  Alliance
• 0:01
  has
• 0:01
  published
• 0:01
  an
• 0:01
  updated
• 0:02
  Mythos
• 0:02
  document:
• 0:03
  a
• 0:03
  complete
• 0:03
  risk
• 0:03
  register,
• 0:04
  priority
• 0:04
  action
• 0:05
  table,
• 0:05
  and
• 0:05
  board
• 0:06
  briefing
• 0:06
  template
• 0:06
  for
• 0:06
  CISOs
• 0:07
  who
• 0:07
  need
• 0:07
  to
• 0:07
  walk
• 0:08
  into
• 0:08
  a
• 0:08
  room
• 0:08
  Monday
• 0:08
  morning
• 0:09
  with
• 0:09
  a
• 0:09
  plan.
• 0:09
  "The
• 0:10
  AI
• 0:10
  Vulnerability
• 0:11
  Storm"
• 0:11
  doesn't
• 0:11
  waste
• 0:12
  time
• 0:12
  on
• 0:12
  whether
• 0:12
  AI-driven
• 0:13
  vulnerability
• 0:13
  discovery
• 0:14
  is
• 0:14
  real
• 0:14
  —
• 0:15
  that
• 0:15
  argument
• 0:15
  ended
• 0:16
  when
• 0:16
  Mozilla
• 0:16
  reported
• 0:17
  271
• 0:18
  Firefox
• 0:18
  bugs
• 0:19
  found
• 0:19
  by
• 0:19
  Mythos
• 0:19
  (three
• 0:19
  warranting
• 0:20
  CVEs),
• 0:21
  and
• 0:21
  when
• 0:21
  MOAK
• 0:21
  demonstrated
• 0:22
  autonomous
• 0:22
  exploit
• 0:23
  generation
• 0:23
  against
• 0:23
  98%
• 0:24
  of
• 0:24
  open-source
• 0:25
  KEVs
• 0:25
  using
• 0:25
  publicly
• 0:26
  available
• 0:26
  frontier
• 0:27
  models.
• 0:27
  Instead,
• 0:28
  the
• 0:28
  paper
• 0:28
  focuses
• 0:28
  on
• 0:29
  the
• 0:29
  structural
• 0:29
  asymmetry:
• 0:30
  AI
• 0:30
  compresses
• 0:31
  the
• 0:31
  disclosure-to-weaponization
• 0:32
  timeline
• 0:32
  to
• 0:32
  hours
• 0:33
  while
• 0:33
  defenders
• 0:33
  still
• 0:34
  operate
• 0:34
  on
• 0:34
  quarterly
• 0:35
  patch
• 0:35
  cycles.
• 0:36
  The
• 0:36
  paper
• 0:36
  introduces
• 0:37
  VulnOps
• 0:37
  as
• 0:37
  a
• 0:37
  permanent
• 0:38
  organizational
• 0:38
  function
• 0:38
  —
• 0:39
  staffed
• 0:39
  and
• 0:40
  automated
• 0:40
  like
• 0:40
  DevOps,
• 0:41
  but
• 0:41
  for
• 0:41
  continuous
• 0:42
  zero-day
• 0:42
  discovery
• 0:43
  and
• 0:43
  remediation
• 0:43
  across
• 0:44
  an
• 0:44
  organization's
• 0:44
  entire
• 0:45
  software
• 0:45
  estate.
• 0:46
  The
• 0:46
  twelve-item
• 0:47
  risk
• 0:47
  register
• 0:47
  maps
• 0:48
  each
• 0:48
  threat
• 0:48
  to
• 0:48
  OWASP
• 0:48
  LLM
• 0:49
  2025,
• 0:50
  OWASP
• 0:50
  Agentic
• 0:50
  2026,
• 0:51
  MITRE
• 0:52
  ATLAS,
• 0:53
  and
• 0:53
  NIST
• 0:53
  CSF
• 0:54
  2.0
• 0:54
  controls,
• 0:55
  with
• 0:55
  honest
• 0:55
  acknowledgments
• 0:56
  of
• 0:56
  where
• 0:56
  frameworks
• 0:57
  don't
• 0:57
  yet
• 0:57
  cover
• 0:57
  the
• 0:57
  new
• 0:57
  threat
• 0:58
  landscape.
• 0:59
  The
• 0:59
  most
• 0:59
  pointed
• 0:59
  recommendation:
• 1:00
  formalize
• 1:01
  AI
• 1:01
  agent
• 1:01
  usage
• 1:01
  across
• 1:02
  all
• 1:02
  security
• 1:02
  functions
• 1:03
  immediately,
• 1:03
  with
• 1:03
  mandatory
• 1:04
  security
• 1:04
  controls.
• 1:05
  "Without
• 1:06
  agents,
• 1:06
  most
• 1:06
  tasks
• 1:07
  on
• 1:07
  this
• 1:07
  list
• 1:07
  will
• 1:07
  be
• 1:07
  untenable,"
• 1:08
  the
• 1:08
  paper
• 1:08
  states,
• 1:09
  "but
• 1:09
  they
• 1:09
  must
• 1:10
  be
• 1:10
  defended."
• 1:11
  The
• 1:11
  agent
• 1:11
  harness
• 1:11
  itself
• 1:12
  —
• 1:13
  prompts,
• 1:13
  tool
• 1:14
  definitions,
• 1:14
  retrieval
• 1:15
  pipelines,
• 1:16
  escalation
• 1:16
  logic
• 1:16
  —
• 1:17
  is
• 1:17
  called
• 1:17
  out
• 1:18
  as
• 1:18
  the
• 1:18
  new
• 1:18
  attack
• 1:18
  surface
• 1:19
  requiring
• 1:19
  the
• 1:19
  same
• 1:20
  audit
• 1:20
  rigor
• 1:20
  as
• 1:20
  the
• 1:20
  agent's
• 1:21
  permissions.
• 1:22
  The
• 1:22
  paper
• 1:22
  draws
• 1:23
  a
• 1:23
  direct
• 1:23
  comparison
• 1:24
  to
• 1:24
  Y2K:
• 1:25
  a
• 1:25
  systemic
• 1:26
  threat
• 1:26
  with
• 1:26
  a
• 1:26
  hard
• 1:26
  deadline
• 1:27
  that
• 1:27
  the
• 1:27
  industry
• 1:27
  met
• 1:28
  through
• 1:28
  coordinated,
• 1:29
  disciplined
• 1:29
  effort.
• 1:31
  Ten
• 1:31
  diagnostic
• 1:31
  questions
• 1:32
  help
• 1:33
  CISOs
• 1:33
  gauge
• 1:33
  their
• 1:34
  actual
• 1:34
  security
• 1:35
  posture,
• 1:35
  from
• 1:36
  whether
• 1:36
  employees
• 1:36
  can
• 1:37
  use
• 1:37
  agentic
• 1:37
  coding
• 1:37
  tools
• 1:38
  today
• 1:38
  to
• 1:39
  whether
• 1:39
  executive
• 1:39
  leadership
• 1:40
  has
• 1:40
  a
• 1:40
  working
• 1:40
  definition
• 1:41
  of
• 1:41
  urgency.
• 1:42
  Meanwhile,
• 1:43
  a
• 1:43
  hands-on
• 1:43
  demonstration
• 1:44
  of
• 1:44
  defense-in-depth
• 1:45
  arrived
• 1:45
  via
• 1:45
  CVE-2026-31431,
• 1:50
  known
• 1:50
  as
• 1:51
  "Copy
• 1:51
  Fail."
• 1:52
  The
• 1:52
  vulnerability
• 1:53
  exploits
• 1:53
  a
• 1:54
  flaw
• 1:54
  in
• 1:54
  the
• 1:54
  Linux
• 1:55
  kernel's
• 1:55
  AF
• 1:55
  ALG
• 1:56
  cryptographic
• 1:57
  API,
• 1:58
  where
• 1:58
  a
• 1:58
  race
• 1:59
  condition
• 1:59
  in
• 1:59
  the
• 1:59
  AEAD
• 1:59
  scatter/gather
• 2:01
  implementation
• 2:02
  allows
• 2:02
  an
• 2:02
  unprivileged
• 2:03
  user
• 2:03
  to
• 2:03
  corrupt
• 2:04
  the
• 2:04
  page
• 2:04
  cache
• 2:05
  and
• 2:05
  overwrite
• 2:05
  arbitrary
• 2:06
  binaries
• 2:06
  —
• 2:07
  specifically
• 2:08
  by
• 2:08
  sending
• 2:09
  crafted
• 2:09
  data
• 2:09
  through
• 2:10
  authencesn(hmac(sha256),cbc(aes))
• 2:14
  and
• 2:14
  using
• 2:14
  splice()
• 2:15
  to
• 2:15
  deposit
• 2:15
  shellcode
• 2:16
  directly
• 2:16
  into
• 2:16
  cached
• 2:17
  pages
• 2:17
  of
• 2:17
  /usr/sbin/su.
• 2:23
  A
• 2:23
  researcher
• 2:23
  at
• 2:23
  GNOME's
• 2:24
  GitLab
• 2:24
  infrastructure
• 2:25
  ran
• 2:25
  the
• 2:25
  full
• 2:25
  exploit
• 2:26
  chain
• 2:26
  inside
• 2:27
  a
• 2:27
  rootless
• 2:27
  Podman
• 2:27
  container
• 2:28
  and
• 2:28
  traced
• 2:29
  the
• 2:29
  kernel's
• 2:29
  response
• 2:30
  with
• 2:30
  eBPF.
• 2:31
  The
• 2:31
  result:
• 2:32
  setuid(0)
• 2:33
  returned
• 2:33
  success
• 2:34
  inside
• 2:34
  the
• 2:34
  user
• 2:35
  namespace,
• 2:36
  but
• 2:36
  container
• 2:36
  root
• 2:36
  mapped
• 2:37
  to
• 2:37
  UID
• 2:37
  1000
• 2:38
  on
• 2:38
  the
• 2:38
  host
• 2:38
  —
• 2:39
  an
• 2:39
  unprivileged
• 2:40
  user.
• 2:41
  The
• 2:41
  exploit
• 2:41
  achieved
• 2:42
  full
• 2:42
  privilege
• 2:42
  escalation
• 2:43
  within
• 2:43
  the
• 2:43
  namespace
• 2:44
  boundary,
• 2:44
  and
• 2:45
  none
• 2:45
  of
• 2:45
  it
• 2:45
  mattered
• 2:45
  outside.
• 2:47
  One
• 2:47
  important
• 2:47
  caveat:
• 2:48
  the
• 2:48
  page
• 2:49
  cache
• 2:49
  is
• 2:49
  shared
• 2:50
  across
• 2:50
  the
• 2:50
  host,
• 2:51
  so
• 2:51
  containers
• 2:51
  reusing
• 2:52
  the
• 2:52
  same
• 2:52
  base
• 2:53
  image
• 2:53
  layers
• 2:53
  could
• 2:53
  execute
• 2:54
  poisoned
• 2:54
  binaries
• 2:55
  —
• 2:56
  a
• 2:56
  cross-container
• 2:56
  isolation
• 2:57
  break
• 2:57
  that
• 2:57
  never
• 2:58
  requires
• 2:58
  escaping
• 2:59
  to
• 2:59
  the
• 2:59
  host.
• 3:00
  For
• 3:00
  production
• 3:00
  environments,
• 3:01
  the
• 3:01
  researcher
• 3:02
  recommends
• 3:02
  enabling
• 3:03
  User
• 3:03
  Namespace
• 3:04
  support
• 3:04
  for
• 3:04
  OpenShift
• 3:05
  pods
• 3:05
  (GA
• 3:06
  since
• 3:06
  4.20)
• 3:07
  and
• 3:07
  investigating
• 3:08
  ephemeral
• 3:08
  microVMs
• 3:09
  via
• 3:09
  Cloud
• 3:09
  Hypervisor
• 3:10
  for
• 3:10
  CI/CD
• 3:14
  The
• 3:14
  agentic
• 3:15
  coding
• 3:15
  community
• 3:16
  has
• 3:16
  largely
• 3:16
  settled
• 3:17
  the
• 3:17
  "whether"
• 3:17
  question
• 3:18
  and
• 3:18
  moved
• 3:18
  deep
• 3:19
  into
• 3:19
  "how."
• 3:20
  Drew
• 3:20
  Breunig's
• 3:21
  "10
• 3:21
  Lessons
• 3:22
  for
• 3:22
  Agentic
• 3:22
  Coding"
• 3:23
  distills
• 3:23
  the
• 3:23
  emerging
• 3:24
  consensus
• 3:25
  into
• 3:25
  durable
• 3:25
  guidelines
• 3:26
  that
• 3:26
  should
• 3:26
  survive
• 3:26
  at
• 3:27
  least
• 3:27
  a
• 3:27
  few
• 3:27
  model
• 3:27
  generations.
• 3:29
  The
• 3:29
  headline
• 3:29
  insight:
• 3:29
  "Code
• 3:30
  is
• 3:30
  cheap,
• 3:30
  but
• 3:31
  maintenance,
• 3:31
  support,
• 3:32
  and
• 3:32
  security
• 3:32
  When
• 3:34
  implementation
• 3:35
  costs
• 3:35
  approach
• 3:35
  zero,
• 3:36
  the
• 3:36
  bottleneck
• 3:36
  shifts
• 3:37
  to
• 3:37
  judgment
• 3:37
  —
• 3:38
  knowing
• 3:38
  which
• 3:39
  code
• 3:39
  to
• 3:39
  write,
• 3:40
  maintaining
• 3:40
  behavioral
• 3:41
  contracts
• 3:41
  through
• 3:42
  tests
• 3:42
  that
• 3:42
  measure
• 3:42
  what
• 3:43
  a
• 3:43
  product
• 3:43
  does
• 3:43
  rather
• 3:44
  than
• 3:44
  how
• 3:44
  it
• 3:44
  does
• 3:44
  it,
• 3:45
  and
• 3:45
  keeping
• 3:45
  specs
• 3:46
  in
• 3:46
  sync
• 3:46
  as
• 3:46
  living
• 3:46
  documents
• 3:47
  rather
• 3:47
  than
• 3:47
  frozen
• 3:48
  artifacts.
• 3:49
  Lesson
• 3:49
  nine
• 3:50
  captures
• 3:50
  the
• 3:50
  hidden
• 3:50
  advantage
• 3:51
  of
• 3:51
  experienced
• 3:51
  developers:
• 3:52
  they
• 3:52
  bring
• 3:53
  intuition
• 3:53
  to
• 3:53
  their
• 3:53
  prompts
• 3:53
  —
• 3:54
  the
• 3:55
  right
• 3:55
  terms,
• 3:56
  the
• 3:56
  right
• 3:56
  framing,
• 3:57
  the
• 3:57
  right
• 3:57
  level
• 3:57
  of
• 3:57
  specificity
• 3:58
  —
• 3:59
  saving
• 4:00
  countless
• 4:00
  cycles
• 4:00
  during
• 4:01
  both
• 4:01
  implementation
• 4:01
  and
• 4:02
  debugging.
• 4:03
  Pair
• 4:03
  that
• 4:03
  technical
• 4:04
  expertise
• 4:04
  with
• 4:04
  great
• 4:04
  taste,
• 4:05
  Breunig
• 4:05
  argues,
• 4:06
  and
• 4:06
  you
• 4:06
  have
• 4:06
  an
• 4:07
  unbeatable
• 4:07
  advantage.
• 4:08
  That
• 4:08
  intuition
• 4:09
  is
• 4:09
  increasingly
• 4:09
  being
• 4:10
  codified
• 4:10
  into
• 4:10
  layered
• 4:11
  architectures.
• 4:12
  Claude
• 4:12
  Code's
• 4:12
  multi-stage
• 4:13
  agentic
• 4:13
  system
• 4:14
  illustrates
• 4:14
  the
• 4:15
  pattern:
• 4:15
  CLAUDE.md
• 4:16
  files
• 4:17
  serve
• 4:17
  as
• 4:17
  persistent
• 4:18
  organizational
• 4:19
  memory
• 4:19
  and
• 4:19
  constitution,
• 4:20
  Skills
• 4:21
  provide
• 4:21
  modular
• 4:22
  on-demand
• 4:22
  expertise
• 4:22
  (database
• 4:23
  migration,
• 4:24
  security
• 4:24
  review,
• 4:25
  Rust
• 4:25
  optimization),
• 4:26
  Hooks
• 4:27
  add
• 4:27
  deterministic
• 4:27
  guardrails
• 4:28
  around
• 4:28
  events
• 4:28
  like
• 4:28
  file
• 4:29
  writes
• 4:29
  and
• 4:29
  tool
• 4:29
  calls,
• 4:30
  and
• 4:30
  Subagents
• 4:31
  operate
• 4:31
  as
• 4:31
  specialized
• 4:32
  delegated
• 4:32
  reasoning
• 4:33
  units
• 4:33
  with
• 4:33
  bounded
• 4:33
  scope
• 4:34
  and
• 4:34
  isolated
• 4:34
  context.
• 4:36
  The
• 4:36
  result,
• 4:36
  as
• 4:36
  Reuven
• 4:37
  Cohen
• 4:37
  describes
• 4:37
  it,
• 4:38
  is
• 4:38
  "a
• 4:38
  layered
• 4:38
  cognitive
• 4:39
  stack
• 4:39
  where
• 4:39
  memory,
• 4:40
  expertise,
• 4:41
  governance,
• 4:41
  delegation,
• 4:42
  and
• 4:42
  execution
• 4:42
  are
• 4:43
  separated
• 4:43
  into
• 4:43
  composable
• 4:44
  primitives."
• 4:45
  It
• 4:45
  starts
• 4:45
  looking
• 4:46
  less
• 4:46
  like
• 4:46
  a
• 4:46
  single
• 4:46
  model
• 4:47
  responding
• 4:47
  to
• 4:47
  prompts
• 4:48
  and
• 4:48
  more
• 4:48
  like
• 4:48
  a
• 4:48
  coordinated
• 4:49
  engineering
• 4:49
  organization.
• 4:50
  The
• 4:50
  details
• 4:51
  matter
• 4:51
  at
• 4:51
  every
• 4:51
  layer.
• 4:52
  Claude
• 4:53
  Code
• 4:53
  versions
• 4:53
  2.1.124
• 4:55
  and
• 4:55
  2.1.126
• 4:58
  reveal
• 4:58
  the
• 4:58
  fine-grained
• 4:59
  tuning
• 4:59
  happening
• 4:59
  inside
• 5:00
  the
• 5:00
  harness:
• 5:01
  a
• 5:01
  new
• 5:01
  system
• 5:01
  reminder
• 5:02
  tells
• 5:02
  the
• 5:02
  agent
• 5:02
  when
• 5:02
  a
• 5:02
  file
• 5:03
  was
• 5:03
  modified
• 5:04
  but
• 5:04
  the
• 5:04
  diff
• 5:04
  was
• 5:04
  omitted
• 5:04
  due
• 5:05
  to
• 5:05
  budget
• 5:05
  constraints,
• 5:06
  directing
• 5:06
  it
• 5:06
  to
• 5:06
  re-read
• 5:07
  if
• 5:07
  needed
• 5:07
  —
• 5:08
  a
• 5:08
  small
• 5:08
  fix
• 5:08
  that
• 5:08
  prevents
• 5:09
  wasted
• 5:09
  tokens
• 5:09
  from
• 5:09
  stale
• 5:10
  content.
• 5:11
  More
• 5:11
  revealing
• 5:12
  is
• 5:12
  the
• 5:12
  removal
• 5:12
  of
• 5:12
  a
• 5:12
  reminder
• 5:13
  that
• 5:13
  asked
• 5:13
  agents
• 5:14
  to
• 5:14
  consider
• 5:14
  whether
• 5:14
  each
• 5:15
  file
• 5:15
  read
• 5:15
  might
• 5:15
  be
• 5:15
  malware,
• 5:16
  a
• 5:16
  guardrail
• 5:17
  apparently
• 5:17
  deemed
• 5:18
  more
• 5:18
  noisy
• 5:18
  than
• 5:18
  useful.
• 5:19
  As
• 5:20
  agentic
• 5:20
  sessions
• 5:20
  grow
• 5:20
  longer
• 5:21
  and
• 5:21
  more
• 5:21
  autonomous,
• 5:22
  observability
• 5:23
  becomes
• 5:23
  critical.
• 5:24
  Argus,
• 5:25
  a
• 5:25
  new
• 5:25
  open-source
• 5:26
  VSCode
• 5:26
  extension,
• 5:27
  provides
• 5:27
  timeline
• 5:28
  visualization
• 5:28
  of
• 5:29
  tool
• 5:29
  calls,
• 5:29
  cost
• 5:30
  tracking,
• 5:30
  and
• 5:30
  context
• 5:31
  usage
• 5:31
  across
• 5:31
  Claude
• 5:32
  Code
• 5:32
  sessions
• 5:32
  —
• 5:34
  addressing
• 5:34
  what
• 5:34
  one
• 5:34
  commenter
• 5:35
  called
• 5:35
  an
• 5:35
  "underrated"
• 5:36
  need
• 5:36
  for
• 5:36
  comparing
• 5:36
  whether
• 5:37
  an
• 5:37
  agent
• 5:37
  actually
• 5:38
  improved
• 5:38
  across
• 5:38
  runs
• 5:39
  or
• 5:39
  just
• 5:39
  got
• 5:39
  lucky
• 5:40
  once.
• 5:43
  The
• 5:43
  Unhyped
• 5:43
  AI
• 5:44
  newsletter
• 5:44
  published
• 5:45
  a
• 5:45
  carefully
• 5:45
  argued
• 5:46
  essay
• 5:46
  on
• 5:46
  what
• 5:46
  may
• 5:46
  be
• 5:47
  the
• 5:47
  most
• 5:47
  subtle
• 5:47
  risk
• 5:48
  in
• 5:48
  the
• 5:48
  current
• 5:48
  AI
• 5:49
  moment:
• 5:50
  the
• 5:50
  ease
• 5:50
  with
• 5:51
  which
• 5:51
  AI
• 5:51
  can
• 5:51
  automate
• 5:52
  the
• 5:52
  appearance
• 5:52
  of
• 5:52
  strategy
• 5:53
  without
• 5:53
  producing
• 5:54
  the
• 5:54
  thing
• 5:54
  itself.
• 5:55
  "A
• 5:55
  real
• 5:56
  strategy
• 5:56
  document
• 5:57
  does
• 5:57
  not
• 5:57
  merely
• 5:57
  sound
• 5:58
  convincing.
• 5:59
  It
• 5:59
  carries
• 6:00
  choice,
• 6:00
  exclusion,
• 6:01
  and
• 6:01
  a
• 6:01
  theory
• 6:02
  of
• 6:02
  consequence,"
• 6:03
  writes
• 6:03
  the
• 6:03
  author.
• 6:04
  The
• 6:04
  piece
• 6:05
  identifies
• 6:05
  a
• 6:05
  second-order
• 6:06
  problem
• 6:06
  worth
• 6:06
  naming:
• 6:07
  AI-polished
• 6:08
  strategy
• 6:08
  documents
• 6:09
  make
• 6:09
  disagreement
• 6:10
  harder
• 6:10
  to
• 6:10
  start.
• 6:11
  When
• 6:11
  someone
• 6:12
  hands
• 6:12
  you
• 6:12
  a
• 6:12
  rough
• 6:12
  draft
• 6:13
  with
• 6:13
  obvious
• 6:13
  gaps,
• 6:14
  you
• 6:14
  argue
• 6:14
  with
• 6:14
  it.
• 6:15
  When
• 6:15
  someone
• 6:15
  hands
• 6:16
  you
• 6:16
  a
• 6:16
  polished
• 6:16
  document
• 6:17
  with
• 6:17
  no
• 6:17
  visible
• 6:17
  seams,
• 6:18
  the
• 6:18
  social
• 6:19
  cost
• 6:19
  of
• 6:19
  saying
• 6:19
  "this
• 6:20
  is
• 6:20
  wrong"
• 6:20
  goes
• 6:20
  up.
• 6:21
  The
• 6:21
  prose
• 6:21
  quality
• 6:22
  becomes
• 6:22
  armor.
• 6:23
  The
• 6:23
  essay
• 6:23
  isn't
• 6:24
  anti-AI
• 6:24
  —
• 6:25
  it
• 6:25
  endorses
• 6:25
  using
• 6:26
  AI
• 6:26
  to
• 6:26
  "gather
• 6:27
  threads,
• 6:27
  test
• 6:27
  framings,
• 6:28
  surface
• 6:28
  angles,
• 6:29
  clear
• 6:29
  undergrowth"
• 6:29
  —
• 6:30
  but
• 6:30
  draws
• 6:30
  a
• 6:31
  hard
• 6:31
  line
• 6:31
  between
• 6:31
  aiding
• 6:31
  thought
• 6:32
  and
• 6:32
  counterfeiting
• 6:32
  it.
• 6:33
  "The
• 6:34
  serpent
• 6:34
  has
• 6:34
  not
• 6:34
  merely
• 6:35
  eaten
• 6:35
  its
• 6:35
  tail.
• 6:36
  It
• 6:36
  has
• 6:36
  offered
• 6:36
  to
• 6:36
  draft
• 6:37
  the
• 6:37
  governance
• 6:37
  memo
• 6:38
  about
• 6:38
  the
• 6:38
  meal."
• 6:39
  The
• 6:39
  autonomy
• 6:39
  gap
• 6:39
  extends
• 6:40
  well
• 6:40
  beyond
• 6:40
  strategy
• 6:41
  documents
• 6:41
  into
• 6:42
  the
• 6:42
  entire
• 6:42
  consumer
• 6:43
  agent
• 6:43
  landscape.
• 6:44
  A
• 6:44
  detailed
• 6:44
  analysis
• 6:45
  of
• 6:45
  current
• 6:45
  agent
• 6:45
  products
• 6:46
  identifies
• 6:46
  what
• 6:46
  it
• 6:46
  calls
• 6:47
  the
• 6:47
  "anticipation
• 6:47
  gap"
• 6:48
  —
• 6:48
  the
• 6:48
  distance
• 6:49
  between
• 6:49
  agents
• 6:49
  that
• 6:49
  can
• 6:50
  act
• 6:50
  and
• 6:50
  agents
• 6:50
  that
• 6:51
  know
• 6:51
  when
• 6:51
  to
• 6:51
  act
• 6:51
  without
• 6:52
  being
• 6:52
  asked.
• 6:53
  The
• 6:53
  core
• 6:53
  observation:
• 6:54
  coding
• 6:54
  agents
• 6:55
  succeed
• 6:55
  because
• 6:55
  they
• 6:55
  have
• 6:56
  clean
• 6:56
  verification
• 6:56
  (tests
• 6:57
  pass
• 6:57
  or
• 6:57
  fail),
• 6:58
  bounded
• 6:58
  scope
• 6:58
  (fix
• 6:59
  this
• 6:59
  bug),
• 6:59
  and
• 6:59
  cheap
• 7:00
  iteration.
• 7:01
  Consumer
• 7:01
  life
• 7:01
  has
• 7:01
  none
• 7:02
  of
• 7:02
  "Did
• 7:03
  the
• 7:03
  agent
• 7:03
  book
• 7:03
  the
• 7:03
  right
• 7:03
  flight?
• 7:04
  How
• 7:04
  do
• 7:04
  you
• 7:04
  define
• 7:05
  right?
• 7:05
  There's
• 7:06
  not
• 7:06
  a
• 7:06
  compiler
• 7:06
  for
• 7:06
  taste."
• 7:08
  The
• 7:08
  proposed
• 7:08
  framework
• 7:09
  is
• 7:09
  a
• 7:09
  five-step
• 7:09
  permission
• 7:10
  ladder:
• 7:10
  read,
• 7:11
  suggest,
• 7:12
  draft,
• 7:12
  act-with-confirmation,
• 7:13
  and
• 7:14
  act
• 7:14
  autonomously.
• 7:16
  Most
• 7:16
  consumer
• 7:16
  agents
• 7:17
  are
• 7:17
  stuck
• 7:17
  at
• 7:17
  step
• 7:17
  one,
• 7:18
  and
• 7:18
  products
• 7:18
  that
• 7:18
  try
• 7:19
  to
• 7:19
  jump
• 7:19
  to
• 7:19
  step
• 7:19
  five
• 7:20
  without
• 7:20
  earning
• 7:20
  trust
• 7:20
  fail
• 7:21
  badly.
• 7:22
  Symphony
• 7:22
  —
• 7:23
  OpenAI's
• 7:23
  open-source
• 7:24
  orchestration
• 7:24
  protocol
• 7:25
  —
• 7:25
  was
• 7:25
  born
• 7:26
  because
• 7:26
  "engineers
• 7:27
  had
• 7:27
  fast
• 7:27
  coding
• 7:27
  agents,
• 7:28
  but
• 7:28
  people
• 7:28
  were
• 7:28
  still
• 7:29
  opening
• 7:29
  their
• 7:29
  sessions
• 7:30
  and
• 7:30
  assigning
• 7:30
  tasks
• 7:31
  and
• 7:31
  checking
• 7:31
  progress
• 7:32
  and
• 7:32
  nudging
• 7:32
  agents."
• 7:33
  Even
• 7:33
  sophisticated
• 7:34
  users
• 7:34
  are
• 7:34
  hitting
• 7:35
  human
• 7:35
  attention
• 7:35
  Anthropic's
• 7:37
  Project
• 7:38
  Deal
• 7:38
  experiment
• 7:38
  offers
• 7:39
  a
• 7:39
  controlled
• 7:39
  test
• 7:40
  of
• 7:40
  what
• 7:40
  happens
• 7:40
  when
• 7:40
  agents
• 7:41
  negotiate
• 7:41
  on
• 7:41
  behalf
• 7:42
  of
• 7:42
  humans.
• 7:43
  The
• 7:43
  company
• 7:43
  created
• 7:44
  an
• 7:44
  internal
• 7:44
  marketplace
• 7:45
  where
• 7:45
  Claude
• 7:45
  bought,
• 7:46
  sold,
• 7:47
  and
• 7:47
  negotiated
• 7:47
  for
• 7:47
  employees
• 7:48
  —
• 7:48
  and
• 7:49
  found
• 7:49
  that
• 7:49
  more
• 7:49
  intelligent
• 7:50
  models
• 7:50
  sold
• 7:50
  items
• 7:51
  at
• 7:51
  higher
• 7:51
  prices.
• 7:52
  The
• 7:52
  community
• 7:52
  reaction
• 7:53
  was
• 7:53
  predictably
• 7:53
  split:
• 7:54
  one
• 7:54
  camp
• 7:55
  sees
• 7:55
  the
• 7:55
  dystopian
• 7:56
  future
• 7:56
  of
• 7:56
  personalized
• 7:57
  price
• 7:57
  gouging
• 7:57
  at
• 7:57
  scale,
• 7:58
  another
• 7:58
  points
• 7:59
  out
• 7:59
  that
• 7:59
  airlines
• 7:59
  have
• 7:59
  been
• 8:00
  doing
• 8:00
  dynamic
• 8:00
  pricing
• 8:01
  with
• 8:01
  ML
• 8:01
  for
• 8:01
  years,
• 8:02
  and
• 8:02
  a
• 8:02
  third
• 8:02
  notes
• 8:03
  the
• 8:03
  experiment
• 8:03
  had
• 8:03
  a
• 8:03
  sample
• 8:04
  size
• 8:04
  of
• 8:04
  one
• 8:04
  with
• 8:04
  questionable
• 8:05
  math
• 8:06
  ("How
• 8:06
  did
• 8:06
  they
• 8:06
  sell
• 8:06
  the
• 8:06
  same
• 8:07
  bike
• 8:07
  twice?").
• 8:08
  The
• 8:08
  gap
• 8:08
  between
• 8:08
  agent
• 8:09
  capability
• 8:09
  and
• 8:09
  trustworthy
• 8:10
  autonomy
• 8:10
  is
• 8:10
  also
• 8:11
  visible
• 8:11
  in
• 8:11
  content
• 8:12
  generation:
• 8:13
  a
• 8:13
  Reddit
• 8:13
  post
• 8:13
  documenting
• 8:14
  45
• 8:14
  seconds
• 8:15
  of
• 8:15
  Facebook
• 8:15
  confidently
• 8:16
  attributing
• 8:16
  the
• 8:16
  White
• 8:16
  House
• 8:16
  shooter
• 8:17
  as
• 8:17
  a
• 8:17
  former
• 8:17
  staffer
• 8:18
  of
• 8:18
  nearly
• 8:18
  every
• 8:18
  major
• 8:19
  sports
• 8:19
  team
• 8:20
  illustrates
• 8:20
  how
• 8:20
  generated
• 8:21
  content
• 8:21
  passes
• 8:21
  through
• 8:22
  platforms
• 8:22
  at
• 8:22
  scale
• 8:23
  without
• 8:23
  verification
• 8:23
  —
• 8:24
  the
• 8:24
  AI-citing-AI
• 8:25
  feedback
• 8:26
  loop
• 8:26
  producing
• 8:26
  confidently
• 8:27
  wrong
• 8:27
  fabrications
• 8:28
  at
• 8:28
  industrial
• 8:28
  speed.
• 8:31
  AMD's
• 8:32
  leaked
• 8:32
  Ryzen
• 8:33
  AI
• 8:33
  Max+
• 8:34
  495
• 8:35
  "Gorgon
• 8:35
  Halo"
• 8:35
  APU
• 8:36
  pushes
• 8:36
  unified
• 8:36
  memory
• 8:37
  to
• 8:37
  192GB
• 8:38
  —
• 8:38
  enough
• 8:39
  to
• 8:39
  load
• 8:39
  models
• 8:39
  that
• 8:39
  previously
• 8:40
  required
• 8:40
  multi-GPU
• 8:41
  rigs.
• 8:42
  But
• 8:42
  the
• 8:42
  community
• 8:42
  response
• 8:43
  is
• 8:43
  measured.
• 8:44
  Memory
• 8:44
  bandwidth
• 8:44
  appears
• 8:45
  to
• 8:45
  remain
• 8:45
  at
• 8:45
  256GB/s
• 8:47
  theoretical
• 8:48
  (measured
• 8:48
  at
• 8:48
  roughly
• 8:48
  180GB/s
• 8:50
  in
• 8:50
  llama.cpp
• 8:51
  on
• 8:51
  current
• 8:52
  Strix
• 8:52
  Halo
• 8:52
  hardware),
• 8:53
  and
• 8:53
  the
• 8:53
  RDNA3
• 8:55
  iGPU's
• 8:55
  compute
• 8:55
  efficiency
• 8:56
  tops
• 8:56
  out
• 8:56
  around
• 8:57
  62%
• 8:57
  of
• 8:58
  theoretical
• 8:58
  FP16
• 8:59
  TFLOPS.
• 9:01
  As
• 9:01
  one
• 9:01
  commenter
• 9:02
  who
• 9:02
  has
• 9:02
  done
• 9:02
  extensive
• 9:03
  kernel
• 9:03
  work
• 9:03
  on
• 9:03
  RDNA3
• 9:04
  notes:
• 9:05
  "At
• 9:05
  long
• 9:05
  context,
• 9:06
  compute
• 9:06
  is
• 9:07
  actually
• 9:07
  what's
• 9:07
  killing
• 9:07
  decode
• 9:08
  speed.
• 9:08
  While
• 9:09
  the
• 9:09
  AMD
• 9:09
  APUs
• 9:10
  remain
• 9:10
  on
• 9:10
  RDNA3,
• 9:11
  this
• 9:11
  won't
• 9:12
  change."
• 9:12
  The
• 9:13
  consensus
• 9:13
  is
• 9:13
  to
• 9:13
  wait
• 9:13
  for
• 9:14
  2027's
• 9:14
  Medusa
• 9:15
  Halo,
• 9:15
  which
• 9:16
  may
• 9:16
  move
• 9:16
  to
• 9:16
  a
• 9:16
  384-bit
• 9:18
  LPDDR6
• 9:19
  bus
• 9:19
  and
• 9:19
  RDNA5
• 9:20
  —
• 9:21
  addressing
• 9:21
  both
• 9:21
  bandwidth
• 9:22
  and
• 9:22
  compute
• 9:22
  efficiency
• 9:23
  simultaneously.
• 9:24
  For
• 9:24
  those
• 9:24
  running
• 9:25
  NVIDIA
• 9:25
  consumer
• 9:26
  hardware
• 9:26
  today,
• 9:27
  the
• 9:27
  club-3090
• 9:28
  repository
• 9:28
  has
• 9:29
  emerged
• 9:29
  as
• 9:29
  a
• 9:29
  definitive
• 9:29
  community
• 9:30
  resource.
• 9:31
  It
• 9:31
  provides
• 9:31
  validated
• 9:32
  Docker
• 9:32
  Compose
• 9:33
  configs,
• 9:34
  benchmark
• 9:34
  scripts,
• 9:35
  and
• 9:35
  multi-engine
• 9:35
  support
• 9:36
  (vLLM,
• 9:37
  llama.cpp,
• 9:38
  SGLang)
• 9:39
  for
• 9:39
  serving
• 9:40
  models
• 9:40
  like
• 9:40
  Qwen3.6-27B
• 9:44
  on
• 9:44
  one
• 9:44
  or
• 9:44
  two
• 9:44
  RTX
• 9:45
  3090s.
• 9:46
  The
• 9:46
  dual-card
• 9:47
  vLLM
• 9:47
  configuration
• 9:48
  hits
• 9:48
  127
• 9:49
  tokens
• 9:49
  per
• 9:50
  second
• 9:50
  with
• 9:50
  DFlash
• 9:51
  drafting,
• 9:51
  while
• 9:52
  llama.cpp
• 9:53
  on
• 9:53
  a
• 9:53
  single
• 9:54
  card
• 9:54
  handles
• 9:54
  the
• 9:54
  full
• 9:55
  262K
• 9:56
  context
• 9:56
  window
• 9:57
  without
• 9:57
  prefill
• 9:57
  cliffs
• 9:58
  at
• 9:58
  roughly
• 9:58
  21
• 9:58
  TPS.
• 9:59
  The
• 10:00
  repo's
• 10:00
  stress-testing
• 10:01
  suite
• 10:01
  —
• 10:01
  including
• 10:02
  a
• 10:02
  7-check
• 10:02
  boundary-case
• 10:03
  stress
• 10:03
  test
• 10:04
  and
• 10:04
  multi-turn
• 10:04
  agent
• 10:05
  traffic
• 10:05
  soak
• 10:05
  test
• 10:05
  —
• 10:06
  reflects
• 10:06
  hobbyist
• 10:07
  infrastructure
• 10:07
  that
• 10:08
  now
• 10:08
  rivals
• 10:08
  production-grade
• 10:09
  serving
• 10:09
  setups.
• 10:10
  Tensor
• 10:10
  parallelism
• 10:11
  is
• 10:11
  finally
• 10:11
  arriving
• 10:12
  in
• 10:12
  llama.cpp,
• 10:14
  and
• 10:14
  the
• 10:14
  benchmarks
• 10:14
  show
• 10:15
  why
• 10:15
  it
• 10:15
  matters.
• 10:16
  Running
• 10:17
  Mistral
• 10:17
  Medium
• 10:17
  3.5
• 10:18
  128B
• 10:19
  (IQ4
• 10:20
  XS,
• 10:21
  62.5
• 10:21
  GiB)
• 10:22
  across
• 10:23
  four
• 10:23
  RTX
• 10:23
  3080
• 10:24
  20GB
• 10:25
  cards,
• 10:26
  tensor-parallel
• 10:26
  mode
• 10:27
  delivers
• 10:27
  21.6
• 10:28
  tokens/sec
• 10:29
  versus
• 10:29
  10.4
• 10:30
  with
• 10:31
  layer
• 10:31
  splitting
• 10:31
  —
• 10:31
  a
• 10:32
  clean
• 10:32
  2x
• 10:32
  improvement
• 10:33
  for
• 10:33
  generation.
• 10:34
  The
• 10:34
  MoE
• 10:34
  model
• 10:35
  Qwen
• 10:35
  3.5
• 10:37
  122B-A10B
• 10:38
  tells
• 10:39
  a
• 10:39
  different
• 10:39
  story:
• 10:40
  tensor
• 10:41
  parallel
• 10:41
  actually
• 10:41
  slightly
• 10:42
  decreases
• 10:42
  generation
• 10:43
  speed
• 10:43
  because
• 10:43
  expert
• 10:44
  routing
• 10:44
  doesn't
• 10:44
  benefit
• 10:45
  from
• 10:45
  the
• 10:45
  same
• 10:45
  cross-card
• 10:46
  parallelism.
• 10:47
  vLLM
• 10:48
  still
• 10:48
  wins
• 10:48
  for
• 10:48
  throughput-oriented
• 10:49
  MoE
• 10:49
  serving,
• 10:50
  hitting
• 10:51
  187
• 10:51
  tok/s
• 10:52
  output
• 10:53
  with
• 10:53
  8-way
• 10:53
  concurrency.
• 10:55
  An
• 10:55
  even
• 10:55
  more
• 10:55
  radical
• 10:56
  approach
• 10:56
  decouples
• 10:57
  attention
• 10:57
  from
• 10:57
  weights
• 10:57
  entirely
• 10:58
  —
• 10:59
  splitting
• 10:59
  the
• 10:59
  attention
• 11:00
  layers
• 11:00
  (a
• 11:00
  couple
• 11:00
  of
• 11:00
  gigabytes)
• 11:01
  onto
• 11:01
  one
• 11:01
  machine
• 11:02
  and
• 11:02
  the
• 11:02
  weight
• 11:02
  matrices
• 11:03
  onto
• 11:03
  another,
• 11:04
  effectively
• 11:04
  bypassing
• 11:05
  the
• 11:05
  memory
• 11:05
  wall
• 11:05
  for
• 11:05
  local
• 11:06
  LLM
• 11:06
  Early
• 11:08
  experiments
• 11:08
  with
• 11:08
  Gemma
• 11:09
  4
• 11:09
  26B
• 11:10
  using
• 11:10
  this
• 11:10
  architecture
• 11:11
  show
• 11:11
  functional
• 11:11
  results,
• 11:12
  with
• 11:12
  a
• 11:12
  working
• 11:13
  implementation
• 11:13
  available
• 11:14
  in
• 11:14
  the
• 11:14
  larql
• 11:14
  repository.
• 11:18
  A
• 11:18
  developer
• 11:18
  has
• 11:18
  built
• 11:19
  a
• 11:19
  "Second
• 11:19
  Thoughts"
• 11:20
  system
• 11:20
  that
• 11:20
  attaches
• 11:21
  a
• 11:21
  small
• 11:21
  transformer
• 11:22
  sidecar
• 11:22
  to
• 11:22
  a
• 11:22
  language
• 11:23
  model,
• 11:23
  reading
• 11:24
  output
• 11:24
  near
• 11:24
  the
• 11:24
  end
• 11:25
  of
• 11:25
  generation
• 11:25
  and
• 11:25
  feeding
• 11:26
  refined
• 11:26
  representations
• 11:27
  back
• 11:27
  near
• 11:27
  the
• 11:27
  top
• 11:28
  as
• 11:28
  a
• 11:28
  refinement
• 11:28
  loop.
• 11:29
  Inspired
• 11:30
  by
• 11:30
  neuroscience
• 11:31
  findings
• 11:31
  about
• 11:31
  bidirectional
• 11:32
  processing
• 11:33
  in
• 11:33
  the
• 11:33
  brain
• 11:33
  (documented
• 11:34
  in
• 11:34
  the
• 11:34
  "Repeat
• 11:35
  Yourself"
• 11:35
  research
• 11:36
  on
• 11:36
  how
• 11:36
  humans
• 11:36
  re-read
• 11:37
  their
• 11:37
  own
• 11:37
  writing
• 11:37
  to
• 11:38
  improve
• 11:38
  it),
• 11:38
  the
• 11:38
  system
• 11:39
  dramatically
• 11:39
  improved
• 11:40
  a
• 11:40
  1.7B
• 11:41
  model's
• 11:41
  coding
• 11:41
  performance
• 11:42
  on
• 11:42
  the
• 11:42
  first
• 11:42
  twenty
• 11:42
  HumanEval
• 11:43
  The
• 11:44
  mini-LLM
• 11:45
  runs
• 11:45
  a
• 11:45
  standard
• 11:45
  forward
• 11:46
  pass
• 11:46
  on
• 11:46
  the
• 11:46
  deep
• 11:46
  representation,
• 11:48
  transforming
• 11:48
  it
• 11:48
  from
• 11:48
  "what
• 11:49
  the
• 11:49
  model
• 11:49
  nearly
• 11:49
  output"
• 11:50
  into
• 11:50
  "what
• 11:50
  the
• 11:51
  early
• 11:51
  layers
• 11:51
  should
• 11:51
  have
• 11:51
  built."
• 11:52
  The
• 11:52
  community
• 11:53
  is
• 11:53
  particularly
• 11:53
  interested
• 11:54
  in
• 11:54
  whether
• 11:54
  a
• 11:54
  second
• 11:54
  pass
• 11:55
  yields
• 11:55
  further
• 11:55
  improvement
• 11:56
  or
• 11:56
  whether
• 11:56
  —
• 11:57
  as
• 11:57
  one
• 11:57
  commenter
• 11:57
  puts
• 11:58
  it
• 11:58
  —
• 11:58
  "the
• 11:58
  model
• 11:59
  stops
• 11:59
  fixing
• 11:59
  real
• 12:00
  bugs
• 12:00
  and
• 12:00
  starts
• 12:00
  inventing
• 12:01
  new
• 12:01
  ones
• 12:01
  to
• 12:01
  justify
• 12:01
  another
• 12:02
  pass."
• 12:03
  Results
• 12:03
  on
• 12:03
  Qwen3.5
• 12:04
  9B
• 12:05
  are
• 12:05
  forthcoming.
• 12:06
  Sculpt
• 12:06
  takes
• 12:07
  a
• 12:07
  different
• 12:07
  approach
• 12:07
  to
• 12:07
  squeezing
• 12:08
  more
• 12:08
  from
• 12:08
  less:
• 12:09
  a
• 12:09
  plug-and-play
• 12:09
  open-source
• 12:10
  pruning
• 12:10
  tool
• 12:11
  that
• 12:11
  shapes
• 12:11
  models
• 12:11
  to
• 12:11
  specific
• 12:12
  workloads
• 12:12
  rather
• 12:12
  than
• 12:13
  optimizing
• 12:13
  for
• 12:13
  general
• 12:13
  benchmarks.
• 12:14
  Born
• 12:15
  from
• 12:15
  work
• 12:15
  on
• 12:15
  biologically
• 12:16
  inspired
• 12:16
  co-activation
• 12:17
  algorithms
• 12:18
  for
• 12:18
  expert
• 12:18
  placement
• 12:18
  on
• 12:18
  chips,
• 12:19
  Sculpt
• 12:19
  analyzes
• 12:20
  how
• 12:20
  a
• 12:20
  model
• 12:20
  responds
• 12:21
  to
• 12:21
  a
• 12:21
  given
• 12:21
  workload,
• 12:22
  identifies
• 12:22
  the
• 12:22
  parts
• 12:23
  that
• 12:23
  aren't
• 12:23
  necessary
• 12:23
  for
• 12:23
  that
• 12:24
  specific
• 12:24
  use
• 12:24
  case,
• 12:25
  and
• 12:25
  produces
• 12:25
  a
• 12:25
  standard
• 12:25
  Hugging
• 12:26
  Face
• 12:26
  checkpoint
• 12:26
  compatible
• 12:27
  with
• 12:27
  vLLM,
• 12:28
  llama.cpp,
• 12:29
  GGUF,
• 12:29
  and
• 12:29
  Ollama
• 12:30
  —
• 12:30
  no
• 12:30
  runtime
• 12:31
  changes
• 12:31
  required.
• 12:32
  The
• 12:32
  author
• 12:32
  sees
• 12:32
  the
• 12:32
  technique
• 12:33
  as
• 12:33
  especially
• 12:33
  relevant
• 12:34
  for
• 12:34
  robotics,
• 12:34
  sensors,
• 12:35
  and
• 12:35
  other
• 12:35
  local-first
• 12:36
  applications
• 12:37
  where
• 12:37
  "smaller,
• 12:37
  faster,
• 12:38
  less
• 12:38
  consumption
• 12:39
  is
• 12:39
  the
• 12:39
  future."
• 12:40
  For
• 12:40
  those
• 12:40
  who
• 12:41
  want
• 12:41
  to
• 12:41
  run
• 12:41
  multiple
• 12:41
  specialized
• 12:42
  models
• 12:42
  concurrently,
• 12:43
  a
• 12:43
  project
• 12:44
  demonstrates
• 12:44
  four
• 12:45
  LLM
• 12:45
  agents
• 12:45
  with
• 12:46
  per-agent
• 12:46
  LoRA
• 12:46
  adapters
• 12:47
  sharing
• 12:47
  a
• 12:48
  single
• 12:48
  RTX
• 12:48
  3070
• 12:49
  8GB
• 12:49
  —
• 12:51
  a
• 12:51
  striking
• 12:51
  demonstration
• 12:52
  of
• 12:52
  operational
• 12:52
  density
• 12:53
  on
• 12:53
  consumer
• 12:54
  hardware
• 12:54
  .
• 12:55
  And
• 12:55
  for
• 12:55
  anyone
• 12:55
  trying
• 12:56
  to
• 12:56
  understand
• 12:56
  what's
• 12:56
  actually
• 12:57
  inside
• 12:57
  these
• 12:57
  architectures,
• 12:58
  HF
• 12:59
  Viewer
• 12:59
  provides
• 13:00
  interactive
• 13:00
  visualizations
• 13:01
  of
• 13:01
  Hugging
• 13:01
  Face
• 13:01
  models
• 13:02
  —
• 13:02
  paste
• 13:03
  a
• 13:03
  URL
• 13:04
  and
• 13:04
  get
• 13:04
  a
• 13:04
  granularity-adjustable
• 13:05
  diagram
• 13:05
  that
• 13:05
  makes
• 13:06
  architectural
• 13:06
  differences
• 13:07
  between
• 13:07
  model
• 13:07
  families
• 13:08
  visible
• 13:08
  at
• 13:08
  a
• 13:08
  glance.
• 13:11
  Communications
• 13:12
  of
• 13:12
  the
• 13:12
  ACM
• 13:12
  surveys
• 13:13
  the
• 13:13
  road
• 13:13
  to
• 13:13
  billion-token
• 13:14
  context
• 13:14
  windows
• 13:14
  —
• 13:15
  a
• 13:15
  goal
• 13:16
  that
• 13:16
  Nvidia's
• 13:16
  Rubin
• 13:16
  CPX
• 13:17
  architecture
• 13:17
  is
• 13:18
  explicitly
• 13:18
  designed
• 13:19
  to
• 13:19
  enable
• 13:19
  by
• 13:19
  2030.
• 13:20
  The
• 13:21
  article
• 13:21
  surfaces
• 13:21
  the
• 13:21
  gap
• 13:22
  between
• 13:22
  advertised
• 13:23
  and
• 13:23
  effective
• 13:23
  context:
• 13:24
  "The
• 13:24
  advertised
• 13:25
  context
• 13:25
  window
• 13:25
  is
• 13:25
  largely
• 13:26
  a
• 13:26
  memory-bound
• 13:26
  definition,
• 13:27
  not
• 13:27
  a
• 13:27
  quality-bound
• 13:28
  one,"
• 13:28
  says
• 13:29
  Ayush
• 13:29
  Goyal
• 13:29
  of
• 13:29
  Veza.
• 13:30
  As
• 13:30
  context
• 13:31
  grows,
• 13:32
  the
• 13:32
  KV
• 13:32
  cache
• 13:32
  overwhelms
• 13:33
  memory
• 13:33
  bandwidth,
• 13:34
  forcing
• 13:35
  serving
• 13:35
  stacks
• 13:35
  to
• 13:35
  page
• 13:36
  state
• 13:36
  to
• 13:36
  cheaper
• 13:36
  tiers
• 13:37
  like
• 13:37
  CPU
• 13:37
  or
• 13:37
  NVMe,
• 13:38
  adding
• 13:39
  latency
• 13:39
  variance
• 13:39
  and
• 13:40
  quality
• 13:40
  hits.
• 13:41
  The
• 13:41
  "attention
• 13:41
  dilution
• 13:42
  problem"
• 13:42
  —
• 13:42
  spreading
• 13:43
  fixed
• 13:43
  attention
• 13:44
  over
• 13:44
  ever
• 13:44
  more
• 13:44
  tokens
• 13:44
  —
• 13:45
  degrades
• 13:46
  signal-to-noise
• 13:47
  until
• 13:47
  vast
• 13:47
  stretches
• 13:48
  of
• 13:48
  context
• 13:48
  become
• 13:49
  what
• 13:49
  Bob
• 13:49
  Gourley
• 13:49
  of
• 13:50
  OODA
• 13:50
  calls
• 13:50
  "context
• 13:51
  rot."
• 13:52
  Rubin
• 13:52
  CPX
• 13:53
  attacks
• 13:53
  this
• 13:53
  with
• 13:53
  an
• 13:53
  inference-first
• 13:54
  design:
• 13:55
  disaggregated
• 13:55
  GDDR7
• 13:58
  memory
• 13:58
  paths
• 13:59
  optimized
• 13:59
  for
• 13:59
  context
• 14:00
  ingestion,
• 14:00
  separated
• 14:01
  from
• 14:01
  token
• 14:01
  generation.
• 14:03
  But
• 14:03
  experts
• 14:03
  note
• 14:03
  the
• 14:03
  billion-token
• 14:04
  target
• 14:04
  will
• 14:04
  likely
• 14:05
  require
• 14:05
  algorithmic
• 14:06
  breakthroughs
• 14:06
  —
• 14:06
  State
• 14:07
  Space
• 14:07
  Models,
• 14:07
  Test-Time
• 14:08
  Training,
• 14:08
  or
• 14:09
  Recursive
• 14:09
  Language
• 14:09
  Models
• 14:09
  —
• 14:10
  alongside
• 14:10
  hardware.
• 14:12
  The
• 14:12
  realistic
• 14:12
  future,
• 14:13
  as
• 14:13
  Goyal
• 14:13
  frames
• 14:13
  it,
• 14:13
  is
• 14:14
  "not
• 14:14
  a
• 14:14
  single
• 14:14
  flat
• 14:14
  attention
• 14:15
  window"
• 14:15
  but
• 14:15
  hierarchical
• 14:16
  attention
• 14:16
  combined
• 14:17
  with
• 14:17
  retrieval
• 14:17
  and
• 14:17
  compression.
• 14:18
  OpenAI's
• 14:19
  engineering
• 14:19
  team
• 14:20
  published
• 14:20
  the
• 14:20
  architecture
• 14:20
  behind
• 14:21
  their
• 14:21
  low-latency
• 14:21
  voice
• 14:22
  AI
• 14:22
  infrastructure,
• 14:23
  serving
• 14:23
  over
• 14:23
  900
• 14:24
  million
• 14:24
  weekly
• 14:24
  active
• 14:24
  users.
• 14:25
  The
• 14:25
  core
• 14:26
  insight:
• 14:27
  standard
• 14:27
  WebRTC's
• 14:28
  one-port-per-session
• 14:28
  model
• 14:28
  doesn't
• 14:29
  fit
• 14:29
  Kubernetes
• 14:29
  at
• 14:30
  Their
• 14:31
  solution
• 14:31
  splits
• 14:31
  the
• 14:31
  system
• 14:32
  into
• 14:32
  a
• 14:32
  stateless
• 14:32
  relay
• 14:33
  —
• 14:33
  lightweight
• 14:34
  UDP
• 14:34
  forwarding
• 14:35
  with
• 14:35
  a
• 14:35
  small
• 14:35
  public
• 14:35
  footprint
• 14:36
  —
• 14:36
  and
• 14:36
  a
• 14:36
  stateful
• 14:37
  transceiver
• 14:37
  that
• 14:37
  owns
• 14:38
  all
• 14:38
  WebRTC
• 14:38
  session
• 14:39
  state
• 14:39
  (ICE,
• 14:39
  DTLS,
• 14:40
  SRTP).
• 14:41
  The
• 14:41
  relay
• 14:42
  reads
• 14:42
  just
• 14:42
  enough
• 14:43
  packet
• 14:43
  metadata
• 14:43
  —
• 14:44
  specifically
• 14:44
  the
• 14:45
  ICE
• 14:45
  username
• 14:45
  fragment,
• 14:46
  encoded
• 14:46
  with
• 14:46
  routing
• 14:47
  hints
• 14:47
  during
• 14:47
  signaling
• 14:47
  —
• 14:48
  to
• 14:48
  forward
• 14:49
  first
• 14:49
  packets
• 14:49
  to
• 14:49
  the
• 14:49
  correct
• 14:50
  transceiver
• 14:50
  without
• 14:50
  any
• 14:51
  external
• 14:51
  lookup.
• 14:52
  The
• 14:52
  implementation
• 14:53
  uses
• 14:53
  Go
• 14:53
  with
• 14:54
  SO
• 14:54
  REUSEPORT
• 14:57
  for
• 14:57
  kernel-level
• 14:58
  packet
• 14:58
  distribution
• 14:58
  across
• 14:59
  workers,
• 14:59
  LockOSThread
• 15:00
  for
• 15:00
  CPU
• 15:00
  pinning,
• 15:01
  and
• 15:01
  pre-allocated
• 15:02
  buffers
• 15:02
  to
• 15:02
  minimize
• 15:03
  GC
• 15:03
  pressure.
• 15:04
  If
• 15:04
  a
• 15:04
  relay
• 15:05
  restarts,
• 15:06
  the
• 15:06
  next
• 15:06
  STUN
• 15:06
  packet
• 15:07
  rebuilds
• 15:07
  the
• 15:07
  route
• 15:07
  from
• 15:07
  the
• 15:07
  ufrag
• 15:08
  hint.
• 15:09
  Combined
• 15:09
  with
• 15:09
  Cloudflare
• 15:10
  geo-steering,
• 15:11
  Global
• 15:11
  Relay
• 15:11
  shortens
• 15:12
  the
• 15:12
  first
• 15:12
  client-to-OpenAI
• 15:13
  hop
• 15:13
  by
• 15:14
  entering
• 15:14
  traffic
• 15:14
  at
• 15:14
  a
• 15:14
  nearby
• 15:15
  edge
• 15:15
  point.
• 15:16
  The
• 15:16
  broader
• 15:17
  lesson:
• 15:18
  the
• 15:18
  best
• 15:18
  place
• 15:18
  to
• 15:18
  add
• 15:19
  complexity
• 15:19
  is
• 15:19
  in
• 15:20
  a
• 15:20
  thin
• 15:20
  routing
• 15:20
  layer,
• 15:21
  not
• 15:21
  in
• 15:21
  every
• 15:21
  backend
• 15:22
  service
• 15:22
  or
• 15:22
  in
• 15:23
  custom
• 15:23
  client
• 15:23
  behavior.
• 15:26
  MIT
• 15:26
  engineers
• 15:27
  have
• 15:27
  built
• 15:27
  a
• 15:27
  computational
• 15:28
  violin
• 15:29
  that
• 15:29
  produces
• 15:29
  realistic
• 15:30
  sound
• 15:30
  from
• 15:30
  first
• 15:31
  principles
• 15:31
  —
• 15:32
  no
• 15:32
  samples,
• 15:33
  no
• 15:33
  averaging
• 15:33
  over
• 15:34
  thousands
• 15:34
  of
• 15:34
  recorded
• 15:34
  notes.
• 15:35
  The
• 15:36
  team
• 15:36
  imported
• 15:36
  CT
• 15:37
  scans
• 15:37
  of
• 15:37
  a
• 15:37
  1715
• 15:38
  Stradivarius
• 15:39
  into
• 15:39
  a
• 15:39
  solid
• 15:40
  modeling
• 15:40
  program,
• 15:41
  divided
• 15:41
  the
• 15:42
  instrument
• 15:42
  and
• 15:42
  surrounding
• 15:42
  air
• 15:43
  into
• 15:43
  millions
• 15:43
  of
• 15:44
  finite
• 15:44
  elements,
• 15:45
  and
• 15:45
  applied
• 15:45
  physics-based
• 15:46
  equations
• 15:46
  of
• 15:46
  stress,
• 15:47
  motion,
• 15:47
  and
• 15:47
  acoustic
• 15:48
  wave
• 15:48
  propagation
• 15:48
  to
• 15:49
  simulate
• 15:49
  how
• 15:49
  each
• 15:49
  material
• 15:50
  element
• 15:50
  interacts
• 15:51
  with
• 15:51
  every
• 15:51
  other.
• 15:52
  The
• 15:52
  result:
• 15:53
  plucked-string
• 15:53
  (pizzicato)
• 15:54
  renditions
• 15:54
  of
• 15:54
  Bach's
• 15:55
  Fugue
• 15:55
  in
• 15:55
  G
• 15:55
  Minor
• 15:55
  and
• 15:55
  "Daisy
• 15:56
  Bell"
• 15:56
  —
• 15:57
  a
• 15:57
  nod
• 15:57
  to
• 15:57
  the
• 15:57
  first
• 15:57
  song
• 15:57
  ever
• 15:58
  produced
• 15:58
  by
• 15:58
  computer-synthesized
• 15:59
  voice
• 15:59
  —
• 16:00
  that
• 16:00
  emerge
• 16:00
  entirely
• 16:01
  from
• 16:01
  the
• 16:01
  physics
• 16:01
  of
• 16:01
  vibrating
• 16:02
  strings,
• 16:02
  wood
• 16:03
  resonance,
• 16:03
  and
• 16:03
  air
• 16:03
  coupling.
• 16:04
  The
• 16:04
  practical
• 16:05
  application
• 16:05
  is
• 16:05
  for
• 16:05
  luthiers:
• 16:06
  tweak
• 16:07
  the
• 16:07
  back
• 16:07
  plate
• 16:07
  thickness
• 16:08
  or
• 16:08
  change
• 16:08
  the
• 16:08
  wood
• 16:08
  type
• 16:08
  and
• 16:09
  hear
• 16:09
  the
• 16:09
  difference
• 16:09
  before
• 16:09
  carving
• 16:10
  a
• 16:10
  single
• 16:10
  part.
• 16:11
  "We're
• 16:11
  not
• 16:11
  saying
• 16:11
  that
• 16:12
  we
• 16:12
  can
• 16:12
  reproduce
• 16:12
  the
• 16:12
  artisan's
• 16:13
  magic,"
• 16:13
  says
• 16:13
  Professor
• 16:14
  Nicholas
• 16:14
  Makris.
• 16:15
  "We're
• 16:16
  just
• 16:16
  trying
• 16:16
  to
• 16:16
  understand
• 16:16
  the
• 16:16
  physics
• 16:17
  of
• 16:17
  violin
• 16:17
  sound."
• 16:18
  The
• 16:18
  model
• 16:19
  currently
• 16:19
  handles
• 16:19
  pizzicato
• 16:20
  only;
• 16:21
  bowing
• 16:21
  involves
• 16:21
  far
• 16:21
  more
• 16:22
  complex
• 16:22
  string-bow
• 16:23
  friction
• 16:23
  dynamics,
• 16:24
  but
• 16:24
  the
• 16:24
  team
• 16:24
  says
• 16:24
  this
• 16:25
  physics-based
• 16:25
  foundation
• 16:26
  could
• 16:26
  eventually
• 16:26
  be
• 16:26
  paired
• 16:26
  with
• 16:27
  a
• 16:27
  bowing
• 16:27
  model
• 16:27
  for
• 16:27
  full
• 16:27
  violin
• 16:28
  simulation.
• 16:29
  On
• 16:29
  the
• 16:29
  neural
• 16:29
  side
• 16:30
  of
• 16:30
  creative
• 16:30
  generation,
• 16:31
  Lightricks
• 16:31
  has
• 16:31
  released
• 16:31
  LTX-2.3-22b-IC-LoRA-HDR,
• 16:34
  a
• 16:34
  22-billion-parameter
• 16:35
  video
• 16:36
  generation
• 16:36
  model
• 16:36
  fine-tuned
• 16:37
  with
• 16:37
  LoRA
• 16:37
  for
• 16:37
  HDR
• 16:38
  aesthetic
• 16:38
  output
• 16:38
  —
• 16:39
  continuing
• 16:39
  the
• 16:40
  trend
• 16:40
  of
• 16:40
  parameter-efficient
• 16:40
  adaptation
• 16:41
  producing
• 16:41
  specialized
• 16:42
  creative
• 16:42
  tools
• 16:42
  from
• 16:42
  general-purpose
• 16:43
  foundations.