Latest / AGI Dreams – Open, Uncensored, & Local - AI News Digest / AGI Dreams Podcast – April 09, 2026

Transcript

AGI Dreams Podcast – April 09, 2026 · AGI Dreams – Open, Uncensored, & Local - AI News Digest

• 0:00
  The
• 0:00
  abliteration
• 0:00
  community
• 0:01
  just
• 0:01
  discovered
• 0:01
  something
• 0:02
  that
• 0:02
  should
• 0:02
  make
• 0:02
  safety
• 0:03
  researchers
• 0:03
  sit
• 0:03
  up:
• 0:04
  Mixture-of-Experts
• 0:05
  models
• 0:05
  don't
• 0:05
  encode
• 0:06
  refusal
• 0:06
  the
• 0:06
  way
• 0:06
  dense
• 0:06
  models
• 0:07
  do,
• 0:08
  and
• 0:08
  the
• 0:08
  standard
• 0:08
  techniques
• 0:09
  for
• 0:09
  removing
• 0:09
  it
• 0:09
  only
• 0:09
  half-work.
• 0:10
  A
• 0:11
  researcher
• 0:11
  adapted
• 0:11
  FailSpy's
• 0:12
  abliteration
• 0:13
  technique
• 0:13
  to
• 0:13
  Qwen3.5-397B-A17B
• 0:18
  running
• 0:19
  at
• 0:19
  4-bit
• 0:19
  on
• 0:19
  a
• 0:19
  Mac
• 0:19
  Studio
• 0:20
  M3
• 0:20
  Ultra
• 0:20
  (512GB),
• 0:22
  targeting
• 0:22
  PRC-specific
• 0:23
  censorship
• 0:24
  while
• 0:24
  attempting
• 0:24
  to
• 0:24
  preserve
• 0:25
  Western
• 0:25
  safety
• 0:25
  refusals.
• 0:27
  The
• 0:27
  results
• 0:27
  revealed
• 0:27
  two
• 0:28
  separable
• 0:28
  refusal
• 0:29
  subspaces
• 0:29
  —
• 0:30
  Chinese-political
• 0:31
  and
• 0:31
  Western-safety
• 0:31
  refusals
• 0:32
  live
• 0:32
  in
• 0:32
  different
• 0:32
  directions
• 0:33
  in
• 0:33
  activation
• 0:33
  space,
• 0:34
  and
• 0:34
  you
• 0:34
  can
• 0:34
  surgically
• 0:35
  remove
• 0:35
  one
• 0:35
  without
• 0:36
  touching
• 0:36
  the
• 0:36
  other.
• 0:37
  The
• 0:37
  more
• 0:37
  consequential
• 0:37
  finding
• 0:38
  is
• 0:38
  architectural.
• 0:39
  On
• 0:39
  dense
• 0:40
  models,
• 0:40
  orthogonalizing
• 0:41
  output
• 0:41
  projections
• 0:42
  (o
• 0:42
  proj,
• 0:42
  down
• 0:42
  proj)
• 0:43
  is
• 0:43
  functionally
• 0:43
  equivalent
• 0:44
  to
• 0:44
  projecting
• 0:44
  the
• 0:44
  refusal
• 0:45
  direction
• 0:45
  out
• 0:45
  of
• 0:45
  the
• 0:45
  residual
• 0:46
  stream
• 0:46
  at
• 0:46
  inference
• 0:46
  time.
• 0:47
  On
• 0:47
  MoE
• 0:48
  models,
• 0:48
  weight-baking
• 0:49
  removes
• 0:49
  political
• 0:49
  refusals
• 0:50
  but
• 0:50
  not
• 0:50
  safety
• 0:50
  refusals.
• 0:52
  The
• 0:52
  inference-time
• 0:52
  hook
• 0:52
  removes
• 0:53
  both.
• 0:54
  The
• 0:54
  hypothesis:
• 0:55
  safety
• 0:55
  refusals
• 0:56
  route
• 0:56
  through
• 0:56
  specialized
• 0:57
  "safety
• 0:58
  experts"
• 0:59
  via
• 0:59
  the
• 0:59
  MoE
• 0:59
  router,
• 1:00
  and
• 1:00
  the
• 1:00
  routing
• 1:00
  decision
• 1:01
  happens
• 1:01
  before
• 1:01
  the
• 1:01
  output
• 1:01
  projection,
• 1:02
  so
• 1:03
  orthogonalizing
• 1:03
  down
• 1:03
  proj
• 1:03
  doesn't
• 1:04
  catch
• 1:04
  it.
• 1:05
  This
• 1:05
  is
• 1:05
  a
• 1:05
  genuinely
• 1:06
  new
• 1:06
  observation
• 1:06
  —
• 1:07
  it
• 1:07
  means
• 1:07
  MoE
• 1:08
  safety
• 1:08
  alignment
• 1:08
  is
• 1:08
  structurally
• 1:09
  more
• 1:09
  resilient
• 1:09
  to
• 1:10
  weight-level
• 1:10
  tampering
• 1:10
  than
• 1:10
  dense
• 1:11
  model
• 1:11
  alignment,
• 1:12
  but
• 1:12
  only
• 1:12
  accidentally.
• 1:14
  The
• 1:14
  397B
• 1:15
  model
• 1:15
  proved
• 1:15
  fragile:
• 1:16
  exactly
• 1:17
  one
• 1:17
  working
• 1:17
  setting
• 1:17
  (top-16
• 1:18
  directions),
• 1:19
  with
• 1:19
  top-18
• 1:20
  causing
• 1:20
  stuck
• 1:20
  repetition
• 1:21
  loops.
• 1:22
  A
• 1:22
  separate
• 1:22
  effort
• 1:23
  abliterating
• 1:23
  India's
• 1:23
  Sarvam
• 1:24
  30B
• 1:24
  and
• 1:25
  105B
• 1:25
  —
• 1:25
  reasoning-capable
• 1:26
  MoE
• 1:26
  models
• 1:26
  —
• 1:27
  uncovered
• 1:28
  another
• 1:28
  wrinkle:
• 1:29
  reasoning
• 1:29
  models
• 1:30
  have
• 1:30
  two
• 1:30
  refusal
• 1:30
  circuits,
• 1:31
  not
• 1:31
  one.
• 1:32
  The
• 1:33
  block
• 1:33
  and
• 1:33
  the
• 1:33
  final
• 1:34
  answer
• 1:34
  can
• 1:34
  disagree,
• 1:35
  with
• 1:35
  the
• 1:35
  model
• 1:35
  reasoning
• 1:36
  toward
• 1:36
  compliance
• 1:36
  in
• 1:36
  its
• 1:36
  chain-of-thought
• 1:37
  and
• 1:38
  then
• 1:38
  refusing
• 1:38
  anyway
• 1:39
  in
• 1:39
  the
• 1:39
  response.
• 1:40
  Perhaps
• 1:40
  most
• 1:41
  striking,
• 1:42
  a
• 1:42
  single
• 1:42
  English-computed
• 1:43
  refusal
• 1:43
  direction
• 1:44
  removed
• 1:44
  refusal
• 1:44
  across
• 1:45
  Malayalam,
• 1:45
  Hindi,
• 1:46
  and
• 1:46
  Kannada,
• 1:47
  suggesting
• 1:47
  refusal
• 1:48
  is
• 1:48
  pre-linguistic
• 1:48
  —
• 1:49
  encoded
• 1:50
  in
• 1:50
  the
• 1:50
  model's
• 1:50
  internal
• 1:51
  representation
• 1:51
  before
• 1:52
  language-specific
• 1:52
  layers.
• 1:57
  A
• 1:57
  new
• 1:57
  arxiv
• 1:57
  paper
• 1:58
  proposes
• 1:58
  a
• 1:58
  fundamentally
• 1:59
  different
• 1:59
  approach
• 2:00
  to
• 2:00
  LLM
• 2:00
  safety
• 2:01
  monitoring:
• 2:01
  instead
• 2:02
  of
• 2:02
  wrapping
• 2:02
  models
• 2:03
  with
• 2:03
  external
• 2:03
  guard
• 2:03
  classifiers
• 2:04
  that
• 2:04
  adversaries
• 2:05
  can
• 2:05
  study
• 2:05
  and
• 2:05
  evade,
• 2:06
  embed
• 2:06
  a
• 2:06
  secret
• 2:06
  keyed
• 2:07
  watermark
• 2:07
  directly
• 2:08
  into
• 2:08
  the
• 2:08
  model's
• 2:08
  internal
• 2:09
  activations.
• 2:10
  The
• 2:10
  method
• 2:10
  fine-tunes
• 2:10
  the
• 2:11
  LLM
• 2:11
  so
• 2:11
  that
• 2:11
  whenever
• 2:12
  it
• 2:12
  produces
• 2:12
  policy-violating
• 2:13
  output,
• 2:14
  its
• 2:14
  hidden
• 2:14
  states
• 2:14
  align
• 2:15
  with
• 2:15
  a
• 2:15
  secret
• 2:15
  direction
• 2:15
  that
• 2:16
  only
• 2:16
  the
• 2:16
  provider
• 2:16
  knows.
• 2:17
  Detection
• 2:18
  requires
• 2:18
  a
• 2:18
  single
• 2:18
  cosine
• 2:19
  similarity
• 2:19
  check
• 2:20
  per
• 2:20
  token
• 2:20
  —
• 2:21
  negligible
• 2:21
  overhead
• 2:21
  compared
• 2:22
  to
• 2:22
  running
• 2:22
  a
• 2:22
  separate
• 2:22
  guard
• 2:23
  model.
• 2:23
  Against
• 2:24
  adaptive
• 2:24
  attackers
• 2:25
  who
• 2:25
  know
• 2:25
  the
• 2:25
  monitoring
• 2:25
  algorithm
• 2:26
  but
• 2:26
  not
• 2:26
  the
• 2:26
  secret
• 2:27
  key,
• 2:27
  activation
• 2:28
  watermarking
• 2:28
  outperforms
• 2:29
  Llama
• 2:29
  Guard
• 2:29
  and
• 2:30
  Qwen
• 2:30
  Guard
• 2:30
  by
• 2:30
  up
• 2:31
  to
• 2:31
  29%
• 2:32
  AUROC.
• 2:33
  Guard
• 2:33
  models
• 2:33
  collapse
• 2:33
  under
• 2:34
  sophisticated
• 2:34
  attacks
• 2:35
  (AutoDAN
• 2:35
  pushes
• 2:35
  their
• 2:35
  ASR
• 2:36
  above
• 2:36
  68%),
• 2:37
  while
• 2:37
  the
• 2:37
  watermark's
• 2:38
  keyed
• 2:38
  randomization
• 2:39
  makes
• 2:39
  transfer
• 2:39
  attacks
• 2:40
  unreliable.
• 2:40
  The
• 2:40
  paper
• 2:41
  formalizes
• 2:41
  this
• 2:41
  as
• 2:41
  a
• 2:42
  security
• 2:42
  game
• 2:42
  and
• 2:42
  demonstrates
• 2:43
  multi-entity
• 2:43
  monitoring:
• 2:44
  the
• 2:44
  provider
• 2:45
  can
• 2:45
  track
• 2:45
  which
• 2:45
  specific
• 2:46
  piece
• 2:46
  of
• 2:46
  sensitive
• 2:46
  knowledge
• 2:46
  was
• 2:47
  accessed,
• 2:47
  with
• 2:47
  95%+
• 2:48
  attribution
• 2:49
  accuracy
• 2:49
  and
• 2:49
  fewer
• 2:50
  than
• 2:50
  one
• 2:50
  false
• 2:50
  alarm
• 2:50
  per
• 2:50
  1,000
• 2:51
  benign
• 2:51
  requests.
• 2:52
  On
• 2:52
  the
• 2:52
  infrastructure
• 2:53
  side,
• 2:53
  iron-proxy
• 2:54
  tackles
• 2:54
  a
• 2:54
  related
• 2:54
  problem
• 2:55
  from
• 2:55
  the
• 2:55
  opposite
• 2:55
  direction:
• 2:56
  what
• 2:56
  happens
• 2:56
  when
• 2:56
  the
• 2:57
  AI
• 2:57
  agent
• 2:57
  itself
• 2:57
  is
• 2:58
  the
• 2:58
  untrusted
• 2:58
  workload?
• 2:59
  Built
• 2:59
  in
• 2:59
  Go,
• 3:00
  it's
• 3:00
  a
• 3:00
  MITM
• 3:01
  egress
• 3:01
  proxy
• 3:01
  with
• 3:01
  a
• 3:01
  built-in
• 3:02
  DNS
• 3:02
  server
• 3:02
  that
• 3:02
  enforces
• 3:03
  default-deny
• 3:03
  at
• 3:04
  the
• 3:04
  network
• 3:04
  boundary.
• 3:05
  Sandboxed
• 3:05
  workloads
• 3:06
  —
• 3:06
  CI
• 3:06
  pipelines,
• 3:07
  AI
• 3:07
  coding
• 3:07
  agents,
• 3:08
  containers
• 3:08
  running
• 3:09
  code
• 3:09
  you
• 3:09
  don't
• 3:09
  fully
• 3:09
  trust
• 3:09
  —
• 3:10
  can
• 3:10
  only
• 3:10
  reach
• 3:11
  domains
• 3:11
  you
• 3:11
  explicitly
• 3:12
  allowlist.
• 3:12
  The
• 3:13
  clever
• 3:13
  part
• 3:13
  is
• 3:13
  secret
• 3:13
  injection:
• 3:14
  real
• 3:14
  API
• 3:15
  keys
• 3:15
  never
• 3:15
  enter
• 3:15
  the
• 3:16
  sandbox.
• 3:16
  Workloads
• 3:17
  use
• 3:17
  proxy
• 3:17
  tokens,
• 3:18
  and
• 3:18
  iron-proxy
• 3:19
  swaps
• 3:19
  in
• 3:19
  real
• 3:19
  credentials
• 3:20
  at
• 3:20
  egress.
• 3:20
  If
• 3:20
  the
• 3:20
  sandbox
• 3:21
  is
• 3:21
  compromised,
• 3:21
  the
• 3:22
  attacker
• 3:22
  gets
• 3:22
  tokens
• 3:22
  that
• 3:23
  are
• 3:23
  worthless
• 3:23
  outside
• 3:23
  the
• 3:23
  proxy.
• 3:24
  Every
• 3:25
  request
• 3:25
  produces
• 3:25
  a
• 3:25
  structured
• 3:26
  JSON
• 3:26
  audit
• 3:26
  trail
• 3:26
  with
• 3:26
  full
• 3:27
  transform
• 3:27
  pipeline
• 3:27
  results,
• 3:28
  and
• 3:28
  the
• 3:28
  whole
• 3:28
  thing
• 3:28
  ships
• 3:29
  as
• 3:29
  a
• 3:29
  single
• 3:29
  binary
• 3:30
  with
• 3:30
  a
• 3:30
  single
• 3:30
  YAML
• 3:30
  config.
• 3:31
  For
• 3:31
  enforcement,
• 3:32
  it
• 3:32
  offers
• 3:32
  three
• 3:32
  tiers:
• 3:33
  DNS-only
• 3:34
  (easy
• 3:34
  to
• 3:34
  bypass),
• 3:35
  DNS
• 3:35
  +
• 3:35
  nftables
• 3:36
  (blocks
• 3:36
  hardcoded
• 3:37
  IPs),
• 3:37
  and
• 3:37
  TPROXY
• 3:38
  (kernel-level
• 3:39
  interception,
• 3:40
  catches
• 3:40
  everything).
• 3:40
  The
• 3:41
  trust
• 3:41
  question
• 3:41
  extends
• 3:41
  to
• 3:41
  what
• 3:42
  agents
• 3:42
  can
• 3:42
  actually
• 3:42
  read.
• 3:43
  The
• 3:43
  Agent
• 3:43
  Reading
• 3:43
  Test
• 3:44
  is
• 3:44
  a
• 3:44
  benchmark
• 3:44
  that
• 3:44
  doesn't
• 3:44
  test
• 3:45
  whether
• 3:45
  AI
• 3:45
  coding
• 3:45
  agents
• 3:46
  can
• 3:46
  reason
• 3:46
  —
• 3:46
  it
• 3:46
  tests
• 3:47
  whether
• 3:47
  they
• 3:47
  can
• 3:47
  reliably
• 3:47
  consume
• 3:48
  web
• 3:48
  documentation.
• 3:49
  Ten
• 3:50
  pages,
• 3:50
  each
• 3:50
  designed
• 3:51
  around
• 3:51
  a
• 3:51
  specific
• 3:51
  failure
• 3:52
  mode:
• 3:52
  150K-character
• 3:53
  pages
• 3:54
  with
• 3:54
  canary
• 3:54
  tokens
• 3:55
  at
• 3:55
  strategic
• 3:55
  positions
• 3:56
  to
• 3:56
  map
• 3:56
  truncation
• 3:56
  limits,
• 3:57
  80K
• 3:57
  of
• 3:57
  inline
• 3:58
  CSS
• 3:58
  before
• 3:58
  real
• 3:59
  content,
• 3:59
  client-side
• 4:00
  rendered
• 4:00
  pages
• 4:00
  that
• 4:01
  return
• 4:01
  empty
• 4:01
  shells,
• 4:02
  tabbed
• 4:02
  content
• 4:02
  where
• 4:02
  only
• 4:03
  the
• 4:03
  first
• 4:03
  variant
• 4:03
  is
• 4:03
  visible,
• 4:04
  HTTP
• 4:05
  200
• 4:05
  pages
• 4:05
  with
• 4:06
  "not
• 4:06
  found"
• 4:06
  messages,
• 4:07
  and
• 4:07
  cross-hostname
• 4:07
  redirects.
• 4:09
  A
• 4:09
  perfect
• 4:09
  score
• 4:09
  of
• 4:09
  20
• 4:10
  is
• 4:10
  unlikely
• 4:10
  for
• 4:11
  any
• 4:11
  current
• 4:11
  agent;
• 4:12
  the
• 4:12
  expected
• 4:13
  range
• 4:13
  is
• 4:13
  14-18.
• 4:15
  The
• 4:15
  uncomfortable
• 4:15
  implication:
• 4:17
  agents
• 4:17
  that
• 4:17
  confidently
• 4:18
  cite
• 4:18
  documentation
• 4:19
  may
• 4:19
  be
• 4:19
  working
• 4:19
  from
• 4:19
  truncated,
• 4:20
  garbled,
• 4:21
  or
• 4:21
  incomplete
• 4:21
  versions
• 4:22
  of
• 4:22
  it.
• 4:23
  These
• 4:23
  infrastructure
• 4:24
  and
• 4:24
  trust
• 4:24
  concerns
• 4:25
  crystallize
• 4:25
  in
• 4:25
  a
• 4:25
  practical
• 4:26
  post
• 4:26
  cataloging
• 4:26
  five
• 4:27
  patterns
• 4:27
  that
• 4:27
  consistently
• 4:28
  appear
• 4:28
  in
• 4:28
  AI
• 4:29
  systems
• 4:29
  that
• 4:29
  work
• 4:29
  in
• 4:29
  development
• 4:30
  but
• 4:30
  fail
• 4:31
  in
• 4:31
  production:
• 4:32
  no
• 4:32
  evaluation
• 4:32
  framework
• 4:33
  (iterating
• 4:33
  by
• 4:33
  feel),
• 4:34
  no
• 4:35
  confidence
• 4:35
  thresholding,
• 4:36
  prompts
• 4:36
  optimized
• 4:37
  on
• 4:37
  demo
• 4:37
  data,
• 4:38
  retrieval
• 4:38
  quality
• 4:39
  buried
• 4:39
  in
• 4:39
  end-to-end
• 4:40
  metrics,
• 4:41
  and
• 4:41
  integration
• 4:41
  layer
• 4:42
  underscoping
• 4:42
  —
• 4:43
  the
• 4:43
  async
• 4:43
  handling,
• 4:44
  graceful
• 4:45
  degradation,
• 4:46
  and
• 4:46
  output
• 4:46
  validation
• 4:47
  that
• 4:47
  typically
• 4:47
  runs
• 4:48
  40-60%
• 4:50
  of
• 4:50
  total
• 4:51
  production
• 4:51
  effort
• 4:52
  but
• 4:52
  never
• 4:52
  shows
• 4:52
  up
• 4:52
  in
• 4:52
  demos.
• 4:53
  The
• 4:53
  litmus
• 4:54
  test:
• 4:54
  "Can
• 4:55
  you
• 4:55
  show
• 4:55
  me
• 4:55
  what
• 4:55
  the
• 4:55
  user
• 4:55
  sees
• 4:56
  when
• 4:56
  the
• 4:56
  AI
• 4:56
  call
• 4:57
  fails?"
• 4:57
  Teams
• 4:58
  who've
• 4:58
  built
• 4:58
  for
• 4:58
  production
• 4:59
  answer
• 4:59
  immediately.
• 5:03
  The
• 5:03
  ecosystem
• 5:04
  around
• 5:04
  Gemma
• 5:05
  4
• 5:05
  is
• 5:05
  already
• 5:06
  producing
• 5:06
  concrete
• 5:07
  tools.
• 5:08
  Parlor
• 5:08
  is
• 5:09
  perhaps
• 5:09
  the
• 5:09
  most
• 5:09
  striking:
• 5:10
  a
• 5:10
  fully
• 5:10
  on-device,
• 5:11
  real-time
• 5:12
  multimodal
• 5:12
  AI
• 5:13
  application
• 5:13
  that
• 5:13
  takes
• 5:14
  audio
• 5:14
  and
• 5:14
  video
• 5:14
  input
• 5:15
  and
• 5:15
  talks
• 5:15
  back,
• 5:16
  running
• 5:16
  entirely
• 5:17
  on
• 5:17
  an
• 5:17
  M3
• 5:17
  Pro
• 5:17
  laptop.
• 5:18
  It
• 5:19
  pairs
• 5:19
  Gemma
• 5:19
  4
• 5:20
  E2B
• 5:20
  for
• 5:20
  understanding
• 5:21
  speech
• 5:21
  and
• 5:21
  vision
• 5:22
  with
• 5:22
  Kokoro
• 5:22
  for
• 5:22
  text-to-speech,
• 5:24
  achieving
• 5:24
  end-to-end
• 5:25
  latency
• 5:25
  of
• 5:25
  2.5-3.0
• 5:27
  seconds
• 5:27
  including
• 5:27
  speech
• 5:28
  recognition,
• 5:29
  response
• 5:29
  generation
• 5:30
  (~83
• 5:30
  tokens/sec
• 5:31
  on
• 5:31
  GPU),
• 5:32
  and
• 5:32
  TTS.
• 5:33
  The
• 5:33
  creator's
• 5:34
  motivation
• 5:34
  is
• 5:34
  practical
• 5:35
  —
• 5:35
  self-hosting
• 5:36
  a
• 5:36
  free
• 5:36
  voice
• 5:37
  AI
• 5:37
  on
• 5:37
  a
• 5:37
  home
• 5:37
  server
• 5:38
  to
• 5:38
  help
• 5:38
  people
• 5:38
  learn
• 5:39
  English,
• 5:39
  with
• 5:39
  hundreds
• 5:40
  of
• 5:40
  monthly
• 5:40
  active
• 5:40
  users.
• 5:41
  Six
• 5:42
  months
• 5:42
  ago,
• 5:43
  this
• 5:43
  required
• 5:43
  an
• 5:43
  RTX
• 5:44
  5090
• 5:45
  for
• 5:45
  voice
• 5:46
  models
• 5:46
  alone.
• 5:47
  The
• 5:47
  fact
• 5:47
  that
• 5:47
  it
• 5:47
  runs
• 5:48
  on
• 5:48
  a
• 5:48
  consumer
• 5:48
  laptop
• 5:49
  with
• 5:49
  3GB
• 5:49
  of
• 5:50
  RAM
• 5:50
  for
• 5:50
  the
• 5:50
  model
• 5:51
  represents
• 5:51
  a
• 5:51
  genuine
• 5:52
  step
• 5:52
  function.
• 5:53
  On
• 5:53
  the
• 5:53
  quantization
• 5:54
  front,
• 5:54
  the
• 5:55
  Gemma-4-E2B-it
• 5:56
  model
• 5:56
  quantized
• 5:57
  to
• 5:57
  NVFP4
• 5:58
  (W4A4)
• 6:00
  using
• 6:00
  NVIDIA
• 6:01
  Model
• 6:01
  Optimizer
• 6:02
  is
• 6:02
  putting
• 6:02
  up
• 6:02
  surprising
• 6:03
  numbers
• 6:03
  on
• 6:04
  a
• 6:04
  single
• 6:04
  DGX
• 6:05
  Spark.
• 6:05
  At
• 6:06
  89
• 6:06
  tokens/sec
• 6:07
  single-user
• 6:08
  decode,
• 6:08
  it
• 6:09
  ranks
• 6:09
  #2
• 6:09
  across
• 6:10
  ~57
• 6:10
  models
• 6:11
  on
• 6:11
  the
• 6:11
  Spark
• 6:11
  Arena
• 6:12
  leaderboard
• 6:12
  —
• 6:13
  behind
• 6:13
  only
• 6:13
  Qwen3.5-0.8B,
• 6:18
  which
• 6:18
  is
• 6:18
  2.5x
• 6:19
  smaller.
• 6:20
  More
• 6:21
  interesting
• 6:21
  is
• 6:21
  the
• 6:21
  concurrency
• 6:22
  behavior:
• 6:23
  at
• 6:23
  10
• 6:23
  concurrent
• 6:23
  sessions,
• 6:24
  the
• 6:24
  E2B
• 6:25
  model
• 6:25
  actually
• 6:25
  gains
• 6:26
  rank,
• 6:26
  taking
• 6:27
  #1
• 6:27
  in
• 6:27
  both
• 6:28
  token
• 6:28
  generation
• 6:28
  and
• 6:29
  prompt
• 6:29
  processing
• 6:29
  at
• 6:29
  4K-16K
• 6:31
  depths.
• 6:32
  The
• 6:32
  Per-Layer
• 6:33
  Embeddings
• 6:33
  architecture
• 6:34
  uses
• 6:34
  sliding
• 6:35
  window
• 6:35
  attention
• 6:35
  on
• 6:36
  28
• 6:36
  of
• 6:36
  35
• 6:37
  layers
• 6:37
  (window=512),
• 6:39
  keeping
• 6:40
  KV
• 6:40
  cache
• 6:40
  tiny
• 6:41
  even
• 6:41
  under
• 6:41
  load.
• 6:42
  The
• 6:42
  quantized
• 6:43
  model
• 6:43
  fits
• 6:43
  in
• 6:44
  7.5GB
• 6:45
  on
• 6:45
  disk
• 6:45
  with
• 6:46
  vision
• 6:46
  and
• 6:46
  audio
• 6:46
  towers
• 6:47
  preserved
• 6:47
  in
• 6:48
  BF16.
• 6:49
  Unsloth
• 6:50
  published
• 6:50
  a
• 6:50
  comprehensive
• 6:51
  Gemma
• 6:51
  4
• 6:51
  fine-tuning
• 6:52
  guide
• 6:52
  that
• 6:52
  doubles
• 6:53
  as
• 6:53
  a
• 6:53
  bug
• 6:53
  report.
• 6:54
  They
• 6:55
  found
• 6:55
  and
• 6:55
  fixed
• 6:55
  several
• 6:56
  upstream
• 6:56
  issues:
• 6:57
  Gemma-4
• 6:58
  E2B
• 6:58
  and
• 6:59
  E4B
• 6:59
  share
• 6:59
  KV
• 7:00
  state
• 7:00
  across
• 7:00
  layers,
• 7:01
  and
• 7:01
  disabling
• 7:02
  the
• 7:02
  cache
• 7:02
  (as
• 7:02
  every
• 7:03
  QLoRA
• 7:03
  tutorial
• 7:04
  sets
• 7:04
  use
• 7:04
  cache=False)
• 7:05
  causes
• 7:06
  the
• 7:06
  KV-shared
• 7:06
  layers
• 7:07
  to
• 7:07
  recompute
• 7:07
  locally,
• 7:08
  producing
• 7:09
  garbage
• 7:09
  logits.
• 7:10
  The
• 7:10
  fix
• 7:11
  is
• 7:11
  simple
• 7:11
  —
• 7:12
  keep
• 7:13
  use
• 7:13
  cache=True
• 7:14
  —
• 7:15
  but
• 7:15
  the
• 7:15
  failure
• 7:16
  mode
• 7:16
  is
• 7:16
  silent
• 7:16
  and
• 7:17
  devastating:
• 7:18
  the
• 7:18
  model
• 7:18
  outputs
• 7:19
  gibberish
• 7:19
  without
• 7:19
  any
• 7:19
  error
• 7:20
  message.
• 7:21
  They
• 7:21
  also
• 7:21
  caught
• 7:21
  a
• 7:21
  gradient
• 7:22
  accumulation
• 7:22
  bug
• 7:23
  inflating
• 7:23
  losses,
• 7:24
  and
• 7:24
  a
• 7:24
  num
• 7:24
  hidden
• 7:24
  layers=0
• 7:26
  misconfiguration
• 7:27
  in
• 7:27
  the
• 7:27
  31B
• 7:27
  and
• 7:27
  26B
• 7:28
  variants
• 7:29
  that
• 7:29
  crashes
• 7:29
  the
• 7:29
  cache
• 7:29
  on
• 7:30
  the
• 7:30
  first
• 7:30
  attention
• 7:30
  forward.
• 7:31
  E2B
• 7:32
  LoRA
• 7:32
  works
• 7:32
  on
• 7:32
  8-10GB
• 7:35
  VRAM;
• 7:36
  31B
• 7:36
  QLoRA
• 7:37
  fits
• 7:37
  in
• 7:37
  22GB.
• 7:39
  Unsloth
• 7:39
  claims
• 7:39
  30%
• 7:40
  less
• 7:40
  memory
• 7:41
  than
• 7:41
  FA2
• 7:41
  setups
• 7:42
  with
• 7:42
  no
• 7:42
  accuracy
• 7:42
  loss.
• 7:43
  The
• 7:43
  community
• 7:44
  quantization
• 7:44
  debate
• 7:45
  is
• 7:45
  settling
• 7:45
  into
• 7:45
  practical
• 7:46
  consensus:
• 7:47
  Bartowski
• 7:48
  GGUF
• 7:48
  quantizations
• 7:49
  are
• 7:49
  generally
• 7:50
  preferred
• 7:50
  for
• 7:50
  consistency
• 7:51
  and
• 7:51
  long-context
• 7:52
  agent
• 7:52
  coding
• 7:52
  sessions,
• 7:53
  while
• 7:54
  Unsloth
• 7:54
  has
• 7:54
  the
• 7:54
  edge
• 7:55
  in
• 7:55
  marketing
• 7:55
  and
• 7:55
  training
• 7:56
  workflows.
• 7:57
  One
• 7:57
  user
• 7:57
  reports
• 7:58
  Bartowski
• 7:58
  IQ2
• 7:59
  M
• 7:59
  for
• 7:59
  the
• 7:59
  26B
• 8:00
  model
• 8:00
  running
• 8:01
  at
• 8:01
  65
• 8:01
  tokens/sec
• 8:02
  on
• 8:02
  an
• 8:02
  RTX
• 8:03
  3060
• 8:04
  12GB
• 8:05
  without
• 8:05
  hallucinations.
• 8:06
  The
• 8:06
  26B-A4B
• 8:08
  at
• 8:09
  Q6
• 8:09
  K
• 8:09
  XL
• 8:10
  fits
• 8:10
  most
• 8:10
  gaming
• 8:11
  GPUs
• 8:11
  with
• 8:11
  partial
• 8:12
  RAM
• 8:12
  offloading.
• 8:13
  Meanwhile,
• 8:14
  inferrs
• 8:14
  enters
• 8:14
  the
• 8:15
  inference
• 8:15
  server
• 8:15
  space
• 8:16
  as
• 8:16
  a
• 8:16
  Rust-based
• 8:16
  alternative
• 8:17
  to
• 8:17
  vLLM
• 8:18
  and
• 8:18
  llama.cpp.
• 8:19
  It
• 8:20
  ships
• 8:20
  as
• 8:20
  a
• 8:20
  single
• 8:20
  binary
• 8:21
  with
• 8:21
  OpenAI,
• 8:21
  Anthropic,
• 8:22
  and
• 8:22
  Ollama-compatible
• 8:23
  APIs,
• 8:24
  supports
• 8:24
  CUDA,
• 8:24
  ROCm,
• 8:25
  Metal,
• 8:26
  Vulkan,
• 8:26
  and
• 8:26
  CPU
• 8:27
  backends,
• 8:27
  and
• 8:27
  features
• 8:28
  a
• 8:28
  "TurboQuant"
• 8:28
  KV
• 8:29
  cache
• 8:29
  management
• 8:29
  strategy
• 8:30
  alongside
• 8:30
  PagedAttention
• 8:31
  support.
• 8:32
  The
• 8:32
  pitch:
• 8:33
  vLLM's
• 8:33
  features
• 8:34
  without
• 8:34
  Python's
• 8:35
  overhead,
• 8:35
  llama.cpp's
• 8:37
  lightweight
• 8:37
  profile
• 8:37
  with
• 8:38
  streaming
• 8:38
  and
• 8:38
  multi-backend
• 8:39
  support.
• 8:45
  An
• 8:45
  AI
• 8:45
  "singer"
• 8:46
  named
• 8:46
  Eddie
• 8:46
  Dalton
• 8:46
  —
• 8:47
  created
• 8:47
  by
• 8:47
  content
• 8:48
  creator
• 8:48
  Dallas
• 8:49
  Little
• 8:49
  —
• 8:49
  now
• 8:50
  occupies
• 8:50
  eleven
• 8:51
  spots
• 8:51
  on
• 8:51
  the
• 8:51
  iTunes
• 8:51
  top
• 8:52
  100
• 8:52
  singles
• 8:52
  chart
• 8:53
  and
• 8:53
  holds
• 8:53
  the
• 8:53
  #3
• 8:53
  album.
• 8:55
  Little
• 8:55
  writes
• 8:55
  songs,
• 8:55
  records
• 8:56
  them
• 8:56
  with
• 8:56
  AI,
• 8:56
  and
• 8:57
  invented
• 8:57
  Dalton's
• 8:57
  look,
• 8:57
  sound,
• 8:58
  and
• 8:58
  videos
• 8:58
  entirely
• 8:59
  through
• 8:59
  prompt-driven
• 8:59
  generation.
• 9:01
  Positions
• 9:01
  3,
• 9:01
  8,
• 9:02
  15,
• 9:02
  22,
• 9:02
  42,
• 9:03
  44,
• 9:03
  51,
• 9:04
  58,
• 9:04
  60,
• 9:04
  68,
• 9:05
  and
• 9:05
  79.
• 9:06
  One
• 9:06
  track,
• 9:06
  "Another
• 9:06
  Day
• 9:07
  Old,"
• 9:07
  has
• 9:07
  1.2
• 9:08
  million
• 9:08
  YouTube
• 9:08
  views.
• 9:09
  But
• 9:09
  the
• 9:09
  numbers
• 9:09
  don't
• 9:09
  add
• 9:10
  up:
• 9:10
  Luminate
• 9:10
  reports
• 9:11
  just
• 9:11
  6,900
• 9:12
  total
• 9:12
  track
• 9:12
  sales,
• 9:13
  no
• 9:13
  radio
• 9:13
  airplay,
• 9:14
  and
• 9:14
  no
• 9:14
  streaming
• 9:15
  presence.
• 9:15
  How
• 9:15
  an
• 9:16
  AI
• 9:16
  act
• 9:16
  with
• 9:16
  under
• 9:16
  7,000
• 9:17
  sales
• 9:17
  holds
• 9:17
  eleven
• 9:18
  chart
• 9:18
  positions
• 9:18
  simultaneously
• 9:19
  is
• 9:19
  a
• 9:19
  question
• 9:20
  nobody
• 9:20
  at
• 9:20
  Apple
• 9:20
  seems
• 9:21
  interested
• 9:21
  in
• 9:21
  answering.
• 9:22
  The
• 9:22
  broader
• 9:22
  point:
• 9:23
  chart
• 9:23
  infrastructure
• 9:23
  was
• 9:24
  designed
• 9:24
  for
• 9:24
  a
• 9:24
  world
• 9:24
  where
• 9:24
  making
• 9:25
  and
• 9:25
  distributing
• 9:25
  music
• 9:26
  had
• 9:26
  real
• 9:26
  costs.
• 9:27
  When
• 9:27
  production
• 9:27
  cost
• 9:27
  drops
• 9:28
  to
• 9:28
  zero
• 9:28
  and
• 9:28
  distribution
• 9:29
  is
• 9:29
  unlimited,
• 9:30
  the
• 9:30
  integrity
• 9:30
  assumptions
• 9:31
  baked
• 9:31
  into
• 9:31
  these
• 9:31
  systems
• 9:31
  stop
• 9:32
  holding.
• 9:33
  The
• 9:33
  supply
• 9:33
  side
• 9:33
  of
• 9:33
  that
• 9:33
  equation
• 9:34
  keeps
• 9:34
  expanding.
• 9:35
  Mistral
• 9:35
  released
• 9:36
  Voxtral
• 9:36
  TTS,
• 9:37
  an
• 9:37
  open-weight
• 9:37
  text-to-voice
• 9:38
  model
• 9:38
  claiming
• 9:39
  to
• 9:39
  clone
• 9:39
  any
• 9:39
  voice
• 9:40
  from
• 9:40
  three
• 9:40
  seconds
• 9:40
  of
• 9:40
  audio
• 9:41
  and
• 9:41
  beat
• 9:41
  ElevenLabs
• 9:42
  on
• 9:42
  quality
• 9:42
  benchmarks.
• 9:44
  Community
• 9:44
  reception
• 9:45
  was
• 9:45
  lukewarm
• 9:45
  —
• 9:46
  the
• 9:46
  local
• 9:46
  install
• 9:47
  doesn't
• 9:47
  actually
• 9:47
  support
• 9:48
  voice
• 9:48
  cloning,
• 9:49
  the
• 9:49
  non-English
• 9:49
  output
• 9:50
  has
• 9:50
  an
• 9:50
  extremely
• 9:51
  hard
• 9:51
  accent
• 9:51
  despite
• 9:52
  multilingual
• 9:52
  claims,
• 9:53
  and
• 9:53
  the
• 9:53
  release
• 9:54
  was
• 9:54
  largely
• 9:54
  a
• 9:54
  reannouncement
• 9:55
  of
• 9:55
  something
• 9:55
  shown
• 9:55
  weeks
• 9:56
  earlier.
• 9:57
  VoxCPM2
• 9:58
  enters
• 9:58
  the
• 9:58
  same
• 9:59
  space
• 9:59
  with
• 9:59
  three
• 9:59
  modes:
• 10:00
  voice
• 10:00
  design
• 10:01
  (create
• 10:01
  a
• 10:01
  brand-new
• 10:02
  voice),
• 10:03
  controllable
• 10:03
  cloning
• 10:04
  (clone
• 10:04
  with
• 10:04
  style
• 10:05
  guidance),
• 10:06
  and
• 10:06
  ultimate
• 10:06
  cloning
• 10:07
  (reproduce
• 10:07
  every
• 10:08
  vocal
• 10:08
  nuance
• 10:08
  through
• 10:09
  audio
• 10:09
  continuation),
• 10:10
  claiming
• 10:11
  state-of-the-art
• 10:11
  results
• 10:12
  on
• 10:12
  Seed-TTS-eval
• 10:13
  and
• 10:14
  other
• 10:14
  zero-shot
• 10:14
  TTS
• 10:15
  benchmarks.
• 10:16
  On
• 10:16
  the
• 10:16
  music
• 10:16
  generation
• 10:17
  side,
• 10:17
  Ace
• 10:18
  Step
• 10:18
  released
• 10:18
  the
• 10:18
  1.5
• 10:19
  XL
• 10:20
  models
• 10:20
  —
• 10:21
  turbo,
• 10:21
  base,
• 10:22
  and
• 10:22
  SFT
• 10:22
  variants
• 10:23
  —
• 10:24
  a
• 10:24
  week
• 10:24
  after
• 10:24
  the
• 10:24
  software
• 10:25
  update,
• 10:25
  having
• 10:26
  apparently
• 10:26
  forgotten
• 10:26
  to
• 10:26
  publish
• 10:27
  the
• 10:27
  model
• 10:27
  weights
• 10:27
  with
• 10:28
  the
• 10:28
  initial
• 10:28
  release.
• 10:32
  Anthropic's
• 10:33
  Claude
• 10:33
  Managed
• 10:33
  Agents
• 10:34
  entered
• 10:34
  public
• 10:34
  beta,
• 10:35
  and
• 10:35
  the
• 10:35
  pitch
• 10:35
  is
• 10:35
  explicit:
• 10:36
  stop
• 10:36
  building
• 10:37
  agent
• 10:37
  infrastructure
• 10:37
  yourself.
• 10:39
  The
• 10:39
  platform
• 10:39
  provides
• 10:39
  secure
• 10:40
  sandboxing,
• 10:41
  state
• 10:41
  management,
• 10:42
  credential
• 10:42
  scoping,
• 10:43
  long-running
• 10:43
  sessions
• 10:43
  (hours,
• 10:44
  even
• 10:44
  if
• 10:44
  you
• 10:44
  disconnect),
• 10:45
  error
• 10:45
  recovery,
• 10:46
  tracing,
• 10:46
  and
• 10:46
  multi-agent
• 10:47
  coordination
• 10:47
  where
• 10:47
  one
• 10:47
  agent
• 10:48
  can
• 10:48
  spin
• 10:48
  up
• 10:48
  others
• 10:48
  for
• 10:49
  parallel
• 10:49
  work.
• 10:50
  The
• 10:50
  gap
• 10:50
  it
• 10:50
  targets
• 10:51
  is
• 10:51
  real
• 10:51
  —
• 10:52
  sandboxing,
• 10:52
  checkpointing,
• 10:53
  scoped
• 10:54
  permissions,
• 10:54
  and
• 10:54
  orchestration
• 10:55
  typically
• 10:55
  take
• 10:55
  months
• 10:55
  to
• 10:56
  build
• 10:56
  in-house.
• 10:57
  The
• 10:57
  community
• 10:57
  response
• 10:57
  was
• 10:58
  mixed:
• 10:58
  some
• 10:58
  builders
• 10:59
  see
• 10:59
  it
• 10:59
  as
• 10:59
  Amazon
• 10:59
  Bedrock
• 11:00
  AgentCore
• 11:00
  with
• 11:00
  Claude
• 11:01
  as
• 11:01
  the
• 11:01
  model,
• 11:01
  while
• 11:01
  others
• 11:02
  are
• 11:02
  more
• 11:02
  pointed
• 11:02
  —
• 11:03
  "just
• 11:03
  give
• 11:03
  us
• 11:03
  a
• 11:03
  working
• 11:04
  model,
• 11:04
  stop
• 11:04
  nerfing
• 11:05
  our
• 11:05
  usage
• 11:05
  time
• 11:05
  and
• 11:05
  model
• 11:06
  quality."
• 11:06
  The
• 11:06
  complaint
• 11:07
  about
• 11:07
  quality
• 11:07
  degradation
• 11:08
  in
• 11:08
  managed
• 11:08
  contexts
• 11:09
  echoes
• 11:09
  a
• 11:09
  persistent
• 11:10
  concern:
• 11:11
  infrastructure
• 11:11
  is
• 11:12
  useless
• 11:12
  if
• 11:12
  the
• 11:12
  model
• 11:12
  running
• 11:13
  inside
• 11:13
  it
• 11:13
  has
• 11:13
  been
• 11:13
  throttled.
• 11:14
  At
• 11:15
  the
• 11:15
  other
• 11:15
  end
• 11:15
  of
• 11:15
  the
• 11:15
  spectrum,
• 11:16
  botctl
• 11:16
  treats
• 11:16
  AI
• 11:17
  agents
• 11:17
  as
• 11:17
  system
• 11:18
  services
• 11:18
  managed
• 11:18
  through
• 11:19
  a
• 11:19
  declarative
• 11:19
  configuration.
• 11:20
  Write
• 11:21
  a
• 11:21
  BOT.md
• 11:22
  file
• 11:22
  with
• 11:22
  YAML
• 11:22
  frontmatter
• 11:23
  for
• 11:23
  settings
• 11:23
  and
• 11:24
  markdown
• 11:24
  body
• 11:24
  for
• 11:24
  the
• 11:24
  prompt,
• 11:25
  and
• 11:25
  botctl
• 11:25
  spawns
• 11:26
  Claude
• 11:26
  with
• 11:26
  your
• 11:26
  tools
• 11:27
  and
• 11:27
  workspace,
• 11:27
  running
• 11:28
  on
• 11:28
  a
• 11:28
  schedule.
• 11:29
  Every
• 11:29
  run
• 11:29
  saves
• 11:29
  its
• 11:30
  session
• 11:30
  for
• 11:30
  resumption,
• 11:30
  you
• 11:30
  can
• 11:31
  send
• 11:31
  messages
• 11:31
  to
• 11:31
  redirect
• 11:32
  running
• 11:32
  bots,
• 11:32
  and
• 11:33
  config
• 11:33
  changes
• 11:33
  are
• 11:33
  picked
• 11:34
  up
• 11:34
  automatically
• 11:34
  on
• 11:34
  the
• 11:34
  next
• 11:35
  run
• 11:35
  cycle
• 11:35
  —
• 11:35
  no
• 11:36
  restarts,
• 11:36
  no
• 11:36
  deploys.
• 11:38
  A
• 11:38
  skills
• 11:38
  system
• 11:39
  lets
• 11:39
  you
• 11:39
  search,
• 11:40
  install,
• 11:40
  and
• 11:40
  share
• 11:40
  reusable
• 11:41
  capability
• 11:41
  modules
• 11:42
  from
• 11:42
  GitHub.
• 11:43
  It's
• 11:43
  the
• 11:43
  systemd
• 11:44
  for
• 11:44
  AI
• 11:44
  agents:
• 11:45
  boring,
• 11:45
  practical
• 11:46
  infrastructure
• 11:46
  that
• 11:46
  treats
• 11:47
  autonomous
• 11:47
  processes
• 11:48
  as
• 11:48
  first-class
• 11:48
  system
• 11:49
  primitives.
• 11:50
  Feynman
• 11:50
  positions
• 11:51
  itself
• 11:51
  as
• 11:51
  an
• 11:51
  open-source
• 11:52
  AI
• 11:52
  research
• 11:52
  agent.
• 11:54
  Point
• 11:54
  it
• 11:54
  at
• 11:54
  a
• 11:54
  topic
• 11:55
  and
• 11:55
  it
• 11:55
  dispatches
• 11:55
  four
• 11:56
  bundled
• 11:56
  agents
• 11:56
  —
• 11:57
  Researcher,
• 11:58
  Reviewer,
• 11:58
  Writer,
• 11:59
  Verifier
• 11:59
  —
• 12:00
  across
• 12:00
  papers,
• 12:01
  web,
• 12:01
  repos,
• 12:01
  and
• 12:01
  docs.
• 12:02
  The
• 12:02
  deep
• 12:03
  research
• 12:03
  workflow
• 12:03
  runs
• 12:04
  multi-agent
• 12:04
  investigation
• 12:05
  with
• 12:05
  parallel
• 12:05
  researchers,
• 12:06
  synthesis,
• 12:07
  and
• 12:07
  verification.
• 12:08
  A
• 12:08
  /replicate
• 12:09
  command
• 12:09
  attempts
• 12:10
  to
• 12:10
  reproduce
• 12:10
  paper
• 12:10
  experiments
• 12:11
  on
• 12:11
  local
• 12:11
  or
• 12:11
  cloud
• 12:11
  GPUs
• 12:12
  via
• 12:12
  Modal
• 12:12
  or
• 12:12
  RunPod
• 12:13
  integration.
• 12:14
  Every
• 12:14
  output
• 12:15
  is
• 12:15
  source-grounded
• 12:15
  with
• 12:15
  direct
• 12:16
  URLs,
• 12:17
  and
• 12:17
  it
• 12:17
  includes
• 12:17
  an
• 12:17
  /audit
• 12:18
  command
• 12:18
  that
• 12:18
  compares
• 12:19
  paper
• 12:19
  claims
• 12:19
  against
• 12:19
  public
• 12:20
  codebases.
• 12:21
  Built
• 12:21
  on
• 12:21
  Pi
• 12:22
  for
• 12:22
  the
• 12:22
  agent
• 12:22
  runtime
• 12:22
  and
• 12:22
  alphaXiv
• 12:23
  for
• 12:23
  paper
• 12:23
  search,
• 12:24
  it's
• 12:24
  the
• 12:24
  kind
• 12:25
  of
• 12:25
  specialized
• 12:25
  tool
• 12:25
  that
• 12:26
  only
• 12:26
  makes
• 12:26
  sense
• 12:26
  once
• 12:27
  agent
• 12:27
  frameworks
• 12:27
  mature
• 12:28
  enough
• 12:28
  to
• 12:28
  support
• 12:28
  it.
• 12:29
  The
• 12:29
  Claude
• 12:29
  Code
• 12:29
  Video
• 12:30
  Toolkit
• 12:30
  takes
• 12:30
  the
• 12:30
  "what
• 12:31
  can
• 12:31
  Claude
• 12:31
  Code
• 12:31
  automate?"
• 12:32
  question
• 12:33
  in
• 12:33
  an
• 12:33
  unexpected
• 12:33
  direction:
• 12:34
  full
• 12:34
  video
• 12:35
  production.
• 12:36
  It
• 12:36
  provides
• 12:37
  skills,
• 12:37
  commands,
• 12:38
  templates,
• 12:39
  and
• 12:39
  tools
• 12:39
  that
• 12:39
  let
• 12:40
  Claude
• 12:40
  Code
• 12:40
  orchestrate
• 12:41
  everything
• 12:41
  from
• 12:41
  script
• 12:41
  to
• 12:41
  final
• 12:42
  render
• 12:42
  —
• 12:43
  AI
• 12:43
  voiceover
• 12:44
  via
• 12:44
  Qwen3-TTS,
• 12:45
  image
• 12:45
  generation
• 12:46
  via
• 12:46
  FLUX.2,
• 12:47
  music
• 12:48
  via
• 12:48
  ACE-Step,
• 12:49
  talking-head
• 12:49
  animation
• 12:50
  via
• 12:50
  SadTalker,
• 12:51
  and
• 12:51
  video
• 12:51
  generation
• 12:51
  via
• 12:52
  LTX-2.
• 12:53
  Projects
• 12:54
  track
• 12:54
  through
• 12:54
  a
• 12:54
  multi-session
• 12:55
  lifecycle
• 12:55
  (planning
• 12:56
  →
• 12:56
  assets
• 12:56
  →
• 12:57
  review
• 12:57
  →
• 12:57
  audio
• 12:57
  →
• 12:58
  editing
• 12:58
  →
• 12:58
  rendering),
• 12:59
  with
• 12:59
  automatic
• 13:00
  reconciliation
• 13:00
  of
• 13:00
  planned
• 13:01
  intent
• 13:01
  versus
• 13:01
  what
• 13:01
  files
• 13:02
  actually
• 13:02
  exist.
• 13:03
  Cloud
• 13:04
  GPU
• 13:04
  work
• 13:04
  runs
• 13:05
  on
• 13:05
  Modal
• 13:05
  ($30/month
• 13:06
  free
• 13:06
  tier)
• 13:07
  or
• 13:07
  RunPod.
• 13:08
  The
• 13:08
  author's
• 13:08
  use
• 13:08
  case
• 13:09
  is
• 13:09
  sprint
• 13:09
  review
• 13:09
  videos,
• 13:10
  but
• 13:10
  the
• 13:10
  framework
• 13:11
  is
• 13:11
  generalizable
• 13:12
  to
• 13:12
  any
• 13:12
  "explainer"
• 13:13
  format.
• 13:17
  Safetensors
• 13:17
  has
• 13:18
  joined
• 13:18
  the
• 13:18
  PyTorch
• 13:18
  Foundation
• 13:19
  as
• 13:19
  a
• 13:19
  foundation-hosted
• 13:20
  project
• 13:21
  under
• 13:21
  the
• 13:21
  Linux
• 13:21
  Foundation,
• 13:22
  alongside
• 13:23
  DeepSpeed,
• 13:23
  vLLM,
• 13:24
  and
• 13:24
  PyTorch
• 13:24
  itself.
• 13:25
  The
• 13:26
  move
• 13:26
  is
• 13:26
  a
• 13:26
  governance
• 13:26
  milestone:
• 13:27
  the
• 13:27
  format
• 13:28
  that
• 13:28
  eliminated
• 13:28
  arbitrary
• 13:29
  code
• 13:29
  execution
• 13:30
  from
• 13:30
  model
• 13:30
  distribution
• 13:31
  now
• 13:31
  has
• 13:31
  vendor-neutral
• 13:32
  trademark,
• 13:32
  repository,
• 13:33
  and
• 13:33
  governance.
• 13:34
  For
• 13:34
  users,
• 13:35
  nothing
• 13:35
  changes
• 13:35
  —
• 13:36
  same
• 13:36
  format,
• 13:37
  same
• 13:37
  APIs,
• 13:38
  same
• 13:38
  Hub
• 13:38
  integration.
• 13:39
  For
• 13:39
  the
• 13:39
  ecosystem,
• 13:40
  it
• 13:41
  signals
• 13:41
  that
• 13:41
  safetensors
• 13:42
  is
• 13:42
  no
• 13:42
  longer
• 13:42
  a
• 13:42
  Hugging
• 13:43
  Face
• 13:43
  project
• 13:43
  that
• 13:43
  happens
• 13:44
  to
• 13:44
  be
• 13:44
  open
• 13:44
  source;
• 13:45
  it's
• 13:45
  community
• 13:45
  infrastructure
• 13:46
  with
• 13:46
  formal
• 13:47
  contributor
• 13:47
  pathways.
• 13:48
  The
• 13:48
  roadmap
• 13:49
  ahead
• 13:49
  targets
• 13:49
  problems
• 13:50
  the
• 13:50
  whole
• 13:50
  ecosystem
• 13:51
  shares:
• 13:52
  device-aware
• 13:52
  loading
• 13:52
  (direct
• 13:53
  to
• 13:53
  CUDA/ROCm
• 13:55
  without
• 13:55
  CPU
• 13:55
  staging),
• 13:56
  first-class
• 13:57
  Tensor
• 13:57
  Parallel
• 13:57
  and
• 13:58
  Pipeline
• 13:58
  Parallel
• 13:58
  loading
• 13:58
  APIs,
• 14:00
  and
• 14:00
  formalized
• 14:00
  support
• 14:00
  for
• 14:01
  FP8,
• 14:01
  GPTQ,
• 14:02
  AWQ,
• 14:03
  and
• 14:03
  sub-byte
• 14:03
  integer
• 14:03
  types.
• 14:04
  The
• 14:05
  real
• 14:05
  news
• 14:05
  buried
• 14:05
  in
• 14:05
  the
• 14:06
  announcement:
• 14:06
  the
• 14:06
  team
• 14:07
  is
• 14:07
  working
• 14:07
  with
• 14:07
  PyTorch
• 14:08
  so
• 14:08
  that
• 14:08
  safetensors
• 14:08
  may
• 14:08
  be
• 14:08
  used
• 14:09
  within
• 14:09
  PyTorch
• 14:09
  core
• 14:10
  as
• 14:10
  a
• 14:10
  serialization
• 14:11
  system
• 14:11
  for
• 14:11
  torch
• 14:11
  models.
• 14:12
  If
• 14:12
  that
• 14:13
  ships,
• 14:13
  pickle-based
• 14:14
  model
• 14:14
  distribution
• 14:15
  finally
• 14:15
  has
• 14:15
  an
• 14:15
  expiration
• 14:16
  date.
• 14:17
  AWS
• 14:17
  announced
• 14:17
  S3
• 14:18
  Files,
• 14:18
  which
• 14:19
  integrates
• 14:19
  Amazon
• 14:20
  EFS
• 14:20
  into
• 14:20
  S3
• 14:21
  and
• 14:21
  allows
• 14:21
  any
• 14:21
  existing
• 14:22
  S3
• 14:22
  data
• 14:22
  to
• 14:22
  be
• 14:23
  accessed
• 14:23
  as
• 14:23
  a
• 14:23
  network-attached
• 14:24
  filesystem.
• 14:25
  Mount
• 14:25
  any
• 14:25
  bucket
• 14:26
  or
• 14:26
  prefix
• 14:26
  inside
• 14:26
  an
• 14:27
  EC2
• 14:27
  VM,
• 14:28
  container,
• 14:28
  or
• 14:28
  Lambda
• 14:29
  function,
• 14:29
  and
• 14:30
  your
• 14:30
  tools
• 14:30
  see
• 14:30
  a
• 14:30
  local
• 14:30
  directory.
• 14:31
  Changes
• 14:32
  propagate
• 14:32
  back
• 14:32
  to
• 14:32
  S3.
• 14:34
  The
• 14:34
  design
• 14:34
  story
• 14:34
  is
• 14:34
  the
• 14:34
  interesting
• 14:35
  part:
• 14:36
  the
• 14:36
  team
• 14:36
  spent
• 14:36
  months
• 14:36
  trying
• 14:37
  to
• 14:37
  unify
• 14:37
  file
• 14:37
  and
• 14:38
  object
• 14:38
  semantics
• 14:38
  into
• 14:39
  a
• 14:39
  single
• 14:39
  invisible
• 14:39
  abstraction,
• 14:40
  failed,
• 14:41
  locked
• 14:41
  their
• 14:42
  senior
• 14:42
  engineers
• 14:42
  in
• 14:42
  a
• 14:42
  room
• 14:43
  over
• 14:43
  Christmas
• 14:43
  2024,
• 14:44
  and
• 14:44
  eventually
• 14:45
  realized
• 14:45
  the
• 14:45
  boundary
• 14:46
  between
• 14:46
  file
• 14:46
  and
• 14:46
  object
• 14:47
  is
• 14:47
  the
• 14:47
  feature,
• 14:47
  not
• 14:48
  the
• 14:48
  problem.
• 14:49
  They
• 14:49
  adopted
• 14:49
  a
• 14:49
  "stage
• 14:50
  and
• 14:50
  commit"
• 14:50
  model
• 14:50
  borrowed
• 14:51
  from
• 14:51
  git:
• 14:51
  changes
• 14:52
  accumulate
• 14:52
  in
• 14:53
  EFS
• 14:53
  and
• 14:53
  commit
• 14:54
  back
• 14:54
  to
• 14:54
  S3
• 14:54
  roughly
• 14:55
  every
• 14:55
  60
• 14:56
  seconds.
• 14:57
  Lazy
• 14:57
  metadata
• 14:57
  hydration
• 14:58
  means
• 14:58
  mounting
• 14:58
  a
• 14:59
  bucket
• 14:59
  with
• 14:59
  millions
• 14:59
  of
• 15:00
  objects
• 15:00
  starts
• 15:00
  immediately.
• 15:01
  For
• 15:02
  high-throughput
• 15:02
  reads,
• 15:03
  "read
• 15:03
  bypass"
• 15:04
  reroutes
• 15:04
  directly
• 15:04
  to
• 15:05
  S3
• 15:05
  via
• 15:05
  parallel
• 15:06
  GETs,
• 15:07
  achieving
• 15:07
  3
• 15:08
  GB/s
• 15:08
  per
• 15:09
  client.
• 15:10
  The
• 15:10
  edges
• 15:10
  are
• 15:10
  honest:
• 15:11
  renames
• 15:12
  are
• 15:12
  expensive
• 15:13
  (S3
• 15:13
  has
• 15:13
  no
• 15:13
  native
• 15:14
  rename),
• 15:14
  explicit
• 15:15
  commit
• 15:15
  control
• 15:16
  isn't
• 15:16
  there
• 15:16
  at
• 15:16
  launch,
• 15:17
  and
• 15:17
  some
• 15:17
  object
• 15:18
  keys
• 15:18
  can't
• 15:18
  be
• 15:19
  represented
• 15:19
  as
• 15:19
  POSIX
• 15:20
  filenames.
• 15:21
  With
• 15:21
  S3
• 15:21
  Tables
• 15:22
  (Iceberg),
• 15:22
  S3
• 15:23
  Vectors,
• 15:23
  and
• 15:23
  now
• 15:24
  S3
• 15:24
  Files,
• 15:25
  the
• 15:25
  object
• 15:25
  store
• 15:25
  is
• 15:26
  quietly
• 15:26
  becoming
• 15:27
  a
• 15:27
  polyglot
• 15:27
  data
• 15:27
  platform.
• 15:28
  On
• 15:28
  the
• 15:28
  research
• 15:29
  side,
• 15:30
  MoECLIP
• 15:30
  brings
• 15:30
  Mixture-of-Experts
• 15:31
  architecture
• 15:32
  to
• 15:32
  zero-shot
• 15:33
  anomaly
• 15:33
  detection
• 15:33
  —
• 15:34
  detecting
• 15:35
  defects
• 15:35
  in
• 15:35
  images
• 15:36
  of
• 15:36
  objects
• 15:36
  the
• 15:36
  model
• 15:37
  has
• 15:37
  never
• 15:37
  seen
• 15:37
  during
• 15:38
  training.
• 15:39
  The
• 15:39
  core
• 15:39
  problem:
• 15:40
  existing
• 15:41
  CLIP-based
• 15:41
  anomaly
• 15:41
  detectors
• 15:42
  apply
• 15:43
  the
• 15:43
  same
• 15:43
  transformation
• 15:44
  to
• 15:44
  every
• 15:44
  image
• 15:44
  patch
• 15:45
  regardless
• 15:46
  of
• 15:46
  whether
• 15:46
  it's
• 15:46
  background,
• 15:47
  object
• 15:47
  body,
• 15:48
  or
• 15:48
  anomaly.
• 15:49
  MoECLIP
• 15:50
  dynamically
• 15:51
  routes
• 15:51
  each
• 15:51
  patch
• 15:52
  to
• 15:52
  a
• 15:52
  specialized
• 15:52
  LoRA
• 15:53
  expert
• 15:53
  based
• 15:53
  on
• 15:54
  its
• 15:54
  characteristics.
• 15:55
  Two
• 15:55
  mechanisms
• 15:56
  prevent
• 15:56
  experts
• 15:57
  from
• 15:57
  learning
• 15:57
  the
• 15:57
  same
• 15:58
  thing:
• 15:58
  Frozen
• 15:59
  Orthogonal
• 15:59
  Feature
• 16:00
  Separation
• 16:00
  (FOFS)
• 16:01
  forces
• 16:01
  experts
• 16:02
  into
• 16:02
  non-overlapping
• 16:03
  input
• 16:03
  subspaces,
• 16:04
  while
• 16:04
  a
• 16:04
  simplex
• 16:05
  equiangular
• 16:05
  tight
• 16:06
  frame
• 16:06
  (ETF)
• 16:06
  loss
• 16:07
  maximizes
• 16:08
  angular
• 16:08
  separation
• 16:09
  of
• 16:09
  expert
• 16:09
  outputs.
• 16:10
  Across
• 16:11
  14
• 16:11
  benchmark
• 16:12
  datasets
• 16:12
  spanning
• 16:13
  industrial
• 16:13
  manufacturing
• 16:14
  and
• 16:14
  medical
• 16:14
  imaging
• 16:15
  (brain
• 16:15
  MRI,
• 16:16
  liver
• 16:16
  CT,
• 16:17
  colon
• 16:17
  polyps),
• 16:18
  MoECLIP
• 16:19
  achieves
• 16:19
  state-of-the-art
• 16:20
  performance
• 16:20
  with
• 16:20
  improvements
• 16:21
  of
• 16:21
  3.0
• 16:22
  AUROC
• 16:22
  on
• 16:22
  image-level
• 16:23
  and
• 16:24
  1.1
• 16:24
  AUROC
• 16:25
  on
• 16:25
  pixel-level
• 16:26
  detection.
• 16:27
  The
• 16:27
  Grad-CAM
• 16:27
  visualizations
• 16:28
  show
• 16:28
  genuine
• 16:29
  specialization:
• 16:30
  one
• 16:30
  expert
• 16:31
  focuses
• 16:31
  on
• 16:31
  anomalies,
• 16:32
  another
• 16:33
  on
• 16:33
  object
• 16:33
  body,
• 16:34
  a
• 16:34
  third
• 16:34
  on
• 16:34
  background,
• 16:35
  each
• 16:35
  with
• 16:36
  distinct
• 16:36
  routing
• 16:36
  patterns.
• 16:37
  The
• 16:37
  practical
• 16:38
  appeal
• 16:38
  for
• 16:38
  industrial
• 16:39
  quality
• 16:39
  inspection
• 16:40
  and
• 16:40
  medical
• 16:40
  screening
• 16:41
  is
• 16:41
  obvious
• 16:41
  —
• 16:42
  zero-shot
• 16:43
  means
• 16:43
  no
• 16:43
  per-factory
• 16:44
  or
• 16:44
  per-pathology
• 16:45
  training
• 16:45
  data
• 16:45
  required.