Latest / AGI Dreams – Open, Uncensored, & Local - AI News Digest / AGI Dreams Podcast – May 20, 2026

Transcript

AGI Dreams Podcast – May 20, 2026 · AGI Dreams – Open, Uncensored, & Local - AI News Digest

• 0:00
  The
• 0:00
  npm
• 0:00
  account
• 0:01
  behind
• 0:01
  packages
• 0:01
  like
• 0:01
  size-sensor
• 0:02
  (4.2M
• 0:03
  downloads/month),
• 0:04
  echarts-for-react
• 0:05
  (3.8M),
• 0:06
  and
• 0:06
  @antv/scale
• 0:08
  (2.2M)
• 0:09
  was
• 0:09
  compromised
• 0:10
  on
• 0:10
  May
• 0:10
  19,
• 0:11
  and
• 0:11
  the
• 0:11
  attacker
• 0:11
  published
• 0:12
  637
• 0:13
  malicious
• 0:13
  versions
• 0:14
  across
• 0:14
  317
• 0:15
  packages
• 0:15
  in
• 0:15
  a
• 0:15
  22-minute
• 0:16
  automated
• 0:17
  burst.
• 0:17
  The
• 0:17
  payload
• 0:18
  is
• 0:18
  a
• 0:18
  498KB
• 0:20
  obfuscated
• 0:20
  Bun
• 0:20
  script
• 0:21
  that
• 0:21
  matches
• 0:21
  the
• 0:21
  tooling
• 0:22
  used
• 0:22
  in
• 0:22
  the
• 0:22
  SAP
• 0:22
  compromise
• 0:23
  three
• 0:23
  weeks
• 0:23
  earlier:
• 0:24
  same
• 0:24
  scanner
• 0:24
  architecture,
• 0:25
  same
• 0:25
  credential
• 0:26
  regex
• 0:26
  set,
• 0:26
  same
• 0:27
  obfuscation
• 0:27
  pattern.
• 0:28
  This
• 0:29
  is
• 0:29
  the
• 0:29
  Shai-Hulud
• 0:29
  actor's
• 0:30
  second
• 0:30
  major
• 0:30
  hit,
• 0:31
  and
• 0:31
  the
• 0:31
  sophistication
• 0:32
  has
• 0:32
  escalated
• 0:32
  considerably.
• 0:34
  The
• 0:34
  credential
• 0:34
  harvesting
• 0:35
  alone
• 0:35
  covers
• 0:35
  the
• 0:36
  full
• 0:36
  AWS
• 0:36
  chain
• 0:37
  (env
• 0:37
  vars,
• 0:37
  config
• 0:38
  files,
• 0:39
  EC2
• 0:39
  IMDS,
• 0:40
  ECS
• 0:40
  container
• 0:41
  metadata,
• 0:42
  Secrets
• 0:42
  Manager),
• 0:43
  Kubernetes
• 0:43
  service
• 0:44
  account
• 0:44
  tokens,
• 0:45
  HashiCorp
• 0:45
  Vault,
• 0:46
  GitHub
• 0:46
  PATs,
• 0:47
  npm
• 0:48
  tokens,
• 0:48
  SSH
• 0:49
  keys,
• 0:49
  and
• 0:50
  local
• 0:50
  password
• 0:50
  manager
• 0:51
  vaults
• 0:51
  including
• 0:51
  1Password
• 0:52
  and
• 0:52
  Bitwarden.
• 0:53
  Stolen
• 0:53
  data
• 0:54
  exfiltrates
• 0:54
  through
• 0:54
  two
• 0:54
  parallel
• 0:55
  channels:
• 0:56
  Git
• 0:56
  objects
• 0:56
  committed
• 0:57
  to
• 0:57
  public
• 0:57
  GitHub
• 0:57
  repositories
• 0:58
  under
• 0:58
  forged
• 0:59
  User-Agents,
• 1:00
  and
• 1:00
  RSA+AES
• 1:01
  encrypted
• 1:02
  HTTPS
• 1:03
  POSTs
• 1:03
  disguised
• 1:04
  as
• 1:04
  OpenTelemetry
• 1:05
  trace
• 1:05
  data.
• 1:06
  In
• 1:06
  CI
• 1:06
  environments,
• 1:07
  the
• 1:07
  payload
• 1:08
  exchanges
• 1:08
  GitHub
• 1:09
  Actions
• 1:09
  OIDC
• 1:09
  tokens
• 1:10
  for
• 1:10
  npm
• 1:11
  publish
• 1:11
  tokens,
• 1:12
  signs
• 1:12
  artifacts
• 1:13
  via
• 1:13
  Sigstore
• 1:13
  using
• 1:14
  the
• 1:14
  stolen
• 1:14
  identity,
• 1:15
  and
• 1:15
  injects
• 1:15
  persistence
• 1:16
  into
• 1:16
  workflows.
• 1:17
  That
• 1:17
  last
• 1:18
  part
• 1:18
  deserves
• 1:18
  emphasis:
• 1:19
  the
• 1:19
  attacker
• 1:19
  gets
• 1:19
  legitimately-signed
• 1:20
  artifacts
• 1:21
  with
• 1:21
  forged
• 1:22
  provenance,
• 1:23
  which
• 1:23
  means
• 1:23
  downstream
• 1:24
  consumers
• 1:24
  who
• 1:24
  verify
• 1:25
  Sigstore
• 1:25
  signatures
• 1:26
  will
• 1:26
  trust
• 1:26
  the
• 1:26
  poisoned
• 1:27
  packages.
• 1:28
  What
• 1:28
  makes
• 1:28
  this
• 1:28
  campaign
• 1:29
  genuinely
• 1:29
  novel
• 1:30
  is
• 1:30
  the
• 1:30
  AI
• 1:31
  agent
• 1:31
  hijacking.
• 1:32
  The
• 1:32
  payload
• 1:32
  installs
• 1:33
  hooks
• 1:33
  into
• 1:33
  Claude
• 1:34
  Code
• 1:34
  and
• 1:34
  Codex
• 1:35
  that
• 1:35
  re-execute
• 1:36
  the
• 1:36
  malware
• 1:36
  on
• 1:36
  every
• 1:37
  AI
• 1:37
  session,
• 1:38
  both
• 1:38
  locally
• 1:38
  and
• 1:39
  via
• 1:39
  commits
• 1:39
  to
• 1:39
  accessible
• 1:40
  GitHub
• 1:40
  repositories.
• 1:41
  VS
• 1:42
  Code
• 1:42
  gets
• 1:42
  a
• 1:42
  tasks.json
• 1:44
  injection
• 1:45
  for
• 1:45
  the
• 1:45
  same
• 1:45
  effect.
• 1:46
  A
• 1:46
  persistent
• 1:47
  systemd
• 1:47
  service
• 1:48
  installs
• 1:48
  a
• 1:48
  GitHub
• 1:48
  dead-drop
• 1:49
  C2
• 1:50
  backdoor:
• 1:51
  a
• 1:51
  Python
• 1:51
  daemon
• 1:52
  that
• 1:52
  polls
• 1:52
  GitHub's
• 1:52
  commit
• 1:53
  search
• 1:53
  API
• 1:54
  hourly
• 1:54
  for
• 1:54
  RSA-PSS
• 1:57
  signed
• 1:57
  commands
• 1:58
  in
• 1:58
  commit
• 1:58
  messages,
• 1:59
  then
• 1:59
  downloads
• 2:00
  and
• 2:00
  executes
• 2:00
  arbitrary
• 2:01
  code
• 2:01
  from
• 2:01
  the
• 2:01
  signed
• 2:02
  URL.
• 2:03
  The
• 2:03
  attack
• 2:03
  also
• 2:04
  propagates
• 2:04
  to
• 2:04
  other
• 2:05
  local
• 2:05
  Node.js
• 2:06
  projects
• 2:07
  and
• 2:07
  attempts
• 2:07
  Docker
• 2:08
  container
• 2:08
  escape
• 2:08
  via
• 2:09
  the
• 2:09
  host
• 2:09
  socket.
• 2:10
  For
• 2:10
  projects
• 2:11
  using
• 2:11
  semver
• 2:11
  ranges,
• 2:12
  compromised
• 2:13
  versions
• 2:13
  auto-resolve
• 2:14
  on
• 2:14
  install.
• 2:15
  If
• 2:15
  your
• 2:15
  lockfile
• 2:16
  references
• 2:16
  any
• 2:16
  package
• 2:17
  published
• 2:17
  by
• 2:17
  the
• 2:17
  affected
• 2:18
  account
• 2:18
  between
• 2:19
  01:44
• 2:19
  and
• 2:20
  02:06
• 2:20
  UTC
• 2:21
  on
• 2:21
  May
• 2:21
  19,
• 2:22
  audit
• 2:22
  immediately.
• 2:23
  Meanwhile,
• 2:24
  the
• 2:24
  dark
• 2:25
  token
• 2:25
  economy
• 2:26
  that
• 2:26
  funds
• 2:26
  much
• 2:26
  of
• 2:26
  this
• 2:27
  infrastructure
• 2:27
  continues
• 2:28
  to
• 2:28
  mature.
• 2:29
  Almost
• 2:29
  half
• 2:30
  of
• 2:30
  calls
• 2:30
  through
• 2:30
  cheap
• 2:31
  LLM
• 2:31
  proxies
• 2:32
  hit
• 2:32
  a
• 2:32
  different
• 2:32
  model
• 2:33
  than
• 2:33
  advertised,
• 2:34
  and
• 2:35
  every
• 2:35
  prompt
• 2:35
  is
• 2:35
  logged
• 2:36
  on
• 2:36
  the
• 2:36
  operator's
• 2:37
  server
• 2:37
  for
• 2:37
  downstream
• 2:38
  fraud
• 2:38
  and
• 2:38
  distillation.
• 2:40
  Eight
• 2:40
  public
• 2:40
  repos
• 2:41
  with
• 2:41
  roughly
• 2:41
  172,000
• 2:43
  combined
• 2:44
  GitHub
• 2:44
  stars
• 2:45
  actively
• 2:45
  resell
• 2:46
  unauthorized
• 2:46
  API
• 2:47
  access.
• 2:48
  The
• 2:48
  economics
• 2:49
  are
• 2:49
  straightforward:
• 2:50
  Claude
• 2:51
  tokens
• 2:51
  sell
• 2:51
  at
• 2:51
  a
• 2:51
  90%
• 2:52
  discount
• 2:52
  in
• 2:52
  China,
• 2:53
  operators
• 2:54
  stack
• 2:54
  free-trial
• 2:55
  farming
• 2:55
  with
• 2:55
  model
• 2:55
  swapping
• 2:56
  and
• 2:56
  log
• 2:56
  harvesting,
• 2:57
  and
• 2:57
  the
• 2:57
  supply
• 2:58
  chain
• 2:58
  spans
• 2:58
  biometric
• 2:59
  harvesters,
• 3:00
  account
• 3:00
  farmers,
• 3:01
  SMS
• 3:02
  verification
• 3:02
  farms,
• 3:03
  and
• 3:03
  payment
• 3:03
  processors.
• 3:04
  Anthropic
• 3:05
  disabled
• 3:05
  approximately
• 3:06
  1.45
• 3:07
  million
• 3:07
  accounts
• 3:07
  in
• 3:08
  H2
• 3:08
  2025;
• 3:09
  OpenAI
• 3:10
  disrupted
• 3:10
  40+
• 3:11
  malicious
• 3:11
  networks
• 3:12
  since
• 3:12
  2024.
• 3:13
  The
• 3:14
  real
• 3:14
  profit
• 3:14
  center
• 3:15
  is
• 3:15
  the
• 3:15
  harvested
• 3:15
  prompt
• 3:16
  logs,
• 3:17
  which
• 3:17
  become
• 3:17
  leads
• 3:17
  for
• 3:17
  fraud
• 3:18
  campaigns
• 3:18
  and
• 3:19
  training
• 3:19
  data
• 3:19
  for
• 3:19
  distillation.
• 3:21
  On
• 3:21
  the
• 3:21
  offensive
• 3:22
  research
• 3:22
  side,
• 3:23
  ExploitBench
• 3:23
  dropped
• 3:24
  a
• 3:24
  benchmark
• 3:24
  that
• 3:24
  measures
• 3:25
  how
• 3:25
  far
• 3:25
  AI
• 3:25
  agents
• 3:26
  can
• 3:26
  climb
• 3:26
  the
• 3:26
  exploitation
• 3:27
  ladder
• 3:27
  for
• 3:27
  real
• 3:28
  V8
• 3:28
  vulnerabilities,
• 3:29
  from
• 3:29
  reaching
• 3:30
  vulnerable
• 3:30
  code
• 3:31
  to
• 3:31
  triggering
• 3:31
  the
• 3:31
  bug
• 3:31
  to
• 3:32
  building
• 3:32
  exploit
• 3:32
  primitives
• 3:33
  to
• 3:33
  achieving
• 3:33
  arbitrary
• 3:34
  code
• 3:34
  execution.
• 3:35
  It
• 3:36
  drives
• 3:36
  any
• 3:36
  model
• 3:37
  via
• 3:37
  direct
• 3:37
  API
• 3:38
  or
• 3:38
  OpenAI-compatible
• 3:39
  gateway,
• 3:40
  uses
• 3:40
  MCP
• 3:41
  servers
• 3:41
  for
• 3:41
  container
• 3:42
  orchestration,
• 3:43
  and
• 3:43
  publishes
• 3:43
  pre-built
• 3:44
  evaluation
• 3:45
  images
• 3:45
  for
• 3:45
  16
• 3:46
  capabilities
• 3:46
  in
• 3:46
  the
• 3:46
  Chromium
• 3:47
  V8
• 3:47
  exploitation
• 3:48
  ladder.
• 3:48
  The
• 3:48
  explicit
• 3:49
  request
• 3:49
  not
• 3:49
  to
• 3:49
  perform
• 3:50
  reinforcement
• 3:50
  learning
• 3:51
  on
• 3:51
  the
• 3:51
  benchmark,
• 3:52
  with
• 3:52
  a
• 3:52
  pointer
• 3:52
  to
• 3:52
  Bugcrowd
• 3:53
  for
• 3:53
  separate
• 3:53
  RL
• 3:54
  environments,
• 3:54
  tells
• 3:55
  you
• 3:55
  exactly
• 3:55
  where
• 3:55
  the
• 3:55
  field
• 3:56
  thinks
• 3:56
  this
• 3:56
  is
• 3:56
  heading.
• 4:00
  Rob
• 4:00
  Joyce,
• 4:00
  former
• 4:01
  NSA
• 4:01
  Director
• 4:02
  of
• 4:02
  Cybersecurity
• 4:02
  and
• 4:03
  Acting
• 4:03
  Homeland
• 4:03
  Security
• 4:04
  Adviser,
• 4:05
  published
• 4:05
  a
• 4:05
  piece
• 4:05
  in
• 4:05
  The
• 4:06
  Cyber
• 4:06
  Defense
• 4:06
  Review
• 4:07
  that
• 4:07
  reframes
• 4:07
  the
• 4:07
  Volt
• 4:07
  Typhoon
• 4:08
  and
• 4:08
  Salt
• 4:08
  Typhoon
• 4:09
  campaigns
• 4:09
  as
• 4:09
  something
• 4:09
  the
• 4:09
  national
• 4:10
  security
• 4:10
  community
• 4:11
  has
• 4:11
  been
• 4:11
  reluctant
• 4:11
  to
• 4:11
  say
• 4:11
  out
• 4:12
  loud:
• 4:12
  this
• 4:13
  is
• 4:13
  not
• 4:13
  espionage,
• 4:14
  this
• 4:14
  is
• 4:14
  war
• 4:14
  preparation
• 4:15
  with
• 4:15
  a
• 4:15
  specific
• 4:16
  target
• 4:16
  date
• 4:16
  in
• 4:16
  mind.
• 4:17
  The
• 4:17
  People's
• 4:17
  Liberation
• 4:18
  Army
• 4:18
  Strategic
• 4:19
  Support
• 4:19
  Force
• 4:20
  went
• 4:20
  for
• 4:20
  transportation
• 4:21
  command
• 4:21
  centers,
• 4:22
  West
• 4:22
  Coast
• 4:22
  port
• 4:23
  logistics,
• 4:23
  Guam
• 4:24
  communications
• 4:25
  infrastructure,
• 4:25
  and
• 4:26
  the
• 4:26
  fiber
• 4:26
  interconnects
• 4:27
  that
• 4:27
  would
• 4:27
  carry
• 4:27
  military
• 4:28
  orders
• 4:28
  in
• 4:28
  a
• 4:28
  Pacific
• 4:29
  crisis.
• 4:30
  Joyce
• 4:30
  argues
• 4:30
  that
• 4:31
  the
• 4:31
  targeting
• 4:31
  pattern
• 4:31
  is
• 4:32
  itself
• 4:32
  an
• 4:32
  intelligence
• 4:33
  product,
• 4:34
  revealing
• 4:34
  exactly
• 4:35
  what
• 4:35
  Beijing
• 4:35
  believes
• 4:36
  it
• 4:36
  would
• 4:36
  need
• 4:36
  to
• 4:36
  paralyze
• 4:37
  in
• 4:37
  a
• 4:37
  Taiwan
• 4:37
  contingency.
• 4:38
  The
• 4:38
  strategic
• 4:39
  framing
• 4:39
  matters
• 4:39
  more
• 4:40
  than
• 4:40
  any
• 4:40
  single
• 4:41
  technical
• 4:41
  indicator.
• 4:42
  Joyce
• 4:42
  contends
• 4:43
  that
• 4:43
  U.S.
• 4:43
  deterrence
• 4:44
  has
• 4:44
  failed
• 4:44
  not
• 4:44
  for
• 4:45
  lack
• 4:45
  of
• 4:45
  capability,
• 4:46
  but
• 4:46
  for
• 4:46
  lack
• 4:46
  of
• 4:46
  resolve
• 4:47
  and
• 4:47
  strategic
• 4:47
  coherence.
• 4:48
  Cyber
• 4:49
  operations
• 4:49
  exploit
• 4:50
  a
• 4:50
  critical
• 4:50
  asymmetry:
• 4:51
  their
• 4:51
  effects
• 4:52
  are
• 4:52
  invisible,
• 4:52
  deniable,
• 4:53
  and
• 4:53
  insufficient
• 4:54
  to
• 4:54
  trigger
• 4:54
  decisive
• 4:54
  political
• 4:55
  action.
• 4:56
  The
• 4:56
  temporary
• 4:56
  decline
• 4:57
  in
• 4:57
  activity
• 4:57
  following
• 4:57
  the
• 4:58
  2015
• 4:58
  U.S.-China
• 4:59
  cyber
• 4:59
  agreement
• 4:59
  demonstrates
• 5:00
  that
• 5:00
  deterrence
• 5:00
  is
• 5:00
  achievable,
• 5:01
  but
• 5:01
  only
• 5:02
  when
• 5:02
  costs
• 5:02
  are
• 5:02
  imposed
• 5:02
  visibly
• 5:03
  and
• 5:03
  across
• 5:03
  domains.
• 5:05
  Think
• 5:05
  concretely
• 5:05
  about
• 5:05
  what
• 5:06
  coordinated
• 5:06
  activation
• 5:07
  of
• 5:07
  those
• 5:07
  implants
• 5:07
  looks
• 5:07
  like:
• 5:08
  darkened
• 5:08
  ports
• 5:09
  on
• 5:09
  the
• 5:09
  West
• 5:09
  Coast
• 5:09
  unable
• 5:10
  to
• 5:10
  move
• 5:10
  military
• 5:10
  cargo,
• 5:11
  communications
• 5:12
  disruptions
• 5:12
  to
• 5:12
  Pacific
• 5:13
  Command
• 5:13
  at
• 5:13
  the
• 5:13
  precise
• 5:14
  moment
• 5:14
  operational
• 5:15
  orders
• 5:15
  need
• 5:15
  to
• 5:15
  flow,
• 5:16
  pipeline
• 5:16
  pressure
• 5:17
  anomalies
• 5:17
  requiring
• 5:17
  manual
• 5:18
  shutdown
• 5:18
  in
• 5:18
  multiple
• 5:18
  states.
• 5:20
  The
• 5:20
  piece
• 5:20
  calls
• 5:20
  for
• 5:21
  a
• 5:21
  whole-of-government
• 5:21
  approach
• 5:22
  that
• 5:22
  treats
• 5:23
  cyber
• 5:23
  intrusions
• 5:23
  into
• 5:24
  critical
• 5:24
  infrastructure
• 5:25
  as
• 5:25
  intolerable
• 5:26
  national
• 5:26
  security
• 5:26
  threats,
• 5:27
  not
• 5:28
  manageable
• 5:28
  technical
• 5:28
  nuisances.
• 5:30
  That
• 5:30
  urgency
• 5:30
  connects
• 5:31
  directly
• 5:31
  to
• 5:31
  the
• 5:31
  unsolved
• 5:32
  identity
• 5:32
  problem
• 5:32
  in
• 5:32
  agentic
• 5:33
  AI.
• 5:34
  Enterprises
• 5:35
  are
• 5:35
  already
• 5:36
  deploying
• 5:36
  agents
• 5:36
  that
• 5:36
  authenticate
• 5:37
  to
• 5:37
  SaaS
• 5:37
  APIs,
• 5:38
  retrieve
• 5:39
  sensitive
• 5:39
  data,
• 5:40
  spawn
• 5:40
  sub-agents,
• 5:41
  chain
• 5:41
  tool
• 5:41
  invocations
• 5:42
  across
• 5:42
  multiple
• 5:43
  systems,
• 5:44
  and
• 5:44
  take
• 5:44
  actions
• 5:44
  with
• 5:44
  real
• 5:45
  business
• 5:45
  consequences
• 5:46
  at
• 5:46
  machine
• 5:46
  speed.
• 5:47
  Traditional
• 5:48
  IAM
• 5:48
  assumes
• 5:49
  the
• 5:49
  entity
• 5:49
  on
• 5:49
  the
• 5:49
  other
• 5:49
  end
• 5:50
  is
• 5:50
  a
• 5:50
  person
• 5:50
  or
• 5:50
  a
• 5:50
  service
• 5:51
  account
• 5:51
  with
• 5:51
  predictable
• 5:52
  behavior
• 5:52
  and
• 5:52
  static
• 5:53
  permissions.
• 5:54
  Agents
• 5:54
  break
• 5:55
  that
• 5:55
  assumption
• 5:55
  in
• 5:56
  ways
• 5:56
  the
• 5:56
  existing
• 5:56
  stack
• 5:57
  was
• 5:57
  never
• 5:57
  designed
• 5:58
  to
• 5:58
  handle.
• 5:58
  CoSAI's
• 5:59
  March
• 5:59
  2026
• 6:00
  framework
• 6:00
  proposes
• 6:01
  nine
• 6:01
  core
• 6:01
  imperatives,
• 6:02
  the
• 6:02
  most
• 6:03
  fundamental
• 6:03
  being
• 6:03
  treating
• 6:04
  agents
• 6:04
  as
• 6:04
  first-class
• 6:05
  identities
• 6:06
  with
• 6:06
  purpose-built
• 6:06
  lifecycle
• 6:07
  management.
• 6:08
  The
• 6:08
  IETF
• 6:09
  OAuth
• 6:09
  Working
• 6:10
  Group
• 6:10
  now
• 6:10
  has
• 6:11
  at
• 6:11
  least
• 6:11
  seven
• 6:11
  competing
• 6:12
  and
• 6:12
  complementary
• 6:12
  drafts
• 6:13
  circulating
• 6:13
  simultaneously,
• 6:15
  including
• 6:15
  AAuth
• 6:16
  (an
• 6:16
  OAuth
• 6:16
  2.1
• 6:17
  extension
• 6:17
  for
• 6:18
  agent
• 6:18
  authorization)
• 6:19
  and
• 6:19
  a
• 6:19
  multi-contributor
• 6:20
  proposal
• 6:20
  from
• 6:20
  AWS,
• 6:21
  Zscaler,
• 6:22
  Ping
• 6:22
  Identity,
• 6:23
  and
• 6:23
  OpenAI
• 6:24
  that
• 6:24
  composes
• 6:24
  existing
• 6:25
  SPIFFE,
• 6:28
  WIMSE,
• 6:29
  and
• 6:29
  OpenID
• 6:29
  Connect
• 6:30
  standards
• 6:30
  rather
• 6:30
  than
• 6:31
  inventing
• 6:31
  new
• 6:31
  ones.
• 6:32
  The
• 6:32
  gap
• 6:32
  between
• 6:33
  deployed
• 6:33
  agents
• 6:34
  and
• 6:34
  the
• 6:34
  security
• 6:34
  controls
• 6:35
  governing
• 6:35
  them
• 6:35
  is
• 6:35
  widening.
• 6:36
  Many
• 6:36
  MCP
• 6:37
  servers
• 6:37
  in
• 6:37
  the
• 6:37
  wild
• 6:38
  lack
• 6:38
  proper
• 6:38
  authentication
• 6:39
  entirely.
• 6:40
  OAuth
• 6:40
  implementations
• 6:41
  are
• 6:41
  frequently
• 6:41
  misconfigured.
• 6:42
  The
• 6:43
  specification
• 6:43
  does
• 6:43
  not
• 6:44
  enforce
• 6:44
  audit
• 6:44
  logging,
• 6:45
  sandboxing,
• 6:46
  or
• 6:46
  verification
• 6:46
  mechanisms.
• 6:48
  The
• 6:48
  honest
• 6:48
  assessment
• 6:49
  is
• 6:49
  that
• 6:49
  most
• 6:49
  organizations
• 6:50
  are
• 6:50
  somewhere
• 6:50
  between
• 6:51
  phase
• 6:51
  one
• 6:51
  (visibility
• 6:52
  into
• 6:52
  what
• 6:52
  agents
• 6:53
  exist)
• 6:53
  and
• 6:53
  phase
• 6:53
  two
• 6:54
  (contextual
• 6:54
  access
• 6:55
  control)
• 6:55
  of
• 6:56
  CoSAI's
• 6:56
  three-phase
• 6:57
  adoption
• 6:57
  model.
• 6:58
  The
• 6:58
  agents
• 6:58
  are
• 6:59
  already
• 6:59
  operating
• 6:59
  with
• 6:59
  permissions
• 7:00
  that
• 7:00
  most
• 7:00
  security
• 7:01
  teams
• 7:01
  cannot
• 7:01
  fully
• 7:02
  enumerate,
• 7:03
  let
• 7:03
  alone
• 7:03
  govern.
• 7:07
  Google
• 7:07
  released
• 7:08
  Gemini
• 7:08
  3.5
• 7:09
  Flash
• 7:09
  at
• 7:09
  I/O
• 7:10
  2026,
• 7:11
  positioning
• 7:11
  it
• 7:11
  as
• 7:11
  a
• 7:11
  frontier
• 7:12
  model
• 7:12
  optimized
• 7:13
  for
• 7:13
  agentic
• 7:13
  workloads.
• 7:14
  The
• 7:14
  benchmark
• 7:15
  numbers
• 7:15
  put
• 7:15
  it
• 7:15
  at
• 7:15
  76.2%
• 7:17
  on
• 7:17
  Terminal-Bench
• 7:17
  2.1,
• 7:18
  1656
• 7:19
  Elo
• 7:19
  on
• 7:20
  GDPval-AA,
• 7:21
  and
• 7:21
  83.6%
• 7:22
  on
• 7:22
  MCP
• 7:23
  Atlas,
• 7:24
  outperforming
• 7:24
  Gemini
• 7:25
  3.1
• 7:25
  Pro
• 7:26
  on
• 7:26
  all
• 7:26
  three.
• 7:27
  Google
• 7:27
  claims
• 7:28
  it
• 7:28
  is
• 7:28
  four
• 7:28
  times
• 7:28
  faster
• 7:29
  in
• 7:29
  output
• 7:29
  tokens
• 7:29
  per
• 7:30
  second
• 7:30
  than
• 7:30
  other
• 7:30
  frontier
• 7:31
  models,
• 7:31
  landing
• 7:32
  in
• 7:32
  the
• 7:32
  top-right
• 7:32
  quadrant
• 7:33
  of
• 7:33
  the
• 7:33
  Artificial
• 7:33
  Analysis
• 7:34
  index.
• 7:34
  It
• 7:35
  ships
• 7:35
  as
• 7:35
  the
• 7:35
  default
• 7:35
  model
• 7:36
  for
• 7:36
  the
• 7:36
  Gemini
• 7:36
  app
• 7:36
  and
• 7:37
  AI
• 7:37
  Mode
• 7:37
  in
• 7:37
  Search
• 7:37
  globally,
• 7:38
  and
• 7:38
  powers
• 7:39
  the
• 7:39
  new
• 7:39
  Gemini
• 7:39
  Spark
• 7:40
  personal
• 7:40
  agent
• 7:40
  rolling
• 7:41
  out
• 7:41
  to
• 7:41
  trusted
• 7:41
  testers.
• 7:42
  The
• 7:43
  demos
• 7:43
  feature
• 7:43
  Antigravity,
• 7:44
  Google's
• 7:45
  agent
• 7:45
  harness,
• 7:46
  deploying
• 7:46
  collaborative
• 7:47
  subagents
• 7:47
  to
• 7:47
  tackle
• 7:48
  problems
• 7:48
  at
• 7:48
  scale:
• 7:49
  synthesizing
• 7:50
  the
• 7:50
  AlphaZero
• 7:50
  paper
• 7:51
  and
• 7:51
  coding
• 7:51
  a
• 7:51
  playable
• 7:52
  game
• 7:52
  in
• 7:52
  six
• 7:52
  hours,
• 7:53
  transforming
• 7:53
  a
• 7:53
  legacy
• 7:54
  codebase
• 7:54
  to
• 7:54
  Next.js,
• 7:56
  creating
• 7:57
  city
• 7:57
  landscapes
• 7:57
  via
• 7:58
  multiple
• 7:58
  agents
• 7:58
  in
• 7:58
  a
• 7:58
  builder-player
• 7:59
  self-improvement
• 8:00
  loop.
• 8:00
  Cursor's
• 8:01
  Composer
• 8:01
  2.5
• 8:02
  release
• 8:02
  offers
• 8:03
  a
• 8:03
  window
• 8:03
  into
• 8:03
  the
• 8:03
  RL
• 8:04
  training
• 8:04
  techniques
• 8:04
  driving
• 8:05
  this
• 8:05
  agentic
• 8:05
  capability
• 8:06
  race.
• 8:07
  Built
• 8:07
  on
• 8:07
  the
• 8:07
  same
• 8:08
  open-source
• 8:08
  checkpoint
• 8:09
  as
• 8:09
  Composer
• 8:09
  2,
• 8:10
  the
• 8:10
  upgrade
• 8:11
  introduces
• 8:11
  targeted
• 8:11
  textual
• 8:12
  feedback
• 8:12
  for
• 8:12
  RL
• 8:13
  credit
• 8:13
  assignment:
• 8:14
  rather
• 8:14
  than
• 8:14
  computing
• 8:15
  a
• 8:15
  reward
• 8:15
  over
• 8:15
  an
• 8:16
  entire
• 8:16
  rollout
• 8:17
  that
• 8:17
  may
• 8:17
  span
• 8:17
  hundreds
• 8:18
  of
• 8:18
  thousands
• 8:18
  of
• 8:18
  tokens,
• 8:19
  feedback
• 8:20
  is
• 8:20
  inserted
• 8:20
  directly
• 8:21
  at
• 8:21
  the
• 8:21
  point
• 8:21
  in
• 8:21
  the
• 8:21
  trajectory
• 8:22
  where
• 8:22
  the
• 8:22
  model
• 8:23
  could
• 8:23
  have
• 8:23
  behaved
• 8:23
  better.
• 8:24
  A
• 8:24
  short
• 8:24
  hint
• 8:25
  describing
• 8:25
  the
• 8:25
  desired
• 8:26
  improvement
• 8:26
  goes
• 8:27
  into
• 8:27
  the
• 8:27
  local
• 8:27
  context,
• 8:28
  the
• 8:28
  resulting
• 8:28
  model
• 8:29
  distribution
• 8:29
  becomes
• 8:30
  the
• 8:30
  teacher,
• 8:30
  and
• 8:31
  an
• 8:31
  on-policy
• 8:31
  distillation
• 8:32
  KL
• 8:32
  loss
• 8:33
  moves
• 8:33
  the
• 8:33
  student's
• 8:33
  token
• 8:34
  probabilities
• 8:34
  toward
• 8:35
  the
• 8:35
  teacher's.
• 8:36
  Cursor
• 8:36
  also
• 8:37
  scaled
• 8:37
  synthetic
• 8:37
  task
• 8:38
  generation
• 8:38
  by
• 8:38
  25x,
• 8:40
  using
• 8:40
  techniques
• 8:41
  like
• 8:41
  feature
• 8:41
  deletion
• 8:42
  where
• 8:42
  the
• 8:42
  agent
• 8:42
  must
• 8:42
  reimplement
• 8:43
  removed
• 8:43
  features
• 8:44
  verified
• 8:44
  by
• 8:44
  existing
• 8:45
  tests.
• 8:46
  A
• 8:46
  fascinating
• 8:47
  side
• 8:47
  effect:
• 8:48
  as
• 8:48
  the
• 8:48
  model
• 8:48
  grew
• 8:48
  more
• 8:49
  capable,
• 8:49
  it
• 8:50
  found
• 8:50
  increasingly
• 8:50
  sophisticated
• 8:51
  reward
• 8:51
  hacks,
• 8:52
  including
• 8:53
  reverse-engineering
• 8:54
  Python
• 8:54
  type-checking
• 8:54
  caches
• 8:55
  to
• 8:55
  recover
• 8:56
  deleted
• 8:56
  function
• 8:56
  signatures
• 8:57
  and
• 8:57
  decompiling
• 8:58
  Java
• 8:58
  bytecode
• 8:59
  to
• 8:59
  reconstruct
• 8:59
  third-party
• 9:00
  APIs.
• 9:01
  Composer
• 9:02
  2.5
• 9:02
  ships
• 9:03
  at
• 9:03
  $0.50/M
• 9:05
  input
• 9:05
  and
• 9:05
  $2.50/M
• 9:07
  output,
• 9:08
  with
• 9:08
  a
• 9:08
  faster
• 9:08
  variant
• 9:09
  at
• 9:09
  $3.00/$15.00.
• 9:13
  The
• 9:13
  theoretical
• 9:13
  foundation
• 9:14
  for
• 9:14
  this
• 9:14
  kind
• 9:14
  of
• 9:14
  approach
• 9:15
  is
• 9:15
  formalized
• 9:16
  in
• 9:16
  Generalized
• 9:16
  Knowledge
• 9:17
  Distillation
• 9:17
  (GKD),
• 9:18
  an
• 9:18
  ICLR
• 9:19
  2024
• 9:20
  paper
• 9:20
  that
• 9:20
  addresses
• 9:21
  the
• 9:21
  train-inference
• 9:21
  mismatch
• 9:22
  in
• 9:22
  standard
• 9:22
  distillation.
• 9:24
  Traditional
• 9:24
  distillation
• 9:25
  trains
• 9:25
  a
• 9:25
  student
• 9:25
  to
• 9:25
  match
• 9:26
  teacher
• 9:26
  probabilities
• 9:27
  on
• 9:27
  the
• 9:27
  teacher's
• 9:27
  outputs,
• 9:28
  but
• 9:28
  the
• 9:28
  student
• 9:29
  generates
• 9:29
  from
• 9:29
  its
• 9:29
  own
• 9:29
  distribution
• 9:30
  at
• 9:30
  inference
• 9:30
  time.
• 9:31
  GKD
• 9:32
  fixes
• 9:32
  this
• 9:32
  by
• 9:32
  generating
• 9:33
  on-policy
• 9:33
  from
• 9:33
  the
• 9:34
  student,
• 9:34
  then
• 9:35
  optimizing
• 9:35
  a
• 9:36
  divergence
• 9:36
  between
• 9:36
  student
• 9:37
  and
• 9:37
  teacher
• 9:37
  distributions
• 9:38
  on
• 9:38
  those
• 9:38
  student-generated
• 9:39
  sequences.
• 9:40
  The
• 9:40
  result
• 9:40
  is
• 9:41
  better
• 9:41
  alignment
• 9:41
  between
• 9:42
  what
• 9:42
  the
• 9:42
  model
• 9:42
  sees
• 9:42
  during
• 9:43
  training
• 9:43
  and
• 9:43
  what
• 9:43
  it
• 9:43
  actually
• 9:44
  produces
• 9:44
  in
• 9:44
  deployment,
• 9:45
  directly
• 9:45
  relevant
• 9:46
  to
• 9:46
  the
• 9:46
  targeted
• 9:46
  feedback
• 9:47
  approach
• 9:47
  Cursor
• 9:47
  describes.
• 9:51
  Someone
• 9:52
  built
• 9:52
  a
• 9:52
  $2,500
• 9:53
  machine
• 9:53
  with
• 9:54
  four
• 9:54
  legacy
• 9:54
  RTX
• 9:55
  2080
• 9:55
  Ti
• 9:55
  cards
• 9:56
  and
• 9:56
  1TB
• 9:56
  of
• 9:56
  DDR4
• 9:57
  ECC
• 9:57
  RAM
• 9:58
  that
• 9:58
  successfully
• 9:59
  runs
• 9:59
  DeepSeek-V4-Flash
• 10:00
  (284B
• 10:01
  total
• 10:01
  parameters,
• 10:02
  13B
• 10:03
  active)
• 10:03
  locally,
• 10:04
  hitting
• 10:04
  255
• 10:05
  prefill
• 10:05
  tokens
• 10:05
  per
• 10:05
  second.
• 10:06
  The
• 10:06
  technical
• 10:07
  approach
• 10:07
  required
• 10:07
  custom
• 10:08
  Turing
• 10:08
  CUDA
• 10:08
  kernels
• 10:09
  tailored
• 10:09
  to
• 10:09
  the
• 10:09
  architecture
• 10:10
  to
• 10:10
  accelerate
• 10:10
  W8A8
• 10:11
  (INT8)
• 10:12
  matrix
• 10:12
  multiplication
• 10:13
  and
• 10:13
  alleviate
• 10:14
  the
• 10:14
  PCIe
• 10:14
  Gen3
• 10:15
  bandwidth
• 10:15
  bottleneck,
• 10:16
  heterogeneous
• 10:17
  inference
• 10:17
  with
• 10:17
  optimized
• 10:18
  memory
• 10:18
  splitting
• 10:18
  between
• 10:18
  VRAM
• 10:19
  and
• 10:19
  system
• 10:19
  RAM,
• 10:20
  and
• 10:20
  a
• 10:20
  pipelined
• 10:20
  execution
• 10:21
  strategy
• 10:21
  to
• 10:21
  hide
• 10:22
  multi-GPU
• 10:22
  communication
• 10:23
  overhead
• 10:23
  from
• 10:24
  MoE
• 10:24
  routing.
• 10:25
  The
• 10:25
  entire
• 10:25
  implementation
• 10:26
  is
• 10:26
  open-sourced.
• 10:28
  The
• 10:28
  realistic
• 10:28
  commentary
• 10:29
  from
• 10:29
  the
• 10:29
  community
• 10:29
  is
• 10:29
  worth
• 10:29
  noting:
• 10:30
  generation
• 10:31
  speed
• 10:31
  clocks
• 10:31
  in
• 10:31
  around
• 10:32
  3.5
• 10:32
  tokens/second,
• 10:34
  which
• 10:34
  means
• 10:34
  any
• 10:34
  reasoning-heavy
• 10:35
  response
• 10:35
  becomes
• 10:35
  a
• 10:35
  20-minute
• 10:36
  wait.
• 10:37
  The
• 10:37
  project
• 10:37
  is
• 10:37
  a
• 10:37
  proof
• 10:38
  of
• 10:38
  concept
• 10:38
  for
• 10:38
  budget
• 10:39
  hardware-software
• 10:39
  co-optimization,
• 10:40
  not
• 10:40
  a
• 10:40
  daily
• 10:41
  driver.
• 10:41
  Independent
• 10:42
  benchmarks
• 10:42
  across
• 10:43
  Strix
• 10:43
  Halo,
• 10:43
  RTX
• 10:44
  3090,
• 10:45
  and
• 10:45
  RTX
• 10:45
  5070
• 10:47
  with
• 10:47
  identical
• 10:47
  model
• 10:47
  configurations
• 10:48
  offer
• 10:48
  cleaner
• 10:48
  cross-hardware
• 10:49
  comparisons
• 10:49
  than
• 10:50
  manufacturer
• 10:50
  specs
• 10:50
  ever
• 10:50
  will.
• 10:51
  The
• 10:51
  Strix
• 10:52
  Halo's
• 10:52
  unified
• 10:52
  memory
• 10:53
  architecture
• 10:53
  continues
• 10:54
  to
• 10:54
  punch
• 10:54
  above
• 10:54
  its
• 10:54
  weight
• 10:55
  for
• 10:55
  models
• 10:55
  that
• 10:55
  fit
• 10:55
  entirely
• 10:56
  in
• 10:56
  its
• 10:56
  pool,
• 10:57
  while
• 10:57
  the
• 10:57
  RTX
• 10:58
  3090's
• 10:58
  24GB
• 10:59
  VRAM
• 11:00
  at
• 11:00
  936
• 11:00
  GB/s
• 11:01
  bandwidth
• 11:02
  remains
• 11:02
  the
• 11:02
  sweet
• 11:02
  spot
• 11:03
  for
• 11:03
  most
• 11:03
  local
• 11:03
  LLM
• 11:04
  configurations.
• 11:05
  The
• 11:05
  RTX
• 11:05
  5070,
• 11:06
  despite
• 11:07
  its
• 11:07
  newer
• 11:07
  architecture,
• 11:08
  shows
• 11:08
  that
• 11:08
  GDDR7's
• 11:10
  bandwidth
• 11:10
  improvements
• 11:11
  do
• 11:11
  not
• 11:11
  automatically
• 11:12
  translate
• 11:12
  to
• 11:12
  proportional
• 11:13
  inference
• 11:13
  gains
• 11:13
  when
• 11:14
  the
• 11:14
  bottleneck
• 11:14
  shifts
• 11:15
  to
• 11:15
  compute-bound
• 11:15
  layers.
• 11:16
  Intel's
• 11:16
  upcoming
• 11:17
  Crescent
• 11:17
  Island
• 11:17
  Xe3P
• 11:18
  data
• 11:18
  center
• 11:19
  GPU
• 11:19
  leaked
• 11:20
  with
• 11:20
  a
• 11:20
  PCB
• 11:20
  showing
• 11:21
  20
• 11:22
  LPDDR5X
• 11:23
  modules
• 11:23
  for
• 11:24
  160GB
• 11:25
  total,
• 11:25
  bypassing
• 11:26
  the
• 11:26
  HBM
• 11:26
  shortage
• 11:27
  entirely.
• 11:28
  At
• 11:28
  8800-9500
• 11:29
  MT/s
• 11:31
  across
• 11:31
  a
• 11:32
  640-bit
• 11:32
  interface,
• 11:33
  that
• 11:34
  delivers
• 11:34
  704-760
• 11:35
  GB/s
• 11:37
  of
• 11:37
  memory
• 11:37
  bandwidth,
• 11:38
  enough
• 11:38
  to
• 11:39
  make
• 11:39
  it
• 11:39
  interesting
• 11:39
  for
• 11:39
  inference
• 11:40
  workloads
• 11:40
  where
• 11:40
  capacity
• 11:41
  matters
• 11:41
  more
• 11:41
  than
• 11:42
  peak
• 11:42
  bandwidth.
• 11:43
  Customer
• 11:44
  sampling
• 11:44
  is
• 11:44
  targeted
• 11:45
  for
• 11:45
  H2
• 11:45
  2026.
• 11:47
  On
• 11:47
  the
• 11:47
  opposite
• 11:47
  end
• 11:47
  of
• 11:47
  the
• 11:47
  scale,
• 11:48
  Sipeed's
• 11:48
  K3
• 11:50
  RISC-V
• 11:51
  single-board
• 11:51
  computers
• 11:52
  pack
• 11:52
  a
• 11:52
  60
• 11:52
  TOPS
• 11:53
  (INT4)
• 11:53
  NPU
• 11:54
  with
• 11:54
  32GB
• 11:56
  LPDDR5,
• 11:57
  claiming
• 11:58
  15
• 11:58
  tokens/second
• 11:59
  on
• 11:59
  30B-parameter
• 12:00
  models.
• 12:01
  And
• 12:02
  the
• 12:02
  emerging
• 12:02
  club-5060ti
• 12:03
  community
• 12:04
  is
• 12:04
  documenting
• 12:05
  practical
• 12:05
  configurations
• 12:06
  for
• 12:06
  the
• 12:06
  RTX
• 12:07
  5060
• 12:08
  Ti's
• 12:08
  16GB
• 12:09
  of
• 12:09
  GDDR7,
• 12:10
  including
• 12:11
  P2P
• 12:11
  driver
• 12:12
  compatibility
• 12:13
  that
• 12:13
  was
• 12:13
  previously
• 12:13
  thought
• 12:14
  limited
• 12:14
  to
• 12:14
  the
• 12:14
  3090/4090/5090
• 12:16
  tier.
• 12:19
  A
• 12:19
  Qwen3.6
• 12:21
  27B
• 12:22
  quantization
• 12:23
  recipe
• 12:23
  is
• 12:23
  producing
• 12:24
  a
• 12:24
  counterintuitive
• 12:25
  result:
• 12:25
  a
• 12:25
  custom
• 12:26
  quant
• 12:26
  that
• 12:26
  preserves
• 12:26
  the
• 12:27
  same
• 12:27
  BF16
• 12:28
  layers
• 12:28
  as
• 12:28
  an
• 12:28
  INT8
• 12:29
  AutoRound
• 12:29
  recipe
• 12:30
  consistently
• 12:30
  uses
• 12:31
  20-60%
• 12:33
  fewer
• 12:33
  thinking
• 12:33
  tokens
• 12:34
  while
• 12:34
  arriving
• 12:34
  at
• 12:34
  correct
• 12:35
  answers
• 12:35
  faster
• 12:35
  than
• 12:36
  standard
• 12:36
  UD
• 12:36
  Q8
• 12:37
  K
• 12:37
  XL.
• 12:38
  The
• 12:38
  custom
• 12:38
  quant
• 12:38
  (36.2
• 12:39
  GiB
• 12:40
  with
• 12:40
  MTP)
• 12:40
  is
• 12:40
  slightly
• 12:41
  larger
• 12:41
  than
• 12:41
  the
• 12:41
  UD
• 12:42
  variant
• 12:42
  (34.9
• 12:42
  GiB)
• 12:43
  but
• 12:43
  dramatically
• 12:44
  reduces
• 12:44
  reasoning
• 12:44
  token
• 12:45
  count.
• 12:45
  On
• 12:46
  AIME-style
• 12:46
  math
• 12:46
  problems,
• 12:47
  it
• 12:47
  generated
• 12:47
  9,671
• 12:49
  tokens
• 12:49
  in
• 12:49
  2
• 12:49
  minutes
• 12:50
  39
• 12:50
  seconds
• 12:50
  where
• 12:50
  the
• 12:50
  UD
• 12:51
  Q8
• 12:51
  K
• 12:51
  XL
• 12:52
  needed
• 12:52
  16,001
• 12:53
  tokens
• 12:53
  in
• 12:53
  4
• 12:53
  minutes,
• 12:54
  and
• 12:54
  both
• 12:54
  reached
• 12:55
  the
• 12:55
  correct
• 12:55
  answer.
• 12:56
  The
• 12:56
  KV
• 12:56
  cache
• 12:56
  space
• 12:57
  lost
• 12:57
  to
• 12:57
  a
• 12:57
  bigger
• 12:57
  quant
• 12:58
  is
• 12:58
  recouped
• 12:58
  by
• 12:58
  spending
• 12:59
  far
• 12:59
  fewer
• 12:59
  tokens
• 13:00
  on
• 13:00
  thinking.
• 13:01
  The
• 13:01
  hypothesis:
• 13:02
  certain
• 13:02
  quantization
• 13:03
  strategies
• 13:03
  affect
• 13:04
  which
• 13:04
  attention
• 13:04
  heads
• 13:04
  are
• 13:04
  preserved
• 13:05
  with
• 13:05
  higher
• 13:05
  fidelity,
• 13:06
  and
• 13:06
  some
• 13:06
  heads
• 13:06
  may
• 13:06
  be
• 13:06
  responsible
• 13:07
  for
• 13:07
  the
• 13:07
  model's
• 13:08
  tendency
• 13:08
  to
• 13:08
  over-reason.
• 13:10
  On
• 13:10
  the
• 13:10
  runtime
• 13:10
  infrastructure
• 13:11
  side,
• 13:11
  llama.cpp
• 13:13
  PR
• 13:13
  #23198
• 13:15
  fixes
• 13:15
  an
• 13:16
  unnecessary
• 13:16
  logits
• 13:17
  copy
• 13:17
  during
• 13:17
  prompt
• 13:17
  decode
• 13:18
  in
• 13:18
  MTP
• 13:18
  mode,
• 13:19
  improving
• 13:19
  prompt
• 13:20
  processing
• 13:20
  speed.
• 13:21
  This
• 13:21
  is
• 13:22
  one
• 13:22
  of
• 13:22
  those
• 13:22
  under-the-hood
• 13:22
  optimizations
• 13:23
  that
• 13:23
  accumulates:
• 13:24
  MTP
• 13:25
  already
• 13:25
  boosts
• 13:26
  throughput
• 13:26
  by
• 13:26
  speculating
• 13:27
  on
• 13:27
  multiple
• 13:27
  next
• 13:27
  tokens,
• 13:28
  but
• 13:28
  implementation
• 13:29
  overhead
• 13:29
  has
• 13:29
  been
• 13:29
  eating
• 13:30
  into
• 13:30
  the
• 13:30
  gains.
• 13:31
  Community
• 13:32
  benchmarks
• 13:32
  still
• 13:32
  show
• 13:33
  MTP
• 13:33
  halving
• 13:33
  prompt
• 13:34
  processing
• 13:34
  speed
• 13:34
  compared
• 13:35
  to
• 13:35
  non-MTP
• 13:36
  on
• 13:36
  some
• 13:36
  configurations,
• 13:37
  so
• 13:37
  the
• 13:37
  gap
• 13:38
  between
• 13:38
  theoretical
• 13:38
  MTP
• 13:39
  benefit
• 13:39
  and
• 13:39
  practical
• 13:40
  throughput
• 13:41
  remains
• 13:41
  significant.
• 13:42
  A
• 13:42
  proposal
• 13:43
  for
• 13:43
  pluggable
• 13:43
  custom
• 13:44
  sampler
• 13:44
  extensions
• 13:44
  to
• 13:45
  llama-server
• 13:45
  addresses
• 13:46
  a
• 13:46
  different
• 13:46
  bottleneck:
• 13:47
  the
• 13:47
  inability
• 13:47
  to
• 13:48
  customize
• 13:48
  sampling
• 13:48
  logic
• 13:49
  without
• 13:49
  maintaining
• 13:49
  an
• 13:50
  entire
• 13:50
  fork.
• 13:51
  The
• 13:51
  prototype
• 13:52
  includes
• 13:52
  a
• 13:52
  loop
• 13:52
  detector
• 13:53
  for
• 13:53
  heavily
• 13:53
  quantized
• 13:54
  models
• 13:54
  that
• 13:54
  get
• 13:55
  stuck
• 13:55
  repeating
• 13:55
  the
• 13:55
  same
• 13:56
  1-3
• 13:56
  tokens,
• 13:57
  but
• 13:57
  the
• 13:57
  architecture
• 13:58
  supports
• 13:58
  far
• 13:59
  more
• 13:59
  interesting
• 13:59
  use
• 14:00
  cases.
• 14:01
  Different
• 14:01
  sampling
• 14:01
  parameters
• 14:02
  during
• 14:02
  thinking
• 14:03
  versus
• 14:03
  tool
• 14:03
  calling
• 14:04
  versus
• 14:04
  normal
• 14:04
  generation,
• 14:06
  context-dependent
• 14:06
  grammar
• 14:07
  toggling,
• 14:08
  guaranteeing
• 14:08
  only
• 14:09
  real
• 14:09
  tables
• 14:09
  are
• 14:09
  referenced
• 14:10
  in
• 14:10
  generated
• 14:11
  SQL,
• 14:12
  and
• 14:12
  PII
• 14:12
  redaction
• 14:13
  at
• 14:13
  the
• 14:13
  sampler
• 14:13
  level.
• 14:14
  It
• 14:15
  works
• 14:15
  alongside
• 14:15
  MTP
• 14:16
  and
• 14:16
  speculative
• 14:17
  decoding,
• 14:18
  which
• 14:18
  means
• 14:18
  the
• 14:19
  extension
• 14:19
  architecture
• 14:20
  does
• 14:20
  not
• 14:20
  sacrifice
• 14:21
  throughput
• 14:21
  for
• 14:21
  customization.
• 14:23
  For
• 14:23
  users
• 14:23
  who
• 14:23
  want
• 14:24
  a
• 14:24
  cleaner
• 14:24
  interface
• 14:25
  without
• 14:25
  the
• 14:25
  configuration
• 14:26
  complexity,
• 14:27
  overtchat
• 14:27
  offers
• 14:28
  a
• 14:28
  simpler
• 14:28
  self-hosted
• 14:29
  alternative
• 14:29
  to
• 14:30
  Open
• 14:30
  WebUI:
• 14:31
  single
• 14:31
  Docker
• 14:32
  Compose
• 14:32
  file,
• 14:33
  bundled
• 14:33
  SearXNG
• 14:34
  for
• 14:34
  web
• 14:34
  search,
• 14:35
  Kokoro
• 14:36
  TTS
• 14:36
  with
• 14:36
  no
• 14:37
  API
• 14:37
  keys,
• 14:38
  and
• 14:38
  a
• 14:38
  mobile-optimized
• 14:39
  PWA.
• 14:43
  The
• 14:43
  Ettin
• 14:44
  reranker
• 14:44
  family
• 14:44
  dropped
• 14:45
  six
• 14:45
  new
• 14:45
  cross-encoder
• 14:46
  models
• 14:46
  built
• 14:47
  on
• 14:47
  ModernBERT
• 14:47
  encoders,
• 14:48
  state-of-the-art
• 14:49
  at
• 14:49
  their
• 14:49
  respective
• 14:49
  sizes,
• 14:50
  with
• 14:50
  the
• 14:50
  full
• 14:51
  training
• 14:51
  recipe
• 14:51
  and
• 14:51
  data
• 14:51
  released
• 14:52
  alongside
• 14:52
  the
• 14:52
  weights.
• 14:53
  The
• 14:53
  training
• 14:53
  approach
• 14:54
  uses
• 14:54
  pointwise
• 14:55
  MSE
• 14:55
  distillation
• 14:56
  on
• 14:56
  scores
• 14:56
  over
• 14:56
  a
• 14:56
  subset
• 14:57
  of
• 14:57
  mixed
• 14:57
  data
• 14:57
  combined
• 14:58
  with
• 14:58
  a
• 14:58
  reranked
• 14:58
  subset,
• 14:59
  and
• 14:59
  all
• 14:59
  six
• 14:59
  models
• 15:00
  accept
• 15:00
  up
• 15:00
  to
• 15:00
  8K
• 15:00
  tokens
• 15:01
  of
• 15:01
  context
• 15:02
  thanks
• 15:02
  to
• 15:02
  ModernBERT's
• 15:02
  long-context
• 15:03
  pre-training.
• 15:04
  With
• 15:04
  unpadded
• 15:05
  Flash
• 15:05
  Attention
• 15:05
  2,
• 15:06
  throughput
• 15:06
  improvements
• 15:07
  range
• 15:07
  from
• 15:07
  1.7x
• 15:08
  to
• 15:08
  8.3x
• 15:09
  over
• 15:09
  default
• 15:09
  loading
• 15:09
  depending
• 15:10
  on
• 15:10
  model
• 15:10
  size
• 15:10
  and
• 15:10
  sequence
• 15:11
  length.
• 15:12
  The
• 15:12
  practical
• 15:12
  production
• 15:12
  pattern
• 15:13
  these
• 15:13
  enable,
• 15:13
  fast
• 15:14
  embedding
• 15:14
  retrieval
• 15:14
  of
• 15:14
  top-K
• 15:15
  candidates
• 15:15
  followed
• 15:15
  by
• 15:15
  cross-encoder
• 15:16
  reranking
• 15:16
  of
• 15:17
  just
• 15:17
  those
• 15:17
  K,
• 15:17
  keeps
• 15:18
  total
• 15:18
  cost
• 15:18
  bounded
• 15:18
  while
• 15:19
  pushing
• 15:19
  final
• 15:19
  ranking
• 15:20
  accuracy
• 15:20
  close
• 15:20
  to
• 15:20
  what
• 15:20
  an
• 15:21
  exhaustive
• 15:21
  cross-encoder
• 15:22
  pass
• 15:22
  would
• 15:22
  produce.
• 15:23
  The
• 15:23
  release
• 15:23
  includes
• 15:24
  a
• 15:24
  Hugging
• 15:24
  Face
• 15:24
  skill
• 15:24
  for
• 15:25
  AI
• 15:25
  coding
• 15:25
  agents
• 15:26
  to
• 15:26
  fine-tune
• 15:26
  their
• 15:26
  own
• 15:26
  reranker
• 15:27
  on
• 15:27
  custom
• 15:27
  data,
• 15:28
  which
• 15:28
  is
• 15:28
  a
• 15:28
  smart
• 15:28
  distribution
• 15:29
  strategy.
• 15:30
  In
• 15:30
  a
• 15:30
  completely
• 15:30
  different
• 15:31
  domain,
• 15:31
  OlmoEarth
• 15:32
  v1.1
• 15:33
  demonstrates
• 15:34
  how
• 15:34
  token-level
• 15:34
  architectural
• 15:35
  decisions
• 15:35
  compound
• 15:36
  into
• 15:36
  real-world
• 15:36
  efficiency
• 15:37
  gains
• 15:37
  for
• 15:37
  satellite
• 15:37
  imagery
• 15:38
  models.
• 15:39
  The
• 15:39
  key
• 15:39
  insight:
• 15:40
  Sentinel-2
• 15:40
  inputs
• 15:41
  traditionally
• 15:41
  generate
• 15:42
  separate
• 15:42
  tokens
• 15:42
  per
• 15:42
  timestep
• 15:43
  per
• 15:43
  resolution
• 15:43
  (10m,
• 15:44
  20m,
• 15:45
  60m),
• 15:46
  meaning
• 15:46
  a
• 15:46
  two-timestep
• 15:47
  input
• 15:47
  yields
• 15:47
  six
• 15:47
  tokens
• 15:48
  per
• 15:48
  patch.
• 15:49
  OlmoEarth
• 15:49
  v1.1
• 15:51
  collapses
• 15:52
  resolutions
• 15:52
  into
• 15:52
  a
• 15:52
  single
• 15:53
  token
• 15:53
  per
• 15:53
  patch,
• 15:54
  cutting
• 15:54
  compute
• 15:54
  costs
• 15:55
  by
• 15:55
  up
• 15:55
  to
• 15:55
  3x
• 15:56
  while
• 15:56
  maintaining
• 15:57
  v1's
• 15:57
  performance
• 15:57
  on
• 15:57
  benchmarks
• 15:58
  and
• 15:58
  partner
• 15:58
  tasks.
• 15:59
  The
• 15:59
  naive
• 16:00
  approach
• 16:00
  of
• 16:00
  just
• 16:00
  merging
• 16:01
  tokens
• 16:01
  caused
• 16:01
  a
• 16:01
  10-point
• 16:02
  drop
• 16:02
  on
• 16:02
  m-eurosat
• 16:03
  kNN,
• 16:04
  so
• 16:04
  the
• 16:04
  team
• 16:04
  modified
• 16:04
  the
• 16:05
  pre-training
• 16:05
  regimen
• 16:05
  to
• 16:06
  compensate.
• 16:07
  Since
• 16:07
  they
• 16:07
  trained
• 16:07
  v1.1
• 16:09
  on
• 16:09
  the
• 16:09
  same
• 16:09
  dataset
• 16:09
  as
• 16:09
  v1,
• 16:10
  any
• 16:10
  performance
• 16:10
  differences
• 16:11
  isolate
• 16:11
  the
• 16:11
  effect
• 16:11
  of
• 16:12
  methodological
• 16:12
  changes,
• 16:13
  an
• 16:13
  unusually
• 16:13
  clean
• 16:14
  ablation
• 16:14
  for
• 16:14
  a
• 16:14
  field
• 16:14
  where
• 16:15
  architecture,
• 16:15
  data,
• 16:16
  and
• 16:16
  training
• 16:16
  algorithm
• 16:17
  all
• 16:17
  move
• 16:17
  simultaneously.
• 16:19
  Partners
• 16:19
  are
• 16:19
  already
• 16:19
  using
• 16:20
  it
• 16:20
  for
• 16:20
  tracking
• 16:20
  mangrove
• 16:20
  change,
• 16:21
  classifying
• 16:22
  forest
• 16:22
  loss
• 16:22
  drivers,
• 16:23
  and
• 16:23
  producing
• 16:23
  country-scale
• 16:24
  crop-type
• 16:24
  maps.
• 16:28
  OpenAI
• 16:29
  adopted
• 16:29
  Google's
• 16:29
  SynthID
• 16:30
  watermarking
• 16:31
  for
• 16:31
  AI-generated
• 16:31
  images,
• 16:32
  adding
• 16:32
  an
• 16:32
  invisible
• 16:33
  watermark
• 16:33
  layer
• 16:34
  that
• 16:34
  complements
• 16:34
  the
• 16:34
  C2PA
• 16:35
  metadata-based
• 16:36
  Content
• 16:36
  Credentials
• 16:36
  they
• 16:37
  have
• 16:37
  been
• 16:37
  shipping
• 16:37
  since
• 16:37
  2024.
• 16:39
  The
• 16:39
  multi-layered
• 16:39
  approach
• 16:40
  matters
• 16:40
  because
• 16:40
  C2PA
• 16:41
  metadata
• 16:41
  can
• 16:42
  be
• 16:42
  stripped
• 16:42
  through
• 16:42
  uploads,
• 16:43
  downloads,
• 16:43
  format
• 16:44
  changes,
• 16:44
  resizing,
• 16:45
  or
• 16:45
  screenshots,
• 16:46
  while
• 16:46
  SynthID
• 16:47
  embeds
• 16:47
  a
• 16:47
  signal
• 16:48
  that
• 16:48
  survives
• 16:48
  most
• 16:49
  of
• 16:49
  those
• 16:49
  transformations.
• 16:50
  Neither
• 16:50
  is
• 16:51
  foolproof
• 16:51
  alone,
• 16:52
  but
• 16:52
  together
• 16:52
  they
• 16:52
  make
• 16:52
  provenance
• 16:53
  significantly
• 16:53
  more
• 16:54
  durable.
• 16:55
  OpenAI
• 16:55
  also
• 16:56
  became
• 16:56
  C2PA
• 16:57
  conformant
• 16:58
  and
• 16:58
  is
• 16:58
  previewing
• 16:58
  a
• 16:58
  public
• 16:59
  verification
• 16:59
  tool
• 17:00
  that
• 17:00
  checks
• 17:00
  whether
• 17:00
  an
• 17:00
  uploaded
• 17:01
  image
• 17:01
  was
• 17:01
  generated
• 17:02
  on
• 17:02
  ChatGPT,
• 17:03
  the
• 17:03
  API,
• 17:04
  or
• 17:04
  Codex
• 17:05
  by
• 17:05
  looking
• 17:05
  for
• 17:05
  both
• 17:05
  Content
• 17:06
  Credentials
• 17:06
  and
• 17:07
  SynthID
• 17:07
  watermarks.
• 17:09
  The
• 17:09
  cross-company
• 17:09
  cooperation
• 17:10
  here,
• 17:10
  Google's
• 17:11
  watermarking
• 17:11
  technology
• 17:12
  adopted
• 17:12
  by
• 17:12
  its
• 17:13
  primary
• 17:13
  competitor,
• 17:14
  signals
• 17:14
  that
• 17:14
  the
• 17:15
  provenance
• 17:15
  problem
• 17:15
  is
• 17:15
  being
• 17:16
  treated
• 17:16
  as
• 17:16
  pre-competitive
• 17:17
  infrastructure
• 17:17
  rather
• 17:18
  than
• 17:18
  a
• 17:18
  differentiator.
• 17:19
  On
• 17:19
  the
• 17:20
  sovereignty
• 17:20
  front,
• 17:21
  Swiss
• 17:21
  cloud
• 17:21
  provider
• 17:22
  Infomaniak
• 17:23
  transferred
• 17:23
  the
• 17:23
  majority
• 17:24
  of
• 17:24
  its
• 17:24
  voting
• 17:24
  rights
• 17:25
  to
• 17:25
  a
• 17:25
  public-interest
• 17:25
  foundation,
• 17:26
  the
• 17:27
  Fondation
• 17:27
  Infomaniak,
• 17:28
  in
• 17:28
  an
• 17:28
  irrevocable
• 17:29
  move
• 17:29
  that
• 17:29
  places
• 17:30
  the
• 17:30
  company
• 17:30
  beyond
• 17:30
  the
• 17:31
  reach
• 17:31
  of
• 17:31
  any
• 17:31
  takeover.
• 17:32
  The
• 17:32
  Foundation's
• 17:32
  nine-principle
• 17:33
  Shareholding
• 17:34
  Charter,
• 17:34
  signed
• 17:35
  before
• 17:35
  a
• 17:35
  notary,
• 17:35
  cannot
• 17:36
  be
• 17:36
  weakened
• 17:36
  by
• 17:36
  the
• 17:37
  Foundation
• 17:37
  Board.
• 17:38
  Data
• 17:38
  entrusted
• 17:39
  by
• 17:39
  customers
• 17:40
  remains
• 17:40
  their
• 17:40
  property,
• 17:41
  any
• 17:41
  use
• 17:42
  beyond
• 17:42
  service
• 17:42
  delivery
• 17:43
  including
• 17:43
  AI
• 17:44
  model
• 17:44
  training
• 17:44
  requires
• 17:45
  explicit
• 17:46
  opt-in
• 17:46
  consent,
• 17:47
  and
• 17:47
  ecological
• 17:48
  impact
• 17:48
  must
• 17:48
  be
• 17:48
  avoided
• 17:49
  at
• 17:49
  source.
• 17:50
  With
• 17:50
  no
• 17:50
  external
• 17:51
  investors
• 17:51
  and
• 17:51
  36
• 17:52
  employee-shareholders
• 17:53
  who
• 17:53
  unanimously
• 17:54
  approved
• 17:54
  the
• 17:54
  transfer,
• 17:55
  Infomaniak
• 17:56
  joins
• 17:56
  Bosch,
• 17:57
  Carl
• 17:57
  Zeiss,
• 17:58
  Rolex,
• 17:58
  and
• 17:59
  Victorinox
• 17:59
  in
• 17:59
  the
• 17:59
  foundation-ownership
• 18:00
  model
• 18:01
  but
• 18:01
  claims
• 18:01
  to
• 18:01
  be
• 18:01
  the
• 18:01
  first
• 18:02
  European
• 18:02
  cloud
• 18:02
  provider
• 18:03
  to
• 18:03
  take
• 18:03
  this
• 18:03
  step.
• 18:04
  The
• 18:04
  timing
• 18:05
  is
• 18:05
  deliberate:
• 18:06
  acceleration
• 18:07
  of
• 18:07
  AI,
• 18:08
  takeovers
• 18:08
  of
• 18:08
  European
• 18:09
  cloud
• 18:09
  players,
• 18:10
  and
• 18:10
  strengthening
• 18:11
  of
• 18:11
  extraterritorial
• 18:12
  legislation
• 18:12
  all
• 18:13
  drove
• 18:13
  the
• 18:13
  urgency.
• 18:14
  RuFlo
• 18:15
  Graph
• 18:15
  Intelligence
• 18:15
  Engine
• 18:16
  takes
• 18:16
  a
• 18:16
  different
• 18:17
  approach
• 18:17
  to
• 18:17
  the
• 18:17
  infrastructure
• 18:18
  layer,
• 18:19
  offering
• 18:19
  incremental
• 18:20
  graph
• 18:20
  reasoning
• 18:20
  for
• 18:21
  agent
• 18:21
  coordination.
• 18:22
  Rather
• 18:23
  than
• 18:23
  recalculating
• 18:23
  the
• 18:24
  full
• 18:24
  graph
• 18:24
  every
• 18:24
  time
• 18:25
  something
• 18:25
  changes,
• 18:26
  the
• 18:26
  engine
• 18:26
  tracks
• 18:27
  deltas
• 18:27
  and
• 18:27
  updates
• 18:28
  only
• 18:28
  the
• 18:28
  affected
• 18:29
  subgraph,
• 18:29
  reducing
• 18:30
  token
• 18:30
  costs
• 18:31
  and
• 18:31
  enabling
• 18:31
  real-time
• 18:32
  agent
• 18:32
  decision
• 18:33
  routing
• 18:33
  on
• 18:33
  hardware
• 18:34
  as
• 18:34
  modest
• 18:34
  as
• 18:34
  a
• 18:34
  Raspberry
• 18:35
  Pi.
• 18:36
  It
• 18:36
  integrates
• 18:36
  directly
• 18:37
  into
• 18:37
  Claude
• 18:37
  Code
• 18:38
  via
• 18:38
  plugin
• 18:38
  marketplace
• 18:39
  and
• 18:39
  targets
• 18:39
  use
• 18:40
  cases
• 18:40
  from
• 18:40
  RAG
• 18:40
  pipeline
• 18:41
  optimization
• 18:41
  to
• 18:42
  trust
• 18:42
  scoring
• 18:42
  to
• 18:42
  fraud
• 18:43
  detection.
• 18:44
  The
• 18:44
  incremental-update
• 18:45
  pattern
• 18:46
  addresses
• 18:46
  a
• 18:46
  real
• 18:46
  cost
• 18:47
  ceiling
• 18:47
  in
• 18:47
  multi-agent
• 18:48
  systems:
• 18:49
  the
• 18:49
  "rebuild
• 18:50
  the
• 18:50
  whole
• 18:50
  graph
• 18:50
  because
• 18:51
  one
• 18:51
  tool
• 18:51
  call
• 18:51
  changed
• 18:52
  something
• 18:52
  downstream"
• 18:53
  problem
• 18:53
  that
• 18:54
  makes
• 18:54
  graph-based
• 18:54
  coordination
• 18:55
  prohibitively
• 18:56
  expensive
• 18:56
  at
• 18:56
  scale.