Latest / The Augur Dispatch / What Should an AI Agent Remember? The Cost Case for Compact Context

Transcript

What Should an AI Agent Remember? The Cost Case for Compact Context · The Augur Dispatch

• 0:06
  SPEAKER_00: An agent, given its full history, finished fewer tasks than one given no memory.
• 0:11
  SPEAKER_00: That's the striking result on Apple's research page today.
• 0:14
  SPEAKER_00: Read the current paper though, and the authors make a narrower claim.
• 0:18
  SPEAKER_00: Compact context costs less, while the completion evidence is still small.
• 0:23
  SPEAKER_00: Before your team decides to remember everything or delete the lot, test which information the next job actually needs.
• 0:30
  SPEAKER_00: This is the Augur Dispatch for Thursday, September 17, 2026.
• 0:35
  SPEAKER_00: The Enterprise study covered 24 tasks.
• 0:38
  SPEAKER_00: Selective memory completed 23, no memory, 19, and full history, 17.
• 0:44
  SPEAKER_00: The authors say those differences aren't statistically established after their correction for multiple comparisons.
• 0:51
  SPEAKER_00: There's a second complication.
• 0:53
  SPEAKER_00: In 18 of the selective condition tasks, the system reused an existing artifact and updated its data without calling the model.
• 1:01
  SPEAKER_00: That's useful automation, but it makes this more than a comparison of memory settings.
• 1:05
  SPEAKER_00: The sensible response is a bounded pilot, not a company-wide order to forget more.
• 1:13
  SPEAKER_00: Picture the analyst who rebuilds a weekly sales report because the agent has forgotten which columns matter and how the director wants the result laid out.
• 1:21
  SPEAKER_00: The useful memory might fit on a page, the task, the data structure, the tools available, and the rules for the output.
• 1:28
  SPEAKER_00: That's what the design retains.
• 1:30
  SPEAKER_00: The conversation's working notes and reasoning traces don't come along.
• 1:34
  SPEAKER_00: In the paper's public data experiment, Compact Memory completed all 12 trials, full history, 8, and no memory, none.
• 1:43
  SPEAKER_00: But the last group wasn't given the earlier formatting requirement again.
• 1:47
  SPEAKER_00: Its reports worked structurally.
• 1:49
  SPEAKER_00: They failed a requirement the agent couldn't see.
• 1:52
  SPEAKER_00: Forgetting the instructions is a fairly dependable way to miss the instructions.
• 1:57
  SPEAKER_00: The difference between compact memory and full history wasn't statistically established on completion either.
• 2:03
  SPEAKER_00: Compact memory used about half the input tokens, which are the pieces of text the model processes and providers commonly charge for.
• 2:11
  SPEAKER_00: The compact workspace also retained the earlier artifact, so this experiment doesn't cleanly separate the value of memory text from the value of having existing work to revise.
• 2:22
  SPEAKER_00: These are the author's experiments, not an independent replication.
• 2:25
  SPEAKER_00: That still gives a workflow in or something concrete to investigate.
• 2:29
  SPEAKER_00: A transcript may carry useful context, but it may charge you repeatedly for a lot of material the next task doesn't need.
• 2:37
  SPEAKER_00: Saving less also means deciding who gets to save anything.
• 2:41
  SPEAKER_00: A memory injection paper describes how one poisoned interaction can steer an agent's later answers without the attacker directly editing its memory store.
• 2:52
  SPEAKER_00: It doesn't show that everyone sharing a workspace becomes compromised.
• 2:56
  SPEAKER_00: That's a separate test, not a finding we can borrow.
• 2:59
  SPEAKER_00: Imagine a team sharing a reporting workspace.
• 3:02
  SPEAKER_00: A convenient instruction saved on Tuesday could become Thursday's unwanted rule, unless someone owns the review and knows how to reverse it.
• 3:11
  SPEAKER_00: The Apple hosted design includes permissions tied to people's roles.
• 3:15
  SPEAKER_00: The practical question is whether those permissions cover the right people and the right changes.
• 3:20
  SPEAKER_00: A short memory can still contain a bad instruction.
• 3:24
  SPEAKER_00: The commercial products aren't simply offering a bigger transcript bucket either.
• 3:29
  SPEAKER_00: Anthropic describes individual memory topics and separate project memory.
• 3:33
  SPEAKER_00: Sakana lets people review retained preferences and turn memory off.
• 3:37
  SPEAKER_00: Their launches make continuity easier to use.
• 3:40
  SPEAKER_00: They don't prove that either vendor has abandoned selection.
• 3:43
  SPEAKER_00: IBM adds another reason to avoid a universal rule.
• 3:48
  SPEAKER_00: Its experiments concern reusable guidelines distilled from earlier work, not whole transcripts.
• 3:55
  SPEAKER_00: Some stronger models benefited from the full guideline set, while other models did better with selective retrieval.
• 4:02
  SPEAKER_00: The amount worth carrying depends on the model and the job.
• 4:05
  SPEAKER_00: Buying everyone a larger filing cabinet won't settle that.
• 4:10
  SPEAKER_00: For a useful first test, choose one recurring report and put its workflow owner in charge.
• 4:15
  SPEAKER_00: Have a security reviewer approve what can enter shared memory and review the results on September 24th.
• 4:21
  SPEAKER_00: Compare the existing workflow with no memory, the full transcript, and a compact page of instructions, keeping the model, tools, tasks, and scoring consistent across the comparison.
• 4:32
  SPEAKER_00: Give each version the information needed for the current task.
• 4:36
  SPEAKER_00: If a requirement must carry over from an earlier session, make that distinction explicit rather than quietly denying one version the instructions.
• 4:44
  SPEAKER_00: Count correctly completed reports, then measure model usage and the time a person spends fixing them.
• 4:50
  SPEAKER_00: Test refreshing an existing report separately from generating a new one.
• 4:54
  SPEAKER_00: Keep the compact version only if it maintains quality while lowering cost or correction time.
• 5:00
  SPEAKER_00: Stop for a material quality drop, an unauthorized memory change, or exposure of sensitive data.
• 5:05
  SPEAKER_00: Expand only after the result repeats on held out cases and another workflow.
• 5:10
  SPEAKER_00: If the transcript wins, find out what useful information your page left out.
• 5:15
  SPEAKER_00: You haven't proved that the agent needed its earlier reasoning merely because that reasoning was in the winning transcript.
• 5:22
  SPEAKER_00: Any lasting advantage comes from knowing your workflow well enough to write and maintain those instructions, and from having realistic tests.
• 5:31
  SPEAKER_00: Your competitors can buy the same memory feature.
• 5:35
  SPEAKER_00: Control over context leads to a different purchasing question: control over the supplier.
• 5:41
  SPEAKER_00: Cohere and Aleph Alpha have signed an agreement to combine under the Cohere name with planned headquarters in Berlin and Toronto.
• 5:49
  SPEAKER_00: Regulatory approval is still required, and the companies expect closing later this year.
• 5:54
  SPEAKER_00: A procurement lead should distinguish the supplier available to sign today from the combined company described in the announcement.
• 6:02
  SPEAKER_00: Ask Council how a change in ownership affects the agreement.
• 6:06
  SPEAKER_00: Mistral's announcement with Mozilla concerns a product already in use, Firefox Smart Window, still in beta.
• 6:12
  SPEAKER_00: It says Mistral models serve users in France and North America, with the United Kingdom and Germany expected later this year.
• 6:19
  SPEAKER_00: Partners such as Mistral agree to zero data retention for those interactions.
• 6:24
  SPEAKER_00: That is a specific commitment for this service, not a blanket promise about everything carrying the vendor's name.
• 6:30
  SPEAKER_00: Compliance teams also need to distinguish political statements from enforceable rules.
• 6:36
  SPEAKER_00: The dispatch's cited commentary reports President Trump calling extinction warnings a hoax.
• 6:42
  SPEAKER_00: That doesn't settle what Washington will do next.
• 6:45
  SPEAKER_00: The European AI office has enforcement powers over general-purpose models from August 2nd.
• 6:51
  SPEAKER_00: Build around the obligations that apply to you while treating the American policy direction as unsettled rather than absent.
• 6:59
  SPEAKER_00: A lower model bill means its own evidence.
• 7:02
  SPEAKER_00: Together AI says some customers cut costs by up to 70% when moving to open models and suggests starting with 10% of traffic.
• 7:11
  SPEAKER_00: That's a vendor-reported result and a migration suggestion, not a saving you can put into next quarter's budget without checking your workload.
• 7:19
  SPEAKER_00: Watch the costs per correctly completed task, including whatever people have to repair.
• 7:25
  SPEAKER_00: Sakana has made Fugu Max available to all Sakana chat users, replacing the earlier Fugu option.
• 7:31
  SPEAKER_00: Users select it from the model picker.
• 7:33
  SPEAKER_00: It routes requests across open models.
• 7:35
  SPEAKER_00: Frontier level performance remains Sakana's claim.
• 7:38
  SPEAKER_00: On Augur's own record, the August call that Together, Fireworks, or OpenRouter would host DeepSeek V4 Flash by September 15th is graded a provisional miss.
• 7:49
  SPEAKER_00: That stays on the record.
• 7:52
  SPEAKER_00: Today's new calls are testable predictions with explicit deadlines, not recommendations to buy a product.
• 8:00
  SPEAKER_00: Augur assigns 60% odds that Cohere publicly announces the completed Aliph Alpha combination by December 31st, 2026.
• 8:09
  SPEAKER_00: A public closing confirmation from either company qualifies.
• 8:13
  SPEAKER_00: No confirmation by the deadline or termination is amiss.
• 8:18
  SPEAKER_00: The other call carries 55% odds.
• 8:21
  SPEAKER_00: Mozilla or Mistral publicly confirms Smart Window with Mistral models is available in both the United Kingdom and Germany by that same date.
• 8:30
  SPEAKER_00: Both countries must be live.
• 8:32
  SPEAKER_00: Either still unsupported or delayed beyond the deadline means amiss.
• 8:36
  SPEAKER_00: For now, pilot compact memory on one workflow.
• 8:40
  SPEAKER_00: Watch the deal closing and the regional rollout.
• 8:43
  SPEAKER_00: Don't turn a small research result into a deletion policy.
• 8:47
  SPEAKER_00: Which recurring task would you trust to a page of maintained instructions?
• 8:51
  SPEAKER_00: And what would you need to see before dropping its transcript?
• 8:55
  SPEAKER_00: That is the Augur Dispatch for Thursday, September 17th, 2026.
• 9:00
  SPEAKER_00: Thank you for listening.
• 9:02
  SPEAKER_00: The written brief with every source is at augurdispatch.ai