Latest / The Augur Dispatch / Liquid AI's local agent model: test the job, not the token speed

Transcript

Liquid AI's local agent model: test the job, not the token speed · The Augur Dispatch

• 0:00
  SPEAKER_00: Liquid AI's new local model has 24 billion parameters, but uses only 2 billion of them for each piece of text it processes.
• 0:08
  SPEAKER_00: That distinction matters when a vendor says a bigger model can still run quickly on a laptop.
• 0:14
  SPEAKER_00: Augur's recommendation is to test a narrow part of an existing agent's work locally, while keeping the current system as the comparison.
• 0:21
  SPEAKER_00: Don't book the savings before the local system has completed the job correctly.
• 0:26
  SPEAKER_00: This is the Augur dispatch for Thursday, October 15th, 2026.
• 0:31
  SPEAKER_00: Liquid reports a score of about 48 on LiveClaw, a test of agent tool use, against about 38 for one competing open model from Quen.
• 0:40
  SPEAKER_00: These are benchmark scores, not percentages of your company's tasks that the model will finish.
• 0:46
  SPEAKER_00: Liquid also says its model leads or ties on five of eight selected tests of tool use and agent work.
• 0:53
  SPEAKER_00: That's enough to make a trial interesting.
• 0:56
  SPEAKER_00: It isn't enough to move a production workflow, and the company's own results explain why.
• 1:01
  SPEAKER_00: Picture a support engineer working through a customer's problem late in the afternoon.
• 1:06
  SPEAKER_00: The assistant needs to retrieve an approved record, read a status field, and prepare an answer for the engineer to check.
• 1:13
  SPEAKER_00: Some of that work may be repetitive and well specified, especially when the relevant information is already in the record, rather than something the model must supply from memory.
• 1:23
  SPEAKER_00: Paying for the same powerful remote model at every step might be unnecessary, but the moment the record is incomplete, the assistant has to recognize that gap, rather than invent a confident answer.
• 1:35
  SPEAKER_00: Liquid's new model is called LFM 2.524B A2B.
• 1:41
  SPEAKER_00: It's a reasoning model, which means it can spend computation working through a problem before producing its answer.
• 1:48
  SPEAKER_00: Liquid reports that larger models still lead on its knowledge-heavy encoding tests.
• 1:53
  SPEAKER_00: We shouldn't turn that into a neat rule that local models can use tools while remote models do all the thinking.
• 1:59
  SPEAKER_00: Tool use itself can require substantial reasoning, and a task that looks routine can become difficult when a tool fails or returns something unexpected.
• 2:08
  SPEAKER_00: For the engineer, the useful question is narrower.
• 2:12
  SPEAKER_00: Can this exact setup handle the recurring jobs that already fill the queue?
• 2:16
  SPEAKER_00: The sales chart has never had to explain a wrong answer to that customer.
• 2:21
  SPEAKER_00: The laptop speed figure is promising, with an important limit.
• 2:26
  SPEAKER_00: Liquid reports about 184 tokens per second on an Apple M5 Max, a high-end chip.
• 2:32
  SPEAKER_00: Tokens are fragments of text, so this measures how quickly the model produces text rather than how quickly it finishes a business task.
• 2:40
  SPEAKER_00: The company also reports more than 21,000 output tokens per second on a single Nvidia server chip.
• 2:47
  SPEAKER_00: That much bigger number combines many requests running at once.
• 2:51
  SPEAKER_00: It isn't the speed one employee would see from one conversation, and it shouldn't be placed beside the laptop number as though both measure the same experience.
• 3:00
  SPEAKER_00: The model's design also matters.
• 3:03
  SPEAKER_00: Although it has 24 billion parameters in total, it activates only 2 billion for each token.
• 3:09
  SPEAKER_00: Parameters are the learned values that help determine a model's output.
• 3:13
  SPEAKER_00: Using only a portion at a time can reduce computation, but the rest of the model still has to be stored somewhere.
• 3:19
  SPEAKER_00: A buyer can't infer memory requirements from the smaller active count, or infer that an ordinary office laptop will match the vendor's high-end machine.
• 3:28
  SPEAKER_00: Earlier speed claims for Liquid's smaller models came from separate posts, not a controlled comparison of identical jobs on one shared computer.
• 3:37
  SPEAKER_00: Return to that support engineer.
• 3:39
  SPEAKER_00: A response that starts quickly but needs three repairs may take longer than the existing system, particularly if the engineer has to inspect each repair before it reaches a customer.
• 3:50
  SPEAKER_00: The stopwatch belongs on the completed job.
• 3:53
  SPEAKER_00: There's another reason to test the whole system.
• 3:56
  SPEAKER_00: Running the model locally doesn't automatically keep the information local.
• 4:00
  SPEAKER_00: An agent may send data through the tools it calls.
• 4:03
  SPEAKER_00: Its logs may leave the machine.
• 4:05
  SPEAKER_00: A remote model might receive the difficult cases if someone has configured automatic escalation.
• 4:11
  SPEAKER_00: Moving the model onto a laptop doesn't close any of those routes by itself.
• 4:15
  SPEAKER_00: Keep confidential inputs out of remote routes unless their transfer has been explicitly authorized, and inspect the tool and logging behavior before describing a workflow as private.
• 4:26
  SPEAKER_00: Software installation creates a related problem.
• 4:29
  SPEAKER_00: Trail of Bits tested 25 package managers, the programs that download software libraries, against tampering at the repository.
• 4:37
  SPEAKER_00: Only two checked every downloaded artifact in its test, and fewer than half provided integrity checking lock files by default.
• 4:45
  SPEAKER_00: A lock file records which versions a project expects.
• 4:49
  SPEAKER_00: It can also record trusted fingerprints of the files, letting the installer reject bytes that no longer match.
• 4:55
  SPEAKER_00: Merely pinning a version number doesn't establish that this check happens.
• 4:59
  SPEAKER_00: The dispatch also discusses the chaindrop supply chain attack, where a compromised maintainer account helped spread malicious releases.
• 5:07
  SPEAKER_00: That crosses a different trust boundary.
• 5:10
  SPEAKER_00: A malicious release can match its published fingerprint perfectly, so checking the download doesn't prove that the code was safe to publish in the first place.
• 5:18
  SPEAKER_00: For the person operating the agent, those are separate responsibilities.
• 5:23
  SPEAKER_00: Enforce trusted integrity checks, review dependency changes, and restrict what installation scripts can do.
• 5:29
  SPEAKER_00: Local ownership gives that person more control, along with the maintenance work.
• 5:33
  SPEAKER_00: The laptop won't volunteer for the on-call rotation.
• 5:37
  SPEAKER_00: The smallest useful trial keeps those responsibilities visible without turning into a fleet migration.
• 5:44
  SPEAKER_00: Use a read-only internal support workflow with sanitized examples from a representative day and an engineering owner who already knows what successful completion looks like.
• 5:54
  SPEAKER_00: Run the proposed local setup and the current approved route on the same jobs, keeping any rights simulated so an experiment can't change a real customer's record.
• 6:03
  SPEAKER_00: The important result is correctly completed work, with the time and effort needed to get there.
• 6:09
  SPEAKER_00: Count retries and human review as well as computing costs.
• 6:12
  SPEAKER_00: A lower model bill isn't a saving if an engineer spends the difference correcting answers or maintaining the machine.
• 6:19
  SPEAKER_00: Augur proposes reviewing the trial on October 22nd.
• 6:23
  SPEAKER_00: That's a suggested working checkpoint, not a deadline from Liquid.
• 6:27
  SPEAKER_00: Stop if the agent takes an unauthorized action or sends sensitive information somewhere it shouldn't.
• 6:33
  SPEAKER_00: Don't expand if correctness falls below the existing workflow or if support costs erase the apparent savings.
• 6:40
  SPEAKER_00: A promising first run earns a broader test on representative jobs, not permission to remove safeguards.
• 6:46
  SPEAKER_00: Include the awkward cases that made the current system expensive to operate, because those are often where an attractive average stops describing the work.
• 6:55
  SPEAKER_00: Now picture the support engineer looking at the results with the team lead.
• 6:59
  SPEAKER_00: If one recurring group of jobs finishes correctly at lower total cost, they can route that group locally and leave the difficult remainder where it is.
• 7:08
  SPEAKER_00: Nobody needs to declare a winner for every possible task.
• 7:12
  SPEAKER_00: The lasting advantage would come from knowing the organization's own workload and improving its routing decisions over time.
• 7:18
  SPEAKER_00: Downloading a model that competitors can also obtain doesn't create that knowledge by itself.
• 7:24
  SPEAKER_00: Signals on the clock are separate from Augur's dated predictions.
• 7:28
  SPEAKER_00: For this decision, watch for independent reruns of Liquid's agent tests and evidence that quality survives the actual local deployment.
• 7:36
  SPEAKER_00: The vendor's reported benchmark results don't establish what a different model format, runtime, or machine will deliver.
• 7:43
  SPEAKER_00: Also watch the stable release notes for Olama's current release candidate work, and whether software ecosystems strengthen their default integrity checks.
• 7:52
  SPEAKER_00: Those developments may reduce operational friction, but they won't replace testing your own workflow.
• 7:57
  SPEAKER_00: Augur's posture is to pilot the bounded workflow, adopt the safeguards regardless of model location, and watch for stronger evidence before making a wider move.
• 8:07
  SPEAKER_00: The written dispatch covers Google's experimental geospatial agent and Anthropic's election use analysis as well.
• 8:14
  SPEAKER_00: Today's episode stays with the local agent decision.
• 8:17
  SPEAKER_00: The written brief with every source is at Augerdispatch.ai.
• 8:22
  SPEAKER_00: Portions of the Augur Dispatch are AI generated.
• 8:25
  SPEAKER_00: That is the Augur Dispatch for Thursday, October 15th, 2026.
• 8:30
  SPEAKER_00: Thank you for listening.