Latest / Models & Agents — Video Edition
Ep 186: Local inference on Apple Silicon just got faster with a tuned fork of the Splash engine…
Models & Agents Local inference on Apple Silicon just got faster with a tuned fork of the Splash engine delivering up to 1.5 times the speed on M5 Max chips. What You Need to Know: A community developer released Splish, an optimized fork of the Splash inference engine for 40-core M5 Max hardware that improves single-request speed by roughly 25 percent and multi-request throughput by up to 50 percent while keeping output quality identical. KT's Auto Model Router placed second on the Router Arena benchmark that evaluates accuracy, cost, and robustness across 8400 queries. A new Towards Data…
The skinny
The skinny isn't ready yet — notes appear once the transcript is processed.