Latest / AnyMessages · Daily Podcast
Three Attention Types, One Cache: SGLang's Hybrid-Model Prefix Caching Plan
Unified Radix Cache puts FULL, SWA, and Mamba reuse rules into a single radix tree, while HiCache extends the components to L3. L3 hit rate reaches 98%, and the Python-to-Rust port cuts tail-turn TTFT by 42%.
The skinny
The skinny isn't ready yet — notes appear once the transcript is processed.