EP5: Speculative Decoding with Nadav Timor

The Information Bottleneck · Sep 16, 2025 · 1h 2m

We discussed the inference optimization technique known as Speculative Decoding with a world class researcher, expert, and ex-coworker of the podcast hosts: Nadav Timor.

Papers and links:

Accelerating LLM Inference with Lossless Speculative Decoding Algorithms for Heterogeneous Vocabularies, Timor et al, ICML 2025, https://arxiv.org/abs/2502.05202

Distributed Speculative Inference (DSI): Speculation Parallelism for Provably Faster Lossless Language Model Inference, Timor et al, ICLR, 2025, https://arxiv.org/abs/2405.14105

Fast Inference from Transformers via Speculative Decoding, Leviathan et…