Latest / Deep End Era. Engineering with Sankalp
How Many Significant Figures? Quantising Large Language Models
Fictional digital lead Saoirse Kowal needs a seventy billion parameter model to fit on two eighty-gigabyte GPUs inside her offshore operator's data centre. Her engineer Teodor, a former instrument technician, asks how many significant figures they really need. In episode one hundred and eighteen we trace low precision from fixed-point DSPs and the first TPU to mixed-precision training, learn how integers and floating-point numbers live in a chip, and compare per-tensor, per-channel and per-block scales to instrument ranging. We meet the systematic outlier features that broke naive…
The skinny
The skinny isn't ready yet — notes appear once the transcript is processed.