Eleven Per Cent of Peak: How GPUs Are Programmed

Deep End Era. Engineering with Sankalp · Oct 28, 2026 · 57m

At eleven at night, Haruto's new attention layer is using eleven per cent of the most powerful chip money can buy. Episode one hundred and eighty eight explains why, and how he fixes it, through the programming models of graphics processors: CUDA's threads and warps, the memory hierarchy that decides almost all performance, the roofline model, tensor cores, FlashAttention, Triton, NVIDIA's new tile model, AMD's ROCm, and the long struggle to break CUDA's dominance. Haruto and his company are fictional; everything else is sourced in the fact-check log. Builds on the GPU hardware and AI chip…