The Straight Line on Log Paper: Scaling Laws and Mixture of Experts

Deep End Era. Engineering with Sankalp · Oct 28, 2026 · 57m

On a Sunday night, fictional AI lead Ines Varga has to tell an investment committee how big a model to train, what it will cost, and how she knows. Her sceptical engineer Bram, a former reservoir modeller, sees the famous straight line on log-log paper and calls it a decline curve. In episode one hundred and fifteen we unpack what loss measures, why training compute is about six times parameters times tokens, how Kaplan's 2020 laws pushed the industry toward giant under-trained models, and how Chinchilla's seventy billion parameters on one point four trillion tokens beat models four times its…