Information Theory

Deep End Era. Engineering with Sankalp · Oct 10, 2026 · 1h 1m

How much information is in a message, and what does it take to send it reliably? Shannon's measure of surprise leads into source coding, Huffman and arithmetic coding, and modern compression techniques. The episode connects compression and prediction to broader questions about intelligence and physical limits, including the cost of forgetting. A long-form route through the ideas that govern efficient descriptions, reliable communication and the distinction between signal and uncertainty.

Chapters

00:00:00 Cold Open: The Last Window

00:02:01 Chapter 1: Shannon and the Measure of Surprise…