Columnar Storage, Late Materialization, and Lakehouse Table Formats: Parquet, ORC, Arrow, Iceberg, Delta, and Why DuckDB Fits on a Laptop

Deep End Era. Engineering with Sankalp · Oct 29, 2026 · 1h 6m

When an analytical query sums three columns out of a two-hundred-column table stored in row-oriented format, ninety-eight percent of every disk read and CPU cache line is wasted hauling unused bytes into memory. We open up the binary internals of Apache Parquet, ORC, Apache Arrow, Dremel nested shredding, Late Materialization, DuckDB, and Apache Iceberg/Delta Lake to see how modern analytical engines scan billions of rows in milliseconds.

Chapters

00:00:00 Cold Open: The Four-Hundred-Thousand-Dollar S3 Scan Bill and the Laptop That Beat a Cluster

00:03:55 NSM vs DSM vs PAX: The Physics of…