Latest / The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI
Airflow in Action: Powering Instacart's Complex Ecosystem
The evolution of data orchestration at Instacart highlights the journey from fragmented systems to robust, standardized infrastructure. This transformation has enabled scalability, reliability and democratization of tools for diverse user personas.
In this episode, we’re joined by Anant Agarwal, Software Engineer at Instacart, who shares insights into Instacart's Airflow journey, from its early adoption in 2019 to the present-day centralized cluster approach. Anant discusses the challenges of managing disparate clusters, the implementation of remote executors, and the strategic…
The skinny
The skinny isn't ready yet — notes appear once the transcript is processed.