Latest / The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI
Orchestrating strictly sequential ETL pipelines at Synechron
Strict sequential execution across DAGs sounds simple until you have scheduled pipelines and event-driven pipelines writing to the same MongoDB collections. Ivana Isailovic, Senior Big Data Engineer at Synechron, joins Marc Lamberti to walk through the three-layer DAG architecture her team built to solve exactly that, plus how they generate 200-task DAGs and how they rebuilt subdag-style group retries in Airflow 3.
Key Takeaways:
(00:00) Introduction.
(02:16) The stack: Snowflake source, MongoDB with a medallion (bronze, silver, gold) layout, Spark for processing, Airflow for orchestration…
The skinny
The skinny isn't ready yet — notes appear once the transcript is processed.