Latest / The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI
Inside the Custom Framework for Managing Airflow Code at Wix with Gil Reich
Efficient orchestration and maintainability are crucial for data engineering at scale. Gil Reich, Data Developer for Data Science at Wix, shares how his team reduced code duplication, standardized pipelines, and improved Airflow task orchestration using a Python-based framework built within the data science team.
In this episode, Gil explains how this internal framework simplifies DAG creation, improves documentation accuracy, and enables consistent task generation for machine learning pipelines. He also shares lessons from complex DAG optimization and maintaining testable code.
Key…
The skinny
The skinny isn't ready yet — notes appear once the transcript is processed.