Latest / The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI
Introducing Airflow’s Common AI Provider with Pavan Kumar Gopidesu and Kaxil Naik
In this episode, we explore the newly released Apache Airflow common AI provider — what problem it solves, how it was built and what's coming next.
Kaxil Naik, Senior Director of Engineering at Astronomer and Apache Airflow PMC member, and Pavan Kumar Gopidesu, Lead Data Engineer at Experian and Apache Airflow PMC member, join us to walk through the provider's first release and the technical decisions behind it.
Key Takeaways:
00:00 Introduction.
04:05 The common AI provider was born from a real production problem.
07:10 Airflow already had the primitives needed for durable agent execution…
The skinny
The skinny isn't ready yet — notes appear once the transcript is processed.