Latest / The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI
Orchestrating AI video intelligence and evaluation pipelines at Firework
How do you orchestrate AI workflows when LLM outputs are non-deterministic and evaluation costs can quietly exceed compute costs? Shawn Feng, Head of Data at [Firework](firework.com), joins the show to walk through how his team uses Airflow to coordinate ingestion, embeddings, and evaluation pipelines for AI features in a video commerce platform. The conversation covers the technical stack, the challenges of validating LLM output, cost guardrails, and what Shawn wants to see next from the Airflow project.
Key Takeaways:
00:00 Introduction.
00:48 What Firework does: a video commerce platform…
The skinny
The skinny isn't ready yet — notes appear once the transcript is processed.