Latest / The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI / Inside Conviva’s Decision To Power Its Data Platform With Airflow with Han Zhang

Transcript

Inside Conviva’s Decision To Power Its Data Platform With Airflow with Han Zhang · The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI

• 0:00
  SPEAKER_02: We have a standalone all striction web service that's provided API endpoint to evoke the workflow execution.
• 0:06
  SPEAKER_02: And the Airflow is from the architectural point of view, it's behind the scenes.
• 0:11
  SPEAKER_02: So the web service directly invoke, create new workflow inside Airflow.
• 0:16
  SPEAKER_02: However, because the built-in functionalities, the dashboard, the monitoring, the log exploration of Airflow, we also open up the Airflow webpage to our internal users.
• 0:47
  SPEAKER_00: Hi everyone.
• 0:48
  SPEAKER_00: Welcome to the Data Flowcast.
• 0:49
  SPEAKER_00: I'm your host, Kenton Dannis, and today I'm joined by Han Zhang, who is a senior software engineer at Conviva.
• 0:56
  SPEAKER_00: Han, how are you doing today?
• 0:58
  SPEAKER_02: I'm good.
• 0:59
  SPEAKER_02: Yeah.
• 1:00
  SPEAKER_02: Thank you, Basi.
• 1:01
  SPEAKER_00: Great.
• 1:01
  SPEAKER_00: Yeah, it's wonderful to have you on.
• 1:03
  SPEAKER_00: For our listeners for today's podcast, we're going to discuss how CanViva uses Airflow as the back end of their data platform.
• 1:10
  SPEAKER_00: But before we get into all of that, I want to start with a little bit of background.
• 1:14
  SPEAKER_00: Can you tell me what CanViva does?
• 1:16
  SPEAKER_02: Sure.
• 1:16
  SPEAKER_02: Yeah.
• 1:17
  SPEAKER_02: Canviva is an intelligence platform for digital businesses.
• 1:22
  SPEAKER_02: We use client-side telemetry data to transform every consumer interaction across apps, websites, AI engines into an outcome-based intelligence.
• 1:32
  SPEAKER_02: We can service real-time actionable insights and help our customers to feel growth and customer satisfaction.
• 1:39
  SPEAKER_02: A relatively scale that we are operating at is we process 5 trillion events every day.
• 1:46
  SPEAKER_02: And it takes less than 30 seconds from the moment we collect the events all the way until those insights surface up on our dashboard.
• 1:53
  SPEAKER_00: Wow, that's fast.
• 1:55
  SPEAKER_02: We had a lot of pressure to do this in real time.
• 1:57
  SPEAKER_00: Okay, great.
• 1:58
  SPEAKER_00: So you have a streaming analytics platform then for your customer data.
• 2:02
  SPEAKER_00: Very interesting.
• 2:04
  SPEAKER_00: And what does your team do?
• 2:06
  SPEAKER_02: Yeah, so for our team, we're a control plane team.
• 2:10
  SPEAKER_02: So our main goal is to orchestrate multiple different internal components to work with the batch workload.
• 2:17
  SPEAKER_02: As I mentioned before, the historically Conviva is heavily focused on the real-time streaming analytics.
• 2:23
  SPEAKER_02: So a lot of components are built optimized for streaming environments.
• 2:28
  SPEAKER_02: In the beginning, like about one and a half year ago, we were tasked with a mission to turn the whole pipeline into a batch-oriented workload.
• 2:36
  SPEAKER_02: And that involves visiting all the components and figure out what changes we need to make and how to adjust them to batch or on-demand workload.
• 2:45
  SPEAKER_00: Yeah, so just to make sure that I'm understanding correctly, so you have these batch pipelines that are helping to support the data platform that powers your streaming analytics business.
• 2:58
  SPEAKER_00: Is that right?
• 2:58
  SPEAKER_00: So you still have a streaming platform, but there's batch processing underneath it, or how does that work together?
• 3:06
  SPEAKER_02: Yeah, so we are more complementary than the streaming to the streaming components.
• 3:11
  SPEAKER_02: So like a lot of functionalities in our main product is powered by the streaming pipeline.
• 3:16
  SPEAKER_02: But for certain functionalities or certain pages, they need batch processing capabilities.
• 3:20
  SPEAKER_02: Like we can't do that in real time, or sometimes the data is already, or we need to enrich the data in an offline process.
• 3:28
  SPEAKER_02: So once we finish that part, we need to rerun the compute engine and ingest it to our database again.
• 3:34
  SPEAKER_02: So we are a complementary part of the product service side.
• 3:38
  SPEAKER_02: Some functionalities are powered by us.
• 3:40
  SPEAKER_00: Okay, got it.
• 3:41
  SPEAKER_00: That makes sense.
• 3:42
  SPEAKER_00: And then how does Airflow fit into that?
• 3:45
  SPEAKER_02: Yeah, so in Airflow is like the heart of our orchestration implementation.
• 3:51
  SPEAKER_02: We picked Airflow because of the orchestration framework that and the capability they offer.
• 3:57
  SPEAKER_02: Let me dive into the architecture of our control plane a little bit.
• 4:00
  SPEAKER_02: So hopefully this helps illustrate where Airflow fits in.
• 4:04
  SPEAKER_02: So there are three major components in our control plane.
• 4:08
  SPEAKER_02: The first one is a web service endpoint, has a dedicated persistent database storage.
• 4:14
  SPEAKER_02: This manages the high-level business logic.
• 4:18
  SPEAKER_02: For example, it manages the instance lifecycle.
• 4:21
  SPEAKER_02: And if something failed, how do we roll back and how do we clean up the intermediate corrupted results?
• 4:27
  SPEAKER_02: And we also want to enforce a read limit.
• 4:30
  SPEAKER_02: Like every customer, they have a certain quota, and we don't want them to have too many running instances at the same time.
• 4:36
  SPEAKER_02: We call it the web service orchestrator components.
• 4:39
  SPEAKER_02: And the heavy lifting is the actual orchestration happens with airflow.
• 4:45
  SPEAKER_02: So the web service invoked the airflow to create a new task, new instance of the target workflow.
• 4:53
  SPEAKER_02: And Airflow orchestrates multiple tasks and coordinated dependencies.
• 4:58
  SPEAKER_02: And once the upstream finishes, evoke the downstream and then do some sanity check.
• 5:03
  SPEAKER_02: And once the workflow finishes, either succeeded or failed, it notified back to the web service.
• 5:09
  SPEAKER_02: So we manage the overall lifecycle of the given task or workflow.
• 5:14
  SPEAKER_02: And the third component is the data plane.
• 5:18
  SPEAKER_02: We have a pool of resources in multiple Kubernetes clusters.
• 5:22
  SPEAKER_02: So Airflow only does the orchestration.
• 5:26
  SPEAKER_02: If it needs to invoke data plane processing, it will create pod or Kubernetes pod operator to actually perform the heavy lifting of data processing and compute the metrics and persist the metrics, notify the show up in the dashboard.
• 5:41
  SPEAKER_00: Okay, interesting.
• 5:42
  SPEAKER_00: So you've built this entire platform around Airflow, where Airflow is powering your data platform.
• 5:49
  SPEAKER_00: I'm curious what made you decide to build this platform in the first place?
• 5:56
  SPEAKER_00: I you touched on it a little bit, I think, in your introduction, but maybe just to reiterate why you needed this complex system as opposed to just using a batch orchestrator.
• 6:06
  SPEAKER_02: This is a very complicated question.
• 6:09
  SPEAKER_02: There are a few parts.
• 6:10
  SPEAKER_02: So I'm breaking into two parts.
• 6:12
  SPEAKER_02: The first one, why we decide to build a batch-oriented workload, and then why we choose this airflow versus others.
• 6:19
  SPEAKER_02: Historically speaking, Conviva is heavily focused on the real-time streaming data analytics.
• 6:25
  SPEAKER_02: So it's everything we process one time, and then the dashboard just keeps loading the pre-computed results.
• 6:31
  SPEAKER_02: About one and a half years ago, uh the company had given us a mission.
• 6:36
  SPEAKER_02: Like, okay, here's what we have.
• 6:38
  SPEAKER_02: All the internal components that we build, our proprietary compute engine and the way we process data and we clean up data and database ingestion, they are all designed for streaming use cases.
• 6:49
  SPEAKER_02: However, there are more and more new applications and new use cases that cannot be satisfied with one-pass streaming processing.
• 6:57
  SPEAKER_02: We need to build up this batch processing capabilities.
• 7:00
  SPEAKER_02: However, given we already have all these components, can we change the way we use them?
• 7:06
  SPEAKER_02: And we need these orchestration capabilities to build up this batch processing type one.
• 7:12
  SPEAKER_02: With that, we realized we we need to pick an option for the orchestration framework, or we could build it ourselves.
• 7:21
  SPEAKER_02: And there's a long story.
• 7:23
  SPEAKER_02: We spent a lot of effort evaluating different options, and we ended up with the airflow.
• 7:28
  SPEAKER_02: If you like, I can jump into the evaluation process we go through and why we decide to end up with Airflow.
• 7:35
  SPEAKER_00: Yeah, I would love to hear more details about how you ended up with Airflow, especially considering it sounds like one of the options was build your own, which is quite challenging, I think.
• 7:44
  SPEAKER_00: So yeah, tell me more about that.
• 7:46
  SPEAKER_02: Yeah.
• 7:47
  SPEAKER_02: So we first did a wide variety of candidates options, and eventually we land up with several options.
• 7:55
  SPEAKER_02: The first one is airflow.
• 7:57
  SPEAKER_02: Yeah, just uh using Airflow assist and this dedicate old orchestration heavy lifting to Airflow.
• 8:04
  SPEAKER_02: The second one is salary.
• 8:05
  SPEAKER_02: Airflow is built on top of salary, yeah.
• 8:08
  SPEAKER_02: So it's much more lightweight options compared to Airflow because you directly just pick whatever you need and build meeting functionality that is needed.
• 8:16
  SPEAKER_02: And the overall code base would be smaller, and the performance will be just if you only care about performance, that's much better than Airflow.
• 8:25
  SPEAKER_02: And the third option is another framework, it's a temporal, it's a Raster-based orchestration framework.
• 8:32
  SPEAKER_02: And we had several evaluation criteria.
• 8:35
  SPEAKER_02: So we are mainly, I'm trying to think of, I'm trying to recall what was the criteria.
• 8:41
  SPEAKER_02: So we basically do that this evaluation matrix with several evaluation uh criteria.
• 8:47
  SPEAKER_02: For example, the first one is performance and overhead, and the second one is how comprehensive it is this framework has for support for observability.
• 8:58
  SPEAKER_02: And the third one is the maturity of the ecosystem, like third-party integrations, and specifically we care about Kubernetes support and how mature it is.
• 9:07
  SPEAKER_02: And the fourth one is our internal expertise, which is do we have any kind of internal operational experience with framework X, Y, Z?
• 9:17
  SPEAKER_02: We don't want to step into like a minefield and then realize everything blows up.
• 9:22
  SPEAKER_02: I guess one more criteria is we are a quite small team and we are tasked with a big ask.
• 9:29
  SPEAKER_02: The time to market or how much development effort and how we can focus to deliver this end goal is also a plus factor.
• 9:36
  SPEAKER_02: The specific consideration, you are asking why we didn't go down the path of building everything by ourselves.
• 9:43
  SPEAKER_02: As a matter of fact, in other projects or other teams, we have used salary before.
• 9:48
  SPEAKER_02: And those are for very latency-sensitive workload, like where we did a comprehensive high-to-head comparison between airflow orchestration overhead versus the other team's implementation, very lightweight orchestration-based salary.
• 10:05
  SPEAKER_02: And of course, the difference is quite noticeable from seconds of orchestration overhead to milliseconds of orchestration overhead.
• 10:12
  SPEAKER_02: However, you give up a lot in this process.
• 10:15
  SPEAKER_02: Like a lot about airflow because of the ecosystem and specifically about the observability side of things.
• 10:24
  SPEAKER_02: Like you can just once you get it up and running, you can go there and check the history of the workload and check the workload logs.
• 10:33
  SPEAKER_02: If something goes wrong, you can see where which step it went wrong and go to the corresponding logs or go to the Kubernetes cluster to figure out.
• 10:41
  SPEAKER_02: Those are the things that we need.
• 10:43
  SPEAKER_02: If we went down to the salary route, we will have to implement everything ourselves.
• 10:49
  SPEAKER_02: And given our resource constraint, we should spend most of our time building the integrations with our internal component.
• 10:57
  SPEAKER_02: Like how to make the streaming design for streaming real-time computer engine, how to make it work in batch mode.
• 11:03
  SPEAKER_02: And our ETR injection to database, how can we modify it so that we can run it with a batch of outputs?
• 11:10
  SPEAKER_02: With the additional overhead, like going from seconds to tens, hundreds of milliseconds, this definitely a good class for the lightweight implementation, but it's not a deal breaker for us because if it's a batch workload, typically it sometimes can take hours to finish if it's a large customer's data, right?
• 11:29
  SPEAKER_02: Yeah.
• 11:29
  SPEAKER_02: Unless you're under a pretty tight constraint about a return to answer in five seconds, then we need to optimize that.
• 11:36
  SPEAKER_02: But we found that in the streaming pipeline.
• 11:38
  SPEAKER_02: There's a different ask, different requirement for our batch workflow.
• 11:41
  SPEAKER_02: I forgot to mention the other comparison.
• 11:43
  SPEAKER_02: We talk about the airflow versus the salary, and the other comparison is another framework for temporal.
• 11:50
  SPEAKER_02: And they're also pretty fast and that they have a variety of support.
• 11:55
  SPEAKER_02: However, what draws us to the low side is with the ecosystem of Airflow is much more mature and way bigger than the new up and coming contenders.
• 12:06
  SPEAKER_02: Yeah, it turns out to be very helpful.
• 12:08
  SPEAKER_02: If you want to work with GCS integration, if you want to work with HDFS integration, it's all pretty standard, and there is answer out there to help you get quickly started.
• 12:17
  SPEAKER_02: Especially in this day and age with the coding agents, the more training data you have, the much better.
• 12:23
  SPEAKER_02: They will greatly accelerate our speed to deliver new features or new workflows.
• 12:30
  SPEAKER_01: This episode is brought to you by Astronomer, creators of Astro, the leading fully managed airflow service designed for the modern data stack.
• 12:40
  SPEAKER_01: Whether you're orchestrating large language models, implementing a data mesh architecture, or streamlining complex workflows, Astro empowers your team to focus on innovation, not infrastructure.
• 12:51
  SPEAKER_01: To see how you can maximize Airflow's potential within your data ecosystem, visit www.astronomer.io.
• 13:00
  SPEAKER_00: Batch orchestration definitely becoming more and more important with these more advanced use cases that it's supporting.
• 13:08
  SPEAKER_00: And I love hearing that the ecosystem and community around Airflow was one of the reasons you chose it as with yeah, being able to support it from people having the background of already using this tool.
• 13:18
  SPEAKER_00: I can definitely imagine that it's a lot easier to use Airflow than to set up your own with salary.
• 13:26
  SPEAKER_00: Even just setting salary up for some open source Airflow users can be challenging.
• 13:31
  SPEAKER_00: So building a platform around it, I think, would be quite difficult.
• 13:34
  SPEAKER_00: And as you mentioned, maybe not performance, performance increases that are needed for badge pipeline.
• 13:41
  SPEAKER_00: So that's really interesting.
• 13:43
  SPEAKER_00: In terms of how your platform is set up now, curious, are the internal users, like the folks on your team, interacting with Airflow directly, or is it used purely as a backend in the architecture that you've set up?
• 13:55
  SPEAKER_02: Yeah, I guess I will say it's a mixture of both.
• 13:59
  SPEAKER_02: As I mentioned, we have a standalone Oxstriction services, web service that's provided the API endpoint to evoke the workflow execution.
• 14:07
  SPEAKER_02: And Airflow is from the architectural point of view, it's behind the scenes.
• 14:12
  SPEAKER_02: So the web service directly invoke, create new workflow inside Airflow.
• 14:17
  SPEAKER_02: However, because the built-in functionalities, the dashboard, the monitoring, the log exploration of Airflow, we all also open up the Airflow web page to our internal users.
• 14:30
  SPEAKER_02: So if they trigger a workload execution, then they can just log into this Airflow dashboard and see how it's going.
• 14:38
  SPEAKER_02: And if it fails, which step failed, and just notify us, let us know, and we can go fix it.
• 14:44
  SPEAKER_02: Yeah.
• 14:45
  SPEAKER_02: So they have direct visibility into the Airflow execution.
• 14:48
  SPEAKER_00: Got it.
• 14:49
  SPEAKER_00: So they interact with the UI, but they're not writing their own DAGs.
• 14:52
  SPEAKER_00: That's all managed by how the platform is set up.
• 14:55
  SPEAKER_02: Right, exactly.
• 14:56
  SPEAKER_02: If there's a new ask for a new type of workflow, they communicate with us.
• 15:01
  SPEAKER_02: We read out a design of the high-level task dependencies, and then we go implement those.
• 15:06
  SPEAKER_02: If something is needed, we implement an adapter to make a new functionality or new components compatible with Airflow tasks.
• 15:16
  SPEAKER_00: Got it.
• 15:16
  SPEAKER_00: Yeah, this is a pattern that I've heard about quite a lot, actually.
• 15:20
  SPEAKER_00: And I'm curious if it has helped you be more efficient and be able to move faster since a lot of the end users don't have to write their own pipelines and you've set it up in a way that is more kind of automated.
• 15:32
  SPEAKER_02: Right.
• 15:32
  SPEAKER_02: Yeah, definitely.
• 15:34
  SPEAKER_02: Because we manage the implementation, the different adapters implementation for our internal components.
• 15:39
  SPEAKER_02: So we know what's there and what's missing.
• 15:42
  SPEAKER_02: And if it's a new workload request coming in, we can reuse all the existing components because we know the inside out of what's in our code base.
• 15:51
  SPEAKER_02: You can imagine if someone from other team they come up, they want to design a new end-to-one workload, and they might tend to implement everything themselves, which is counterproductive because we already have the building blocks.
• 16:05
  SPEAKER_02: We have an adapter to pull our central configuration services.
• 16:11
  SPEAKER_02: We already have capability to do that, and we already have a way to emote batch mode execution of our compute engine.
• 16:17
  SPEAKER_02: And then we know the quirks and features of all these existing components.
• 16:22
  SPEAKER_02: And then we can make it much faster and get everything, get something up and running like in less a day.
• 16:27
  SPEAKER_02: Yeah.
• 16:28
  SPEAKER_00: Got it.
• 16:29
  SPEAKER_00: Yeah, that makes sense.
• 16:30
  SPEAKER_00: And now that you've been working with this platform for a while, are there any optimizations that you're looking to make in the future?
• 16:36
  SPEAKER_02: Yes, there are quite a few optimizations at which we have implemented in our own installation or in our own airflow instances.
• 16:46
  SPEAKER_02: Chances are there's something already out there that we just haven't integrated yet because of the ecosystem, the huge variety of ecosystems.
• 16:54
  SPEAKER_02: But there's just something, some missing gaps in our current implementation.
• 16:59
  SPEAKER_02: For example, the first thing is the user management and integration with our LDAP services.
• 17:07
  SPEAKER_02: As I mentioned, we open up our web dashboard to convey engineers and convival users, they can monitor the workload.
• 17:14
  SPEAKER_02: However, inside the company, we use LDAP and everybody has an account there already.
• 17:20
  SPEAKER_02: It's just an extra hurdle we need to figure out how to give them access.
• 17:24
  SPEAKER_02: We have to create an Airflow account for some of the different set of credentials.
• 17:29
  SPEAKER_02: It would be better if more seamless if they can just directly log in with their own LDAP credentials.
• 17:35
  SPEAKER_02: And the second optimizer I wish to see is more easy-to-use integration with the secret management services.
• 17:44
  SPEAKER_02: For example, we use Vault internally, we store all our sensitive credentials there.
• 17:48
  SPEAKER_02: However, for our Airflow, we leverage their variables and connections, specifically connections to store the credentials.
• 17:55
  SPEAKER_02: It's just extra hoop we need to jump through.
• 17:58
  SPEAKER_02: Like when we need to when we have a oh, here's a service account for this GCS project, and we have to copy that from the lot to our Airflow cluster configuration.
• 18:10
  SPEAKER_02: Yeah.
• 18:11
  SPEAKER_02: And it makes synchronization that a bit difficult, but it's manageable.
• 18:16
  SPEAKER_00: Yeah, for sure.
• 18:17
  SPEAKER_00: So you haven't set up the secrets backend yet.
• 18:20
  SPEAKER_02: No, not yet.
• 18:21
  SPEAKER_02: Yeah.
• 18:22
  SPEAKER_02: And the third one, the final one, is probably how maybe inside or more easy to use scheduling algorithm for the scheduler when you have multiple task instances.
• 18:34
  SPEAKER_02: We have multiple workloads share the same airflow instance and they create their own task instances.
• 18:41
  SPEAKER_02: And there a lot of them are queuing or pending.
• 18:45
  SPEAKER_02: Rather than relying on first in, first out, first come, first serve this scheduling algorithm.
• 18:50
  SPEAKER_02: We might want to have different weight or different, more customized scheduling.
• 18:55
  SPEAKER_02: For this workload, we want to make sure we got the process as soon as possible.
• 18:59
  SPEAKER_02: And for this batch for the other workload, they can wait and just make some progress every few hours.
• 19:11
  SPEAKER_00: Yeah, definitely.
• 19:12
  SPEAKER_00: So using like a priority weight or something on your tasks.
• 19:16
  SPEAKER_00: I think is what it's called in airflow.
• 19:18
  SPEAKER_00: I think.
• 19:19
  SPEAKER_02: Possibly.
• 19:20
  SPEAKER_02: We looked into that.
• 19:21
  SPEAKER_02: Even that is not maybe 100% fit for what we want to achieve.
• 19:25
  SPEAKER_02: It's a very tailored, it may be hard to calculate the priority weight for different task instance.
• 19:30
  SPEAKER_02: This is like for this workload, what's the SLA and how far progress are we on?
• 19:36
  SPEAKER_02: And therefore, the other workload-specific scheduling in the same resource tool.
• 19:41
  SPEAKER_00: Okay, got it.
• 19:42
  SPEAKER_00: That makes sense.
• 19:43
  SPEAKER_00: Maybe a related question.
• 19:44
  SPEAKER_00: I always like to end my episodes with a standard final question to folks, and that is what would you most like to see from the Airflow project in the near future?
• 19:53
  SPEAKER_00: So maybe it would help with some of your optimizations, or maybe not.
• 19:56
  SPEAKER_02: Yeah.
• 19:57
  SPEAKER_02: The biggest thing I wish we can have is the version control or history tracking of the values we set, variables and connections.
• 20:07
  SPEAKER_02: Yeah, because for example, in variables, we store different configurations and the pod resources is also part of the variable values.
• 20:15
  SPEAKER_02: And if maybe make some tweaks or adjustments, the history is lost, at least from the UI.
• 20:21
  SPEAKER_02: I'm not sure about inside under the hood whether they still store that in the database.
• 20:24
  SPEAKER_02: Yeah.
• 20:25
  SPEAKER_02: And also for credentials, it will be better to have a visibility into what had changed and what in the past and who has changed it from one value to what.
• 20:36
  unknown: Yeah.
• 20:37
  SPEAKER_00: That's really interesting.
• 20:38
  SPEAKER_00: I would imagine some of that information exists in the audit log, although I'm not positive.
• 20:43
  SPEAKER_00: But it sounds like what you're looking for is more like the DAG versioning feature that came in our flow three, but for connections and variables as well.
• 20:51
  SPEAKER_02: Yeah, some may also refer to the Git history, like on GitHub, you just click on the history and see who made the commit that changed it.
• 20:58
  SPEAKER_00: I can totally see where that would be helpful.
• 21:01
  SPEAKER_00: Okay, cool.
• 21:02
  SPEAKER_00: Hun, thank you for going through all of that.
• 21:03
  SPEAKER_00: This has been really interesting hearing more about your use case.
• 21:06
  SPEAKER_00: Just to close us out, what is the best way for folks to get in touch with you?
• 21:09
  SPEAKER_02: Yeah, feel free to connect with me on LinkedIn, send me a message.
• 21:13
  SPEAKER_02: I will get notification.
• 21:14
  SPEAKER_02: I'm not very active, but I will get notification.
• 21:18
  SPEAKER_00: Okay, great.
• 21:18
  SPEAKER_00: We will include that in the show notes.
• 21:20
  SPEAKER_00: Thank you again for coming on.
• 21:21
  SPEAKER_00: It's been really great talking to you.
• 21:23
  SPEAKER_02: Yeah, thank you for having me.
• 21:27
  SPEAKER_01: Thanks for joining us for the latest episode of the Data Flowcast, mastering airflow for data engineering and AI.
• 21:34
  SPEAKER_01: You'll find links in the show notes to any resources mentioned in today's show.
• 21:37
  SPEAKER_01: If you're enjoying our podcast, please subscribe so you never miss an episode and leave us a five star review.