Latest / The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI / Cutting-Edge Data Engineering at Teya with Alexandre Magno Lima Martins

Transcript

Cutting-Edge Data Engineering at Teya with Alexandre Magno Lima Martins · The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI

• 0:00
  SPEAKER_02: If there is any way of like maybe separating secrets per teams in Airflow, of course, you would need to create the concept of teams or like domains inside Airflow to be able to create a secret per team.
• 0:14
  SPEAKER_02: So those like minor things that actually if you want to use the Airflow in a multi-tenant way, right now you would need to deal with these things yourself.
• 0:23
  SPEAKER_02: And uh if it comes with the next release in the future ones, it would be very good.
• 0:30
  SPEAKER_00: You're listening to the Data Flowcast, where we unlock the secrets of mastering airflow for data engineering in Teya.
• 0:39
  SPEAKER_01: Hello everyone, welcome, welcome to the Data Flowcast.
• 0:42
  SPEAKER_01: I'm your host, Mark Marty, and today I'm joined by Alexandre Manu Limba Martins, senior data engineer at Teya.
• 0:50
  SPEAKER_01: And the first thing that I would like to ask you, Alexandre, is how are you doing?
• 0:54
  SPEAKER_02: Hey Mark, how are you?
• 0:56
  SPEAKER_02: Yeah, I'm very happy to be here actually to talk a little bit about what we did in Teya and I don't know, help foster the Airflow community be a little bit.
• 1:06
  SPEAKER_02: So very happy.
• 1:07
  SPEAKER_02: Awesome.
• 1:08
  SPEAKER_01: I'm very excited because I know that you did a couple of medium blog posts that are truly amazing.
• 1:14
  SPEAKER_01: And I can't recommend everyone enough to check out those articles because they're truly amazing to learn how to handle Airflow.
• 1:21
  SPEAKER_01: We'll talk about that later in the episode, right?
• 1:24
  SPEAKER_01: So just before getting started, I would like to briefly explain the primary takeaways that you will get from this episode.
• 1:32
  SPEAKER_01: So you will be able to you will know how to make airflow self-serve for users.
• 1:37
  SPEAKER_01: We'll talk a little bit about that.
• 1:39
  SPEAKER_01: Also, how to generate DAGs based on dbt models.
• 1:43
  SPEAKER_01: I'm sure that there are many dbt users that are listening to us.
• 1:47
  SPEAKER_01: And last but not least, some useful tips around optimization and executors.
• 1:54
  SPEAKER_01: Just can you actually start by talking a little bit about your role at Teya and what you do?
• 2:01
  SPEAKER_02: Okay, cool.
• 2:02
  SPEAKER_02: I'm a data engineer at Teya, so I'm more like responsible to build a data platform.
• 2:10
  SPEAKER_02: Actually, so what we are uh calling nowadays a data platform engineer, basically responsible for deploying and like maintaining all our data infrastructure components, so we're related to the extraction of data, storage, and also transformation of data as well.
• 2:30
  SPEAKER_01: Okay.
• 2:31
  SPEAKER_01: And just to have a better idea of how you run Airflow in your organization, like what kind of use cases do you need to manage today?
• 2:40
  SPEAKER_02: The Airflow is our primary orchestrator, right?
• 2:43
  SPEAKER_02: Our main use case, I would say, it's CBT, the orchestration of the DBT project that really powers all the analytics downstream.
• 2:53
  SPEAKER_02: But also we have a couple of DAX running that is part of our machine learning uh initiative, fraud as well, so fraud prevention, and some custom like housekeeping workloads as well that we use to run in Airflow, but the primary use case would be DBT.
• 3:10
  SPEAKER_01: DBT, okay.
• 3:10
  SPEAKER_01: That's super interesting.
• 3:11
  SPEAKER_01: And how many teams are working on Airflow in your organization?
• 3:15
  SPEAKER_01: Do you have many different teams working on it?
• 3:18
  SPEAKER_02: Yeah, currently it's being used by the data teams or the teams that relate to data somehow.
• 3:25
  SPEAKER_02: Okay.
• 3:25
  SPEAKER_02: So it's not just the data engineering teams.
• 3:28
  SPEAKER_02: You also have like data scientists, machine learning engineers also using it.
• 3:33
  SPEAKER_02: We want in the future to actually expand this usage to the other engineering teams inside the e-content.
• 3:42
  SPEAKER_02: That would be your traditional software engineer that has a use case for some sort of like workload orchestration.
• 3:51
  SPEAKER_02: But we're not there yet.
• 3:52
  SPEAKER_02: That's a plan for the future.
• 3:54
  SPEAKER_01: Okay.
• 3:55
  SPEAKER_01: Yeah, it takes time.
• 3:56
  SPEAKER_01: It takes time.
• 3:56
  SPEAKER_01: And actually, speaking of that, is there anything that you have achieved or implemented with Airflow that you are really proud of?
• 4:03
  SPEAKER_01: Because, as you said, you have many different teams and at least like profiles that work on your Airflow instance.
• 4:10
  SPEAKER_01: So I know that you are working on something pretty, pretty great.
• 4:13
  SPEAKER_01: Can you tell us more about that?
• 4:15
  SPEAKER_02: Yeah, so I would say that for us, the most exciting thing would be the self-service fashion that we did the whole thing, that we did the whole platform, especially when we talk about our main use case, which is DBT, right?
• 4:30
  SPEAKER_02: We have a DAG generation pipeline that builds on top of the DBT project.
• 4:36
  SPEAKER_02: So when data analysts or even project managers that know a little bit of SQL and can build DBT models, they can see their models being translated into Airflow DAGs, go into Airflow and run those models as Airflow DAGs without requiring writing the DAGs per se, like without requiring knowing what an Airflow DAG is, right?
• 5:04
  SPEAKER_02: Besides like how to we trigger it, how to clear a task.
• 5:09
  SPEAKER_02: So this is also this interaction with the Airflow UI is very important, but they don't need to know like how to actually write a DAG, they don't need to even know Python.
• 5:19
  SPEAKER_01: Wow.
• 5:20
  SPEAKER_01: So actually, I truly need to know more about that because I know that DAG generation is something that many users struggle with or even wonder how to uh implement that in their uh company.
• 5:33
  SPEAKER_01: So can you tell us more a little bit about how you generate those DAGs?
• 5:37
  SPEAKER_01: For example, if I'm a data scientist and I know, as you said, I don't I don't know how to write a DAG in Airflow.
• 5:44
  SPEAKER_01: What should I do with your uh solution?
• 5:47
  SPEAKER_02: So, yeah, our data generation, our DAG generation pipelines more focused on the dbt project.
• 5:54
  SPEAKER_02: For the other use cases, like data scientists, machine learning engineers, they are usually more comfortable with Python with the concept of YA DAG.
• 6:04
  SPEAKER_02: So they can generate DAGs freely as they basically want to, but of course, we also enforce some guidelines in DAG development in our CICD flows.
• 6:15
  SPEAKER_02: It needs to be tags, it needs to be the the DAG needs to have the appropriate tags, a description, the access, the control layers as well.
• 6:24
  SPEAKER_02: So we enforce through CI CD checks a little bit of minimum things that the DAG really needs to have.
• 6:33
  SPEAKER_02: But when we talk about the DBT project, we have an entirely automated flow of tag generation, right?
• 6:39
  SPEAKER_02: So we basically read off from the project, we parse the dbt manifest, and we have a separate file that we created that lives inside the dbt project where people can like register their models to be deployed.
• 6:55
  SPEAKER_02: So they can actually group models however they want, basically.
• 6:59
  SPEAKER_02: You can, if you want to deploy a single model, so like a single model becomes a DAG, or you can deploy how many models you want inside a DAG.
• 7:08
  SPEAKER_02: So you can like slice and dice your dbt project in any way you like, basically.
• 7:14
  SPEAKER_02: What usually uh people do is they separate the project for domains, right?
• 7:20
  SPEAKER_02: So each DAG, each domain will have its own set of DAGs.
• 7:24
  SPEAKER_02: What we call model groups.
• 7:27
  SPEAKER_02: It's like usually models that need to be run and scheduled together, like that they make sense as a single unit.
• 7:35
  SPEAKER_02: And all the models inside this model group is going to run as a single DAG in Airflow.
• 7:41
  SPEAKER_02: And inside this DAG, it will follow the normal DBT lineage.
• 7:45
  SPEAKER_02: So you actually the tasks inside the DAG will follow the same dbt lineage for model the execution.
• 7:55
  SPEAKER_02: And we also did a thing that Cosmos did as well, which is to uh split run and test into set into separate tasks.
• 8:03
  SPEAKER_02: So every model will run followed by tests.
• 8:07
  SPEAKER_01: Okay.
• 8:08
  SPEAKER_01: So basically, if I'm a data analyst, I actually don't interact with DAGs directly, I just interact with a file to say that this dbt model should be exactly generally like should be generated through a DAG, and that's all I do.
• 8:23
  SPEAKER_02: That's it.
• 8:23
  SPEAKER_02: And you you also on that file pass like the access control layers and tags and like whatever you want to have a more friendly the experience when like I searching for this DAG in Airflow.
• 8:37
  SPEAKER_02: So people can like you actually write the DAG description, you apply tags, and they can determine which roles have the access to view and uh trigger this DAG.
• 8:48
  SPEAKER_02: So all in this deployment file that we call it.
• 8:52
  SPEAKER_01: And this is super interesting.
• 8:53
  SPEAKER_01: This deployment file, what it is it's a YAML file?
• 8:55
  SPEAKER_01: Is it a YAML file?
• 8:57
  SPEAKER_01: Okay, here we go.
• 8:57
  SPEAKER_01: And so you have a script that goes over those deployment files, or is it a DAG that goes over those deployment files?
• 9:04
  SPEAKER_02: Is it a script?
• 9:05
  SPEAKER_02: Is it actually part of our DAG generation project, right?
• 9:09
  SPEAKER_02: Because this is one thing as well that we needed to consider when I starting the project, like how to generate DAGs.
• 9:17
  SPEAKER_02: And then of course we have those two methods, which is the single file method or the like multiple file methods, right?
• 9:24
  SPEAKER_02: So we started with the single file method, but then we it didn't work very well because as the dbt project like was growing and more and more models were added to actually parse the dbt project at your runtime, it was like your flow was like struggling to do that because the manifest gets like bigger every time we need to read the manifest.
• 9:46
  SPEAKER_02: So we actually changed that to a multiple file method where the DAGs are generated before pushing it to Gearflow.
• 9:55
  SPEAKER_02: So DAGs are basically generated in CICD after merge.
• 10:00
  SPEAKER_02: So after the checks are like applied and we basically validate, then we generate all DAGs based off from the DT project, and then we push them to a bucket where basically Gearflow is just going to read as a normal DAG, right?
• 10:18
  SPEAKER_02: So we are using this multiple file methods.
• 10:21
  SPEAKER_01: That's really amazing because so you actually saw that using the multi-file method was delivered a better performance than if you were using the single file method.
• 10:33
  SPEAKER_01: You saw that in actually, how did you see that it didn't work?
• 10:36
  SPEAKER_01: Did you see that in in your metrics?
• 10:39
  SPEAKER_02: Yes, yeah, we were monitoring some airflow metrics, especially the bag parse time, like and the scheduler loop time.
• 10:48
  SPEAKER_02: So we saw that the scheduler, the average scheduler loop time was like increasing a lot as the dbt project was growing, just because like the manifest was like getting larger, and like in every so like basically every time that the a flow it parses DAG, it had to run this uh script that generates all the DAG, so you had to parse it all over again.
• 11:14
  SPEAKER_02: And when we changed to the multiple file method, the scheduler loop time just yellow.
• 11:19
  SPEAKER_02: Wow, and the scheduler itself became more performed.
• 11:23
  SPEAKER_01: Wow.
• 11:23
  SPEAKER_01: Out of curiosity, did you try to trick some airflow settings first before going to this method, or did you just decide to use the multifile method?
• 11:35
  SPEAKER_02: Yeah, one option was to use the separate DAG parser, right?
• 11:40
  SPEAKER_02: That the Airflow has to not parse the DAGs using the uh scheduler, but to have a separate component for that.
• 11:47
  SPEAKER_02: That could be one option, but we decided to go straight to the multiple file methods because that was the recommended method for this case.
• 11:55
  SPEAKER_02: Yeah, we just went uh straight with it.
• 11:58
  SPEAKER_02: Okay.
• 11:59
  SPEAKER_00: This episode is brought to you by Astronomer, creators of Astro, the leading fully managed airflow service designed for the modern data stack.
• 12:09
  SPEAKER_00: Whether you're orchestrating large language models, implementing a data mesh architecture, or streamlining complex workflows, Astro empowers your team to focus on innovation, not infrastructure.
• 12:21
  SPEAKER_00: To see how you can maximize airflow's potential within your data ecosystem, visit www.astronomer.io.
• 12:28
  SPEAKER_01: That's amazing to hear because again, some people advocate for the single file method, some people advocate for the other method.
• 12:36
  SPEAKER_01: At the end of the day, truly, and as you exactly, as you said, you need to look at your metrics, test different things, and see what works best for you and your use case.
• 12:45
  SPEAKER_01: So that's really good to hear.
• 12:47
  SPEAKER_01: And just to circle back on the data on the DBT models, I imagine that when you have the DBT models, you had some challenges right along the way to generate those DAGs based on the DBT models.
• 13:00
  SPEAKER_01: I know that you have an interesting story about the executor, and more specifically, the Kubernetes executor.
• 13:07
  SPEAKER_01: So tell you, can you tell us more about the Kubernetes executor, which is an extremely popular executor?
• 13:13
  SPEAKER_01: But in your case, why it might not have been the best option?
• 13:18
  SPEAKER_02: Yeah, sure.
• 13:19
  SPEAKER_02: That was a fun one, actually.
• 13:21
  SPEAKER_02: Oh yeah.
• 13:22
  SPEAKER_02: So because like here we run Airflow on Kubernetes, right?
• 13:26
  SPEAKER_02: So of course, our first option would was like to yeah, let's start by using the Kubernetes executor, as we are running in Kubernetes.
• 13:37
  SPEAKER_02: It's the most logical sense, logical choice.
• 13:40
  SPEAKER_02: And then we started doing that, but what happened was that as our dbt project became larger, we started to have like many tasks because like we orchestrate like more or less on average 2,000 models.
• 13:54
  SPEAKER_02: So at our peak, we had like over 3,000 models.
• 13:59
  SPEAKER_02: So you can imagine the number of tasks.
• 14:03
  SPEAKER_02: And in if you use the Kubernetes executor, one task equals one pod, right?
• 14:09
  SPEAKER_02: So you can imagine the number of pods that we were uh scheduling per day in our Kubernetes cluster.
• 14:17
  SPEAKER_02: That's not necessarily the problem.
• 14:19
  SPEAKER_02: The problem was that our dbt models usually run very fast, they are like usually very short-lived uh tasks, right?
• 14:27
  SPEAKER_02: They run for one minute, two minutes, and they are done.
• 14:31
  SPEAKER_02: And we have some peak times that like group large groups of models need to actually be scheduled in that same time.
• 14:39
  SPEAKER_02: So, what happened was that like in some specific moments, we had a large, large spike in the number of tasks, thus in the number of pods, right?
• 14:50
  SPEAKER_02: So our Kubernetes cluster that was running with I don't know, like 100 pods, actually it blew up to a thousand pods in a matter of five minutes.
• 15:01
  SPEAKER_02: And then what happens is that we also, the cloud, the engineering team in our company, they deploy a tool, which is called uh eCarpenter, which is Kubernetes resource management tool.
• 15:12
  SPEAKER_02: So basically, what iCarpenter does is that it finds Kubernetes nodes that are yonder utilized and like reschedule pods on those nodes to actually lower the total the amount of nodes, saving you cost like at the end of the day, right?
• 15:31
  SPEAKER_02: And as our tasks were very fast, we could see that we were like scaling the number of nodes in the cluster, but like very quickly, those nodes were becoming sub-utilized because all the pods scheduled on those nodes were like being completed.
• 15:50
  SPEAKER_02: However, some jobs took more than one minute or uh two minutes, right?
• 15:55
  SPEAKER_02: Some jobs were like long-living jobs.
• 15:57
  SPEAKER_02: And what happens that iCarpenter, when trying to deschedule those nodes, were like killing those jobs in favor of like, I know let me schedule this pod in like the other node.
• 16:08
  SPEAKER_02: But it was a running BBT job.
• 16:12
  SPEAKER_02: It was a problem.
• 16:13
  SPEAKER_02: It was a problem because we started to have like many jobs being killed because of these spikes in the number of pods and thus in the number of cluster nodes as well.
• 16:23
  SPEAKER_02: And then to solve that, we decided to migrate to the salary executor.
• 16:27
  SPEAKER_02: So in salary, we don't have this problem because we have like long-living worker nodes, and then they can just like accept new tasks, right?
• 16:37
  SPEAKER_02: And those tasks can run very fast.
• 16:40
  SPEAKER_02: We didn't have this problem when we migrate to salary.
• 16:43
  SPEAKER_01: That's a super interesting story.
• 16:45
  SPEAKER_01: And actually, so that's a very good point of why the salary executor is most of the time better than the Kubernetes executor.
• 16:52
  SPEAKER_01: It's because it can run your tasks faster, especially if you have short-living tasks.
• 16:58
  SPEAKER_01: It's much better to use the salary executor in this case.
• 17:01
  SPEAKER_01: But then also a downside of the salary executor is that you have some workers waiting, consuming resources while waiting, and so they are actually doing nothing.
• 17:12
  SPEAKER_01: Are you thinking of have you ever thought of how to fix this maybe issue?
• 17:17
  SPEAKER_01: Maybe it's not one yet, but have you ever thought of doing something to improve this downside, maybe?
• 17:24
  SPEAKER_02: Yeah, one way to actually improve that is by using the uh KEDA, the autoscaler.
• 17:30
  SPEAKER_02: Yeah, so by using KEDA, you can like uh scale in or out the number of like salary workers.
• 17:37
  SPEAKER_02: So in periods that you don't have as many jobs running, you can run with like just one or maybe two workers.
• 17:44
  SPEAKER_02: But like when you have like more and more uh tasks being scheduled, you can use that metric, which is the number of tasks in the queued state, right?
• 17:54
  SPEAKER_02: So actually as your scale as your scale out metric in KEDA, so you like increase the number of workers.
• 18:02
  SPEAKER_02: That's a very good way out as well.
• 18:04
  SPEAKER_01: Yeah, no, I agree, I agree.
• 18:05
  SPEAKER_01: Definitely Keda is one solution.
• 18:07
  SPEAKER_01: Also, I I heard about I've never tried it yet, but I heard about the Celery Kubernetes executor, I believe, which is a mix of both worlds.
• 18:16
  SPEAKER_01: I know that many people are looking into it.
• 18:18
  SPEAKER_01: I don't know, did you look into it yet?
• 18:20
  SPEAKER_02: Or it's on our backlog, but yeah, we want to do that as well.
• 18:26
  SPEAKER_02: That would require a little bit of investigation and testing as well, the side on the R side because it's uh a big change, but we know that it can be yet improved.
• 18:36
  SPEAKER_02: Although, like right now, when we need to run very uh custom tasks that would require a very custom Python packages, usually related to data science or machine learning, we can actually and we do use the Kubernetes pod operator.
• 18:54
  SPEAKER_02: So for those cases, we spin up a totally separate pod to have runtime isolation as well.
• 19:01
  SPEAKER_02: So we also use it.
• 19:03
  SPEAKER_01: Yeah, that's a pretty popular operator.
• 19:05
  SPEAKER_01: Many actually, I don't know if you do that, but many companies have a layer on top of the Kubernetes pod operator so that this layer defines the amount of CPUs or memories and so on based on the use case.
• 19:19
  SPEAKER_01: So you don't interact with, for example, as a data scientist, I'm not gonna interact with the Kubernetes pod operator directly, but I will interact with a custom operator built on top of it.
• 19:30
  SPEAKER_01: So I don't have control over the resources and so on.
• 19:32
  SPEAKER_01: Is it something that you are doing or not yet?
• 19:35
  SPEAKER_02: That's a very good idea.
• 19:36
  SPEAKER_02: We don't do that, but it makes a lot of sense because to create this layer of the abstraction right on top of the Kubernetes pod operator, it makes a lot of sense.
• 19:45
  SPEAKER_02: We don't do that just because the use cases that we do need it, they're like very very custom, and the people that use it know what they are doing basically.
• 19:55
  SPEAKER_02: They're like very comfortable with it.
• 19:58
  SPEAKER_02: Yeah.
• 19:58
  SPEAKER_01: Good for you.
• 20:00
  SPEAKER_01: Good for you that they know what they do.
• 20:02
  SPEAKER_01: Okay, just also very quickly, because obviously we talked about dbt and generating DAGs using dbt models.
• 20:09
  SPEAKER_01: Why aren't you uh using Cosmos?
• 20:12
  SPEAKER_02: That's a very fair question.
• 20:14
  SPEAKER_02: So our project started like more than a little bit more than two years ago.
• 20:18
  SPEAKER_02: So at that time, I don't think Cosmos was released, I can be wrong, but if it was released, it was not very uh popular.
• 20:27
  SPEAKER_02: So we didn't even uh consider it as a choice, and as well, because as far as I know, like Cosmos doesn't have a way to actually deal with cross-dEG dependencies or cross-dig communication.
• 20:41
  SPEAKER_02: You can split your DPT project into many DAGs using Cosmos as well.
• 20:47
  SPEAKER_02: However, I don't see how you can connect those DAGs.
• 20:52
  SPEAKER_02: And that's a thing that was a requirement for us when we started building the project that we are running right now.
• 20:58
  SPEAKER_02: Because in our case, if one model from deck A references another model has a dependency on another model that is being scheduled on that B, we are going to create a sensor on that A to repoke for the latest state of the dependent model in that B.
• 21:16
  SPEAKER_02: So we do this activity with sensors.
• 21:18
  SPEAKER_01: Okay, I see.
• 21:19
  SPEAKER_01: That's good to know.
• 21:20
  SPEAKER_01: And that might be uh something that the community will work on for Cosmos to make it better, because uh that's the kind of feedback that we love to hear.
• 21:27
  SPEAKER_01: As a final question, what would you most like to see from the development of the Airflow project in the near future?
• 21:34
  SPEAKER_01: Like, is there any any any feature that you would like to see in Airflow in the next release of Airflow?
• 21:42
  SPEAKER_01: Anything that you would like to see as a new feature or an improvement that will make your life better?
• 21:46
  SPEAKER_02: Yeah, like for us, everything that would improve the ability of using airflow as more like a multi-attendant airflow, right?
• 21:57
  SPEAKER_02: More use cases related to more teams using the same Airflow instance.
• 22:04
  SPEAKER_02: That would be very nice.
• 22:05
  SPEAKER_02: And ways to deal with it, like one very specific example is the usage of uh secrets.
• 22:12
  SPEAKER_02: If there is any way of like maybe separating secrets per teams in Airflow, of course, you would need to create the concept of teams or like domains inside Airflow to be able to create a secret per team.
• 22:26
  SPEAKER_02: So those like minor things that actually if you want to use the Airflow in a multi-tenant way, right now you would need to deal with these things yourself, and uh if it comes with the next release in the future ones, it would be very good.
• 22:41
  SPEAKER_01: Agreed.
• 22:41
  SPEAKER_01: Definitely multi-tenancy has been a topic for a while in Airflow, to say the least.
• 22:46
  SPEAKER_01: Okay, awesome.
• 22:47
  SPEAKER_01: Thank you for joining, Alexandre.
• 22:49
  SPEAKER_01: Honestly, it was I keep saying that, but it was truly amazing.
• 22:52
  SPEAKER_01: And uh, what's the best way for folks to get in touch with you if I want to stay in touch with you?
• 22:58
  SPEAKER_02: Yeah, I guess LinkedIn is would be the best way.
• 23:02
  SPEAKER_02: Yeah, it's just okay.
• 23:03
  SPEAKER_02: Type my name there, you can maybe find it.
• 23:06
  SPEAKER_02: I don't know.
• 23:07
  SPEAKER_02: Yeah, and LinkedIn would be good.
• 23:09
  SPEAKER_01: Awesome.
• 23:10
  SPEAKER_01: Thank you again.
• 23:11
  SPEAKER_01: Thank you for joining the data flowcast.
• 23:13
  SPEAKER_01: Again, it was amazing.
• 23:15
  SPEAKER_01: We a lot of insights, and thank you so much.
• 23:18
  SPEAKER_01: Hope you are uh gonna have a great day and hope to see you soon.
• 23:22
  SPEAKER_02: I enjoy it very much as well.
• 23:24
  SPEAKER_02: Thank you, Mark.
• 23:25
  SPEAKER_02: Thank you, Alexa.
• 23:26
  SPEAKER_02: Bye-bye.
• 23:28
  SPEAKER_00: Thanks for joining us for the latest episode of the Data Flowcast, mastering airflow for data engineering and AI.
• 23:34
  SPEAKER_00: You'll find links in the show notes to any resources mentioned in today's show.
• 23:38
  SPEAKER_00: If you're enjoying our podcast, please subscribe so you never miss an episode and leave us a five-star review.