Latest / The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI / Scaling Airflow at Wix for Analytics and AI with Ethan Shalev

Transcript

Scaling Airflow at Wix for Analytics and AI with Ethan Shalev · The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI

• 0:00
  SPEAKER_01: I think that there's definitely more DAGs that we're migrating now.
• 0:02
  SPEAKER_01: So there's a challenge of scale.
• 0:04
  SPEAKER_01: We're also supporting probably 50% more data engineers now than we did three or four years ago.
• 0:08
  SPEAKER_01: The process itself is really straightforward.
• 0:10
  SPEAKER_01: Like I said, there are a few things that we've encountered along the way.
• 0:12
  SPEAKER_01: Connections not working properly.
• 0:13
  SPEAKER_01: We've also taken the opportunity to migrate from open source airflow to astronomer.
• 0:17
  SPEAKER_01: So that has a whole bunch of other challenges along the way, such as we've gone from hosting our own secrets to using the default secret backend.
• 0:23
  SPEAKER_01: So that provided some other opportunities or challenges for us along the way.
• 0:27
  SPEAKER_01: But we're learning as we go along, and that's the kind of thing that you fix once and then you just keep on going.
• 0:48
  SPEAKER_02: Hi everyone.
• 0:49
  SPEAKER_02: Welcome to the Data Flowcast.
• 0:51
  SPEAKER_02: I'm your host, Kent and Dannis, and today I'm joined by Ethan Chilev, who is a data engineer at Wix.
• 0:57
  SPEAKER_02: Ethan, how are you?
• 0:58
  SPEAKER_01: Great.
• 0:58
  SPEAKER_01: Thanks for having me.
• 0:59
  SPEAKER_01: How are you doing?
• 1:00
  SPEAKER_02: Yeah, I'm really good.
• 1:01
  SPEAKER_02: It's great to have you on.
• 1:02
  SPEAKER_02: Thanks for joining us.
• 1:03
  SPEAKER_02: I'm excited to talk to you today for our listeners for today's podcast.
• 1:07
  SPEAKER_02: We're going to discuss how Wix uses Airflow for analytics and AI, which I think is going to be a really exciting topic.
• 1:13
  SPEAKER_02: But before we dive into that, let's start with some background for folks who might not be familiar.
• 1:18
  SPEAKER_02: Can you tell us about what Wix does?
• 1:20
  SPEAKER_01: So Wix has been around for 19 years.
• 1:22
  SPEAKER_01: Our vision is to be a place where anyone can create their dream online.
• 1:26
  SPEAKER_01: What that means in practice is that we're a one-stop shop where people can create websites for their personal blog or their wedding venue or their small medium business.
• 1:34
  SPEAKER_01: We also have some very large companies who are running their e-com platform on Wix.
• 1:38
  SPEAKER_01: And in recent years, we've dabbled into AI.
• 1:40
  SPEAKER_01: We have a few new AI tools that we're creating and rolling out that allow you to create anything you want, basically apps with a single prompt and things like that.
• 1:49
  SPEAKER_02: That's great.
• 1:50
  SPEAKER_02: I think Wix is definitely one of those companies where you're behind probably a lot of the websites that people interact with in day to day and they have no idea if you're not the ones building it, but definitely a very wide reach.
• 2:02
  SPEAKER_01: Yeah.
• 2:03
  SPEAKER_01: Wix actually operates about 4% of websites on the internet.
• 2:06
  SPEAKER_01: So we're definitely out there.
• 2:08
  SPEAKER_02: Whoa, that is a crazy stat.
• 2:10
  SPEAKER_02: That's really cool.
• 2:11
  SPEAKER_02: So definitely very wide reach.
• 2:13
  SPEAKER_02: So tell me, what do you do and what does your team do?
• 2:16
  SPEAKER_02: And how do you use Airflow?
• 2:17
  SPEAKER_01: So we have over 100 data engineers at Wix providing operational support for the various companies inside of Wix for every one of the products that we offer, enabling our analysts and product managers mostly to get a timely answer to the questions that they need.
• 2:31
  SPEAKER_01: Most of our data comes in through Trino and we're writing it to iceberg tables.
• 2:36
  SPEAKER_01: And we're doing more and more real-time analytics, providing both internal and external users insight into the operations of their own websites or the products that they provide.
• 2:46
  SPEAKER_02: Okay, great.
• 2:46
  SPEAKER_02: And so Airflow and your data engineering teams are supporting both internal analytics for your business as well as actually in-product analytics that your customers use directly.
• 2:57
  SPEAKER_01: Is that right?
• 2:58
  SPEAKER_01: We're dabbling into that more and more, but right now, mostly for internal purposes.
• 3:02
  SPEAKER_01: We're using Airflow to orchestrate scheduling ETLs for our data warehouse tables, where we have a very large suite of tables that we provide for product managers to get the answers that they need.
• 3:15
  SPEAKER_02: Okay, great.
• 3:16
  SPEAKER_02: So it sounds like Airflow is pretty embedded for your team then.
• 3:19
  SPEAKER_01: Absolutely.
• 3:20
  SPEAKER_01: Yeah.
• 3:20
  SPEAKER_02: Yeah, that's awesome.
• 3:22
  SPEAKER_02: And I understand you're currently working on migrating from Airflow two to Airflow three, which, based on the scale you've described, I'm sure is a bit of an undertaking.
• 3:32
  SPEAKER_02: Tell me about that process and how it's going.
• 3:34
  SPEAKER_01: Yeah, so that's been a multi-month process.
• 3:36
  SPEAKER_01: We started pretty much when Airflow three came out in April.
• 3:39
  SPEAKER_01: We started mapping out what we would need to do in order to perform the migration.
• 3:44
  SPEAKER_01: We do have at this point close to 8,000 DAGs running in three different Airflow clusters.
• 3:49
  SPEAKER_01: So it's yeah, it's no small undertaking.
• 3:51
  SPEAKER_01: And my role as a data engineer, I'm also the organization's Airflow tech specialist or tech lead.
• 3:56
  SPEAKER_01: So I've been spearheading this along with our data frameworks team.
• 4:00
  SPEAKER_01: And there's a lot of work that went into preparing the ground for the migration.
• 4:04
  SPEAKER_01: It included things like mapping out what kind of imports, which classes we're using that are no longer supported by Airflow 3, or all sorts of internal libraries that we've developed that maybe we don't need anymore because Airflow 3 now supports them natively, things that we don't want to carry on because we've been carrying a lot of legacy code since Airflow 1.
• 4:22
  SPEAKER_01: And I thought this was a good opportunity to kill that.
• 4:25
  SPEAKER_01: So the process has been going on for several months.
• 4:27
  SPEAKER_01: We've now migrated one entire cluster that runs over 5,000 of our DAGs.
• 4:32
  SPEAKER_01: And we have another cluster that's running 1,200 DAGs, give or take.
• 4:36
  SPEAKER_01: And as of today, we're at about 35% fully migrated.
• 4:40
  SPEAKER_01: We kicked it off literally like two weeks ago, and we're already on track to finish within a few more weeks.
• 4:44
  SPEAKER_02: Oh, that's great.
• 4:46
  SPEAKER_01: Yeah.
• 4:46
  SPEAKER_01: I have to say that the process of migrating was really daunting.
• 4:49
  SPEAKER_01: Like I thought it would be a huge undertaking.
• 4:52
  SPEAKER_01: And you read the documentation, it really says just run rough and it'll take care of everything for you.
• 4:56
  SPEAKER_01: I was like, that can't be that simple.
• 4:58
  SPEAKER_01: In practice, there are a few caveats a little beyond that, but that really does cover 80% of the use cases.
• 5:03
  SPEAKER_01: There are some things that are documented that aren't automated, like deprecated templates and a few other things that we've discovered along the way.
• 5:10
  SPEAKER_01: But all in all, we've really been able to run through this.
• 5:13
  SPEAKER_01: One of the things I've done is basically map out a lot of the challenges and changes that need to be done along the way.
• 5:20
  SPEAKER_01: I've written a playbook to give our data engineers of these are the steps that you need to take and run and make sure that your DAGs run locally.
• 5:26
  SPEAKER_01: And once you're done with that, then you could deploy them.
• 5:28
  SPEAKER_01: One of our data engineers took that and put it into a rule for cursor and built on that a little bit.
• 5:33
  SPEAKER_01: So we now have a cursor rule with a single prompt.
• 5:35
  SPEAKER_01: I wouldn't say a click of a button, but with a single prompt, we give it a directory of DAGs, it runs on them and runs the migration for us.
• 5:41
  SPEAKER_01: And it's been working really well for us.
• 5:43
  SPEAKER_02: Okay, cool.
• 5:44
  SPEAKER_02: And so you've been able to use AI to speed that process up for you a little bit.
• 5:48
  SPEAKER_02: And has that been working better than using the rough sort of auto upgrade or whatever the command is where the dash fix or something like that?
• 5:57
  SPEAKER_01: Yeah, the dash fix is great that doesn't cover all of our use cases because there are things that Rough doesn't catch.
• 6:02
  SPEAKER_01: So one of the first things in the cursor rule is, in fact, to run Rough and then have it fix things that it can.
• 6:08
  SPEAKER_01: But we've also used this opportunity to clear out a whole lot of legacy code and replace certain libraries with other ones.
• 6:14
  SPEAKER_01: So really with just a single command, we were able to do the migration and clean up our code base quite nicely.
• 6:18
  SPEAKER_02: Yeah, that's great to hear.
• 6:20
  SPEAKER_02: I'm curious because you mentioned a little bit about cleaning up some of the code from Airflow One and wondering if you also went through the Airflow one to two migration.
• 6:27
  SPEAKER_02: I'm curious what the comparison is like there in terms of ease of use.
• 6:31
  SPEAKER_02: And of course, you didn't have cursor back in the one to two migration days.
• 6:35
  SPEAKER_02: So hopefully that's a big part of the help.
• 6:37
  SPEAKER_01: Yeah, in both cases, we did a blue-green migration.
• 6:40
  SPEAKER_01: We created a new GitHub repository or a new deployment for it.
• 6:43
  SPEAKER_01: And that was several years ago.
• 6:46
  SPEAKER_01: I think that there's definitely more DAGs that we're migrating now.
• 6:48
  SPEAKER_01: So there's a challenge of scale.
• 6:50
  SPEAKER_01: We're also supporting probably 50% more data engineers now than we did three or four years ago.
• 6:54
  SPEAKER_01: But the process itself is really straightforward.
• 6:56
  SPEAKER_01: Like I said, there are a few things that we've encountered along the way.
• 6:59
  SPEAKER_01: Connections not working properly.
• 7:00
  SPEAKER_01: We've also taken the opportunity to migrate from open source airflow to astronomer.
• 7:04
  SPEAKER_01: So that has a whole bunch of other challenges along the way.
• 7:07
  SPEAKER_01: It's such as we've gone from hosting our own secret to using a default secret backend.
• 7:11
  SPEAKER_01: So that provided some other opportunities or challenges for us along the way.
• 7:15
  SPEAKER_01: But we're learning as we go along, and that's the kind of thing that you fix once and then you just keep on going.
• 7:19
  SPEAKER_02: Hopefully that's a good challenge and that it requires a little bit of migration.
• 7:23
  SPEAKER_02: But once you're there, it's a better experience.
• 7:25
  SPEAKER_02: At least that's the hope.
• 7:26
  SPEAKER_02: And now that you're at least well on the way to moving to Airflow 3, are there any particular features within the new release that you think your team will take advantage of?
• 7:35
  SPEAKER_01: Oh, certainly we're already using the backfills feature.
• 7:38
  SPEAKER_01: Till now it was only available in CLI, and I don't think any of us have ever used it more than once or twice.
• 7:42
  SPEAKER_01: But now I personally have used it a couple of times, fixing broken pipelines that we've had, and we've shown it off to our data engineers, and they're really excited about that one.
• 7:50
  SPEAKER_01: DAG versioning is another one where, you know, it has its pluses and its minuses where people are used to just pushing their code and rerunning a DAG run and expecting it to run with a new code.
• 7:59
  SPEAKER_01: So it's a bit of a learning curve, but it's a nice feature to have.
• 8:02
  SPEAKER_01: I'm very excited about asset aware scheduling, but we haven't gotten into that yet.
• 8:06
  SPEAKER_01: We're going to focus on the migration, get done with that.
• 8:08
  SPEAKER_01: And then one of my goals for the upcoming months is pushing people to start adopting asset aware scheduling because I think that could really speed up our pipelines.
• 8:16
  SPEAKER_02: Yeah.
• 8:16
  SPEAKER_02: Do you use data sets already from Airflow 2?
• 8:18
  SPEAKER_02: Or will this be like a net new employment?
• 8:20
  SPEAKER_01: Yeah, I'm glad to say that we haven't because that would make things a bit more challenging.
• 8:23
  SPEAKER_01: So we're going straight from our own in-house.
• 8:25
  SPEAKER_01: We had a what do you call it?
• 8:27
  SPEAKER_01: A data supervisor sensor that we've developed internally to see that our tables are up to date.
• 8:33
  SPEAKER_01: And then that would be like the first task in every one of our DAGs.
• 8:36
  SPEAKER_01: So you'd run that, and then once the sensor was returned success, then our Trino operators or whatever came down the line would start running.
• 8:42
  SPEAKER_01: So I'm looking forward to gradually phasing that out in favor of facet aware scheduling.
• 8:47
  SPEAKER_02: Okay.
• 8:48
  SPEAKER_02: Nice.
• 8:48
  SPEAKER_02: Yeah, it sounds like that'll be a big improvement there.
• 8:51
  SPEAKER_02: Speaking of Airflow features on a slightly different topic, you spoke at the recent Airflow Summit about using deferrable operators to improve performance and reduce costs.
• 9:01
  SPEAKER_02: Of course, deferable operators, not a new feature, but certainly one that continues to grow in adoption.
• 9:06
  SPEAKER_02: Can you tell me a little bit more about that initiative?
• 9:09
  SPEAKER_01: Yeah, so that's something that came up about a year ago.
• 9:12
  SPEAKER_01: We were looking not so much the cost side of things, but the operational aspect of it and how we could reduce the load on our Airflow cluster to make it more available because our schedule was being choked up with lots of tasks running at the same time.
• 9:26
  SPEAKER_01: And what I landed on ultimately was trying to see if we could make some of our in-house operators deferable so that they're not taking up a worker slot while they're waiting for Trino or other third-party operators to run.
• 9:37
  SPEAKER_01: So that was a bit of a learning experience for me, understanding how deferral works.
• 9:41
  SPEAKER_01: It was also a little daunting and intimidating at first trying to understand how this feature works.
• 9:45
  SPEAKER_01: Once I dove into it, I realized it's actually quite simple.
• 9:47
  SPEAKER_01: There's really not much to it, and I presented that in a summit.
• 9:50
  SPEAKER_01: So I created one deferable operator as a proof of concept.
• 9:54
  SPEAKER_01: And once I did that, we've communicated internally, and we now have another two or three operators that just in the past few weeks we've migrated to a deferrable mode.
• 10:02
  SPEAKER_01: During the migration, we saw that we were able to drop the CPU usage on one of our clusters to about, I think it was 4% down from close to 50, just by switching from sensor with three schedule to defer all.
• 10:13
  SPEAKER_01: So that's definitely something that's worth doing.
• 10:15
  SPEAKER_01: And like I said in the summit, if you're not using deferrable mode, then please do look into it.
• 10:19
  SPEAKER_01: I think you'll only benefit from it.
• 10:23
  SPEAKER_00: This episode is brought to you by Astronomer, creators of Astro, the leading fully managed airflow service designed for the modern data stack.
• 10:32
  SPEAKER_00: Whether you're orchestrating large language models, implementing a data mesh architecture, or streamlining complex workflows, Astro empowers your team to focus on innovation, not infrastructure.
• 10:44
  SPEAKER_00: To see how you can maximize airflow's potential within your data ecosystem, visit www.astonomer.io.
• 10:52
  SPEAKER_02: It's something that we've seen a lot of our customers use, especially the ones at larger scales, like you're describing, where you do have so many different tasks running, including like potentially at the same time.
• 11:03
  SPEAKER_02: It can really offload a lot of that stress on your workers.
• 11:06
  SPEAKER_02: And then of course the cost improvements come along with it.
• 11:09
  SPEAKER_02: Very cool.
• 11:10
  SPEAKER_02: Have the performance increases that you've seen using those deferrable operators helped you implement any additional use cases, or is that part of something you're doing to plan for additional scale in the future?
• 11:21
  SPEAKER_01: At this point, we're using it in order to reduce the workload on the servers and also to reduce costs.
• 11:26
  SPEAKER_01: But if we're talking about improving our scale, we've actually found that using AI in our airflow development has been a huge boon to our velocity.
• 11:35
  SPEAKER_01: We've created several cursor rules for writing airflow DAGs, which was rather interesting.
• 11:40
  SPEAKER_01: So, for example, now we could just write a prompt.
• 11:42
  SPEAKER_01: I need a DAG that reads from this table and creates a slowly changing dimension or whatever type of ETL we need.
• 11:48
  SPEAKER_01: And with the rules that we have and with the MCPs that we've developed internally that can read our airflow DAGs and our open metadata table structure, it pretty much automatically creates an entire framework for you where you can get a DAG within a matter of minutes.
• 12:02
  SPEAKER_01: You'll have to probably adjust it a little bit.
• 12:04
  SPEAKER_01: But this has really accelerated our velocity to the point where writing DAGs is no longer much of an issue.
• 12:10
  SPEAKER_01: We've gotten to the point, in fact, we're in some of our teams, the data engineers have pulled in their analysts who are now using Airflow on cursor to write DAGs themselves instead of being dependent on the engineers themselves to do it.
• 12:20
  SPEAKER_02: Yeah, very cool.
• 12:21
  SPEAKER_02: And so to tie it back to what we were just talking about, does using deferrable operators and kind of other aspects of running Airflow at a higher scale, I assume, allow you to develop DAGs and deploy them much quicker because you don't have to worry about how they're running.
• 12:34
  SPEAKER_01: Yeah.
• 12:35
  SPEAKER_01: I've never really worried because you can always turn a dial and just get more CPU, but now we don't even have to do that.
• 12:40
  SPEAKER_02: Yeah.
• 12:40
  SPEAKER_02: More cost effective.
• 12:42
  SPEAKER_01: Exactly.
• 12:43
  SPEAKER_01: Absolutely.
• 12:43
  SPEAKER_02: Yeah.
• 12:44
  SPEAKER_02: That's really cool to hear that the AI has been so helpful for you.
• 12:47
  SPEAKER_02: It's certainly something that at Astronomer we've invested in as well.
• 12:51
  SPEAKER_02: Do you ever hit any issues with like hallucinations or things like that or lack of airflow context with using cursor, or are you able to prompt your way out of that?
• 13:02
  SPEAKER_01: It's definitely something that's come up, especially with the switch from Airflow 2 to Airflow 3.
• 13:06
  SPEAKER_01: It's not so much hallucination, but so much it's as it's not on point with the new documentation for Airflow 3.
• 13:11
  SPEAKER_01: So you really do need to be on top of that and pay attention to what it's doing.
• 13:14
  SPEAKER_01: It's not the kind of thing you could just run into production right away without CI test that'll catch things.
• 13:19
  SPEAKER_01: And another aspect is the fact that our code base does still have a lot of legacy DAGs that are not with task flow and every engineer having their own flavor of code that they write.
• 13:28
  SPEAKER_01: So it's difficult to have a constant structure of a DAG that is created.
• 13:33
  SPEAKER_01: So that's something we're still working on tweaking and trying to have, you know, you give it a prompt and you know what to expect coming out the other side.
• 13:38
  SPEAKER_02: Yeah, makes sense.
• 13:39
  SPEAKER_02: So it's a help, but not something you would use blindly.
• 13:42
  SPEAKER_01: Exactly.
• 13:43
  SPEAKER_01: I don't think AI is yet at the point where you just release AI code into production.
• 13:47
  SPEAKER_02: Probably not a good idea.
• 13:49
  unknown: No.
• 13:50
  SPEAKER_02: Yeah, at Astronomer, we've worked on our Astra IDE, which is similar to what you're describing, but just with a lot of airflow context baked in to try and make it better at doing things for a specific airflow version or something like that.
• 14:02
  SPEAKER_02: But we've certainly found that yeah, it helps people reduce or increase the velocity, I should say, a lot of how fast they're able to ship new pipelines, especially if they don't have a ton of airflow knowledge.
• 14:14
  SPEAKER_02: I think some airflow knowledge is still required, again, at least in the review process, but lowers the barrier to entry.
• 14:19
  SPEAKER_01: So that's similar to what we've done with the rules that we've created.
• 14:22
  SPEAKER_01: Also with the MCPs that we have connecting it really to our code base and to our databases, I think that really helps it understand not only what the structure of the deck should be, but also which tables to go to and which columns to pull.
• 14:34
  SPEAKER_02: Okay, great.
• 14:35
  SPEAKER_02: You mentioned a little bit earlier when you were talking about migrating to Airflow 3, that you're also maybe the process is done or in the process now of migrating to using Astronomers, our managed Airflow service.
• 14:48
  SPEAKER_02: What was your motivation for doing that?
• 14:50
  SPEAKER_02: What has that experience been like?
• 14:53
  SPEAKER_01: So we had two separate motivations for that.
• 14:55
  SPEAKER_01: One of them was lowering the workload on our data frameworks team, because till now every upgrade required them having their own a separate server to run tests on the latest version, make sure everything's working properly, them leading the migration.
• 15:08
  SPEAKER_01: And this really led to the fact that in three or four years, we've only had three version upgrades from 1.7 to 2.3 and 2.6.
• 15:18
  SPEAKER_01: And we've been on 2.6 for the past two and a half years, give or take.
• 15:21
  SPEAKER_01: So one of the motivations was really to offset that burden onto somebody else that would be good at it and do that for us and allow us to have more rapid, more incremental upgrades and enjoy the benefits of new features as they come out.
• 15:34
  SPEAKER_01: And just the capacity of our data frameworks team has been choked up by this and all sorts of other responsibilities that they have.
• 15:41
  SPEAKER_01: So being able to offload that has been an advantage.
• 15:44
  SPEAKER_02: Yeah, totally makes sense.
• 15:45
  SPEAKER_02: Well, that's great to hear.
• 15:47
  SPEAKER_02: Okay.
• 15:47
  SPEAKER_02: Ethan, thanks so much for going through all that.
• 15:49
  SPEAKER_02: That was really interesting.
• 15:50
  SPEAKER_02: I always like to end with a standard final question.
• 15:53
  SPEAKER_02: That's what would you most like to see from the Airflow project in the near future?
• 15:56
  SPEAKER_01: So I think that as a company with so many data engineers, each one working on their own subdivision of DAGs, I think something that could really benefit is focus on multi-tenancy and improving the UI for separate groups.
• 16:09
  SPEAKER_01: So any engineer could say filter on the DAGs that they want to have that saved or various different configurations and personalizations, maybe even create a dashboard where you could see only your DAGs or the DAGs that you care about and not just 10 favorites or whatever, and keep track of that.
• 16:25
  SPEAKER_01: So I think that's something that should be looked into.
• 16:28
  SPEAKER_02: I think that's a great answer.
• 16:30
  SPEAKER_02: Probably multi-tenancy is the most common answer I hear when I ask this question on the podcast.
• 16:35
  SPEAKER_02: And I like what you said about personalizing the UI because I do think the favoriting DAGs that came in Airflow 3 was a step forward in that direction, something that people had asked for a really long time.
• 16:45
  SPEAKER_02: And it's certainly very helpful if you have two or three out of a hundred DAGs that everybody's gonna look at every day.
• 16:51
  SPEAKER_02: But to your point, I think when you've got thousands of DAGs, maybe favoriting 50 of them is not the most efficient.
• 16:59
  SPEAKER_01: I would be happy to start my day opening Airflow and just have a dashboard of my DAGs and their status from the past day, things like that.
• 17:05
  SPEAKER_02: That would be really cool.
• 17:07
  SPEAKER_02: Definitely good feedback to give to the Airflow project.
• 17:10
  SPEAKER_02: Yeah, I like that.
• 17:11
  SPEAKER_02: Okay, Ethan, this has been really great.
• 17:13
  SPEAKER_02: Thanks so much for coming on.
• 17:15
  SPEAKER_02: What is the best way for folks to get in touch with you?
• 17:17
  SPEAKER_01: So you can find me on LinkedIn.
• 17:19
  SPEAKER_01: If you search for my name on Google, you'll find LinkedIn and then my talk at the Airflow Summit.
• 17:23
  SPEAKER_01: So feel free to do that.
• 17:24
  SPEAKER_02: Yeah, that's great.
• 17:25
  SPEAKER_02: It was a great talk.
• 17:26
  SPEAKER_02: It is published now.
• 17:27
  SPEAKER_02: Certainly will still be by the time this episode comes out.
• 17:30
  SPEAKER_02: So yeah, we will throw that in the show notes.
• 17:33
  SPEAKER_02: Definitely recommend it.
• 17:34
  SPEAKER_02: Okay.
• 17:34
  SPEAKER_02: Thanks again, Ethan.
• 17:36
  SPEAKER_02: It's been really great talking to you.
• 17:37
  unknown: Thanks.
• 17:37
  SPEAKER_01: You too, Kenton.
• 17:38
  SPEAKER_01: Take care.
• 17:41
  SPEAKER_00: Thanks for joining us for the latest episode of the Data Flowcast, mastering airflow for data engineering and AI.
• 17:48
  SPEAKER_00: You'll find links in the show notes to any resources mentioned in today's show.
• 17:52
  SPEAKER_00: If you're enjoying our podcast, please subscribe so you never miss an episode and leave us a five star review.