Latest / The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI / Building the Future of Airflow Execution at Astronomer with Ian Buss and Piotr Chomiak

Transcript

Building the Future of Airflow Execution at Astronomer with Ian Buss and Piotr Chomiak · The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI

• 0:00
  SPEAKER_02: We've been thinking a long time about how can we build a better and a more efficient way of allocating tasks to workers, making sure tasks end up on the right workers in a more efficient way and looking to improve worker utilization as well.
• 0:12
  SPEAKER_04: What we're working on is delivering task-level CPU and memory utilization metrics.
• 0:16
  SPEAKER_04: Users will have granular data to work with respect to understanding how much compute their tasks are taking and how to best right-size their workers, their worker queues, and things of that sort.
• 0:26
  SPEAKER_04: In the future, we're gonna actually leverage some of that data ourselves to better allocate tasks to workers.
• 0:49
  SPEAKER_03: Hey Airflow community, exciting news.
• 0:52
  SPEAKER_03: This fall, we're launching Beyond Analytics, a virtual data ops conference happening September 16th from 12 to 3 p.m.
• 0:59
  SPEAKER_03: Eastern.
• 1:00
  SPEAKER_03: You'll hear from data leaders on how modern data ops and orchestration are powering AI, machine learning, and production grade data products.
• 1:07
  SPEAKER_03: And here's the best part.
• 1:08
  SPEAKER_03: You can join Mark Lumberti for a fast-track Airflow-3 certification prep session and get a discount code for a free certification, a$150 value.
• 1:18
  SPEAKER_03: The conference is virtual, it's free, and it's made just for you.
• 1:21
  SPEAKER_03: You can register now at astronomer.io slash beyond slash data flowcast.
• 1:29
  SPEAKER_03: Hi everyone, welcome to the Data Flowcast.
• 1:32
  SPEAKER_03: I'm your host, Kenton Dannis, and today I'm joined by two folks from Astronomer, Peter Shomiak, who is a principal product manager, and Ian Buss, who is a principal software engineer.
• 1:42
  SPEAKER_03: How are you guys doing?
• 1:44
  SPEAKER_04: Good.
• 1:44
  SPEAKER_04: Thanks.
• 1:45
  SPEAKER_04: Fantastic.
• 1:46
  SPEAKER_04: Good to meet you.
• 1:46
  SPEAKER_03: It's great to have both of you on.
• 1:48
  SPEAKER_03: I always love talking to other fellow astronomers about all things airflow.
• 1:53
  SPEAKER_03: And I'm excited about our topic today.
• 1:55
  SPEAKER_03: We're going to discuss the new Astro Executor and remote execution.
• 1:59
  SPEAKER_03: We've done a couple of other episodes focused on remote execution from an open source perspective before, but today we're going to talk about it more from the astronomer perspective.
• 2:09
  SPEAKER_03: So yeah, I'm excited to dig into that.
• 2:12
  SPEAKER_03: And before we talk about all of that, I want to start with a little bit of background on both of you.
• 2:17
  SPEAKER_03: So can you tell me what your role is at astronomer and how you came to be here?
• 2:23
  SPEAKER_02: I guess I'll go first.
• 2:24
  SPEAKER_02: Yeah, so I'm a principal software engineer, astronomer.
• 2:26
  SPEAKER_02: I'm currently a tech lead in the what we call the core airflow engineering team, which is a back-end engineering team in the Astro product.
• 2:34
  SPEAKER_02: And we cover essentially everything related to getting an Airflow instance running on Astro and integrating it with the upstream releases that our internal open source team produce of Airflow.
• 2:46
  SPEAKER_02: So that's customer-facing APIs and UIs for creating an update, updating your Airflow instance, the backend infrastructure for running Airflow on our clusters.
• 2:55
  SPEAKER_03: So all of the magic that makes managed Airflow service happen.
• 2:59
  SPEAKER_04: That's right.
• 3:00
  SPEAKER_03: Love it.
• 3:01
  SPEAKER_03: And Peter.
• 3:02
  SPEAKER_03: Yeah.
• 3:03
  SPEAKER_04: Peter Jomek.
• 3:04
  SPEAKER_04: I've been with Astronomer for four years now as a product manager.
• 3:08
  SPEAKER_04: I have the pleasure of working with Ian and his team, delivering really that application layer of managed airflow.
• 3:15
  SPEAKER_04: But I also have the pleasure of working with our infrastructure team that delivers all the infrastructure that powers our managed airflow product across all three clouds.
• 3:25
  SPEAKER_04: So working across both and delivering our managed airflow product with a key focus on how we do that successfully and at scale for the enterprise type customer in particular has been a key focus of mine.
• 3:38
  SPEAKER_04: It's a bit of a cheesy description, but from the product perspective, on the full stack of our managed airflow product.
• 3:44
  SPEAKER_04: Yeah.
• 3:44
  SPEAKER_04: Happy to be here.
• 3:45
  SPEAKER_03: That's great.
• 3:46
  SPEAKER_03: And I think it you both touched on it a little bit, but since we're going to talk more about Astro features today, I think it would be helpful to give a little bit more background on Astro.
• 3:56
  SPEAKER_03: I think a lot of our listeners might be familiar with Astro as astronomers' managed airflow service, but maybe don't know too much more beyond that.
• 4:04
  SPEAKER_03: So how would you describe it in a couple of minutes if all somebody knew was it's a thing where you can run Airflow?
• 4:11
  SPEAKER_04: Yeah, I'm happy to take that one.
• 4:12
  SPEAKER_04: So definitely like our core product is our managed airflow experience.
• 4:17
  SPEAKER_04: And as part of that, as most folks know, we offer a host of features, obviously built on enterprise grade security and reliability, but backed by the hypervise, we're able to do all sorts of things with respect to intelligent autoscaling, multi-tenancy, task optimized worker queues.
• 4:33
  SPEAKER_04: And that's certainly been our focus.
• 4:35
  SPEAKER_04: But over the past year or so, we've been expanding our product to really cover the data ops market, if you will.
• 4:42
  SPEAKER_04: And we've launched our observed product, which gives you visibility into your data pipelines down to what we refer to as the data product, right?
• 4:52
  SPEAKER_04: So what is that end sort of result or business value that your pipelines are generating?
• 4:57
  SPEAKER_04: And you're able to monitor those, see the lineage across all the various tasks and data assets that ultimately make a data product.
• 5:05
  SPEAKER_04: So we've launched a product there and is available as part of our astral product.
• 5:09
  SPEAKER_04: And then right now we're also getting into what we call the build space with our IDE.
• 5:15
  SPEAKER_04: So this is in private preview, but we are working on a capability that's going to give data engineers, analysts, what have you, really an IDE backed by AI that will help you build data pipelines with obviously all the sort of experience and know-how that we have and that we built into the models that'll help you build these data pipelines across the whole stack.
• 5:41
  SPEAKER_04: We refer to now the entire product portfolio as our data ops platform.
• 5:46
  SPEAKER_04: Build, run, and observe is what we do here at Astronomer.
• 5:49
  SPEAKER_03: That's great.
• 5:50
  SPEAKER_03: Very concise description of all of the goodness that Astro provides beyond the managed airflow part or as part of the managed airflow service.
• 6:00
  SPEAKER_03: I think most of our listeners will be familiar with that back in April, of course, Airflow 3 was released, which was a very exciting milestone in the Airflow project and was supported on Astro from day one.
• 6:13
  SPEAKER_03: But with that, we also rolled out the Astro Executor as part of our managed service.
• 6:19
  SPEAKER_03: So let's start there.
• 6:21
  SPEAKER_03: And can you tell our listeners what the Astro Executor is and what makes it different from the other available executors in Airflow?
• 6:29
  SPEAKER_02: Sure.
• 6:29
  SPEAKER_02: Yes.
• 6:29
  SPEAKER_02: So with Airflow 3, we had the opportunity to like deliver what we believe is like the next generation of executor in Airflow.
• 6:37
  SPEAKER_02: An Astro Executor essentially has a few different components, one of which is a component that lives in the Airflow 3 API server.
• 6:45
  SPEAKER_02: And then there's another component that lives in the scheduler.
• 6:47
  SPEAKER_02: And together those form the executor.
• 6:49
  SPEAKER_02: And then we also have as part of new Astro Agent architecture, we have a worker that's part of that that can interact with the Astro Executor as well.
• 6:57
  SPEAKER_02: So there's like a three-part uh architecture.
• 7:00
  SPEAKER_02: But winding back a bit to us why we built it in the first place, we have obviously, as we know, we have Celery Executor in Airflow.
• 7:07
  SPEAKER_02: And Astronomer, we operate many thousands of production effort deployments, a lot of which are celery-based.
• 7:14
  SPEAKER_02: And as part of managing and operating those deployments, we've encountered a fair share of reliability issues that we found with celery, especially around its kind of broker-related architecture.
• 7:24
  SPEAKER_02: I don't know, for those that don't know, the way that celery works is that when a task is scheduled or queued, the executor will push out a task request to an intermediate broker, and then the workers will uphold the work from the worker.
• 7:36
  SPEAKER_02: And we found that can actually be sometimes a bit brittle.
• 7:38
  SPEAKER_02: And it's an extra piece in the architecture that perhaps we don't need.
• 7:42
  SPEAKER_02: We've obviously done a lot in Astronomer to try and minimize these issues, and we've got it to a pretty stable place.
• 7:47
  SPEAKER_02: But there are some things that are inherent in the architecture that we wanted to address.
• 7:51
  SPEAKER_02: So actually with Airflow 3, we had this ideal opportunity with the new task SDK to build the next generation style executor that was good for both hosted, by which we mean uh running in our own clusters, but also for remote execution where we bring the customers' infrastructure and they can run the task themselves on their own workers on their own infrastructure.
• 8:10
  SPEAKER_02: So we've been thinking a long time about how can we build a better and a more efficient way of allocating tasks to workers, making sure tasks end up on the right workers in a more efficient way, and looking to improve worker utilization as well.
• 8:23
  SPEAKER_02: So all this together led us to build this new executor.
• 8:27
  SPEAKER_02: And by doing it, building our own internal executor for Astro, we can really focus in on uh building a great customer experience and actually building on top of the Astro Executor as really open source components to with advanced monitoring, auto-scaling, all those good things that we can build in with deep integration into the platform.
• 8:44
  SPEAKER_02: Um so Astro Executor itself is a drop-in replacement for Celery, is no code changes required.
• 8:49
  SPEAKER_02: Anything that works in it for Sellery in FO3 will work in Astro Exec.
• 8:54
  SPEAKER_02: And unlike Celery, there's no broker involved, but it's an entirely pool-based architecture.
• 9:00
  SPEAKER_02: So workers will aren't beat in regularly to the API server.
• 9:04
  SPEAKER_02: They'll pull their work, the API server will allocate work to individual workers.
• 9:09
  SPEAKER_02: And then that's the kind of the basic architecture of Astro Exec.
• 9:12
  SPEAKER_03: That's great.
• 9:13
  SPEAKER_03: Thanks for going through that.
• 9:14
  SPEAKER_03: And so just to summarize a little bit of what you said, leveraging the architectural changes that happened in Airflow 3 as part of the breaking changes that that major release allowed with the you mentioned the task SDK was a big one.
• 9:26
  SPEAKER_03: We've used that re-architecturing to build an executor that is a little more reliable and more scalable than the celery that most folks probably use.
• 9:36
  SPEAKER_03: Yeah.
• 9:36
  SPEAKER_03: Um yeah.
• 9:37
  SPEAKER_02: And I just want to emphasize that the architecture was built with the future in mind.
• 9:41
  SPEAKER_02: So it's going to open up a whole lot of stuff that simply wasn't possible with the existing executors in Airflow.
• 9:47
  SPEAKER_03: And speaking of the future, here, is there anything you want to touch on in terms of kind of what we're looking at going forward?
• 9:53
  SPEAKER_03: This was released back in April, but of course is still being developed and optimized and all of that.
• 9:59
  SPEAKER_04: Yeah, for sure.
• 10:00
  SPEAKER_04: Yeah.
• 10:01
  SPEAKER_04: Definitely give a sneak peek as to what we have planned.
• 10:04
  SPEAKER_04: And as you noted, this is just the beginning.
• 10:06
  SPEAKER_04: We're going to continue innovating with this executor and really, as Ian said, up-leveling the experience of running Airflow on Astro.
• 10:15
  SPEAKER_04: This quarter, what we're working on is delivering task-level CPU and memory utilization metrics.
• 10:21
  SPEAKER_04: Users will have granular data to work with respect to understanding how much compute their tasks are taking and how to best write-size their workers, their worker queues, and things of that sort.
• 10:32
  SPEAKER_04: In the future, we're going to actually leverage some of that data ourselves to better allocate tasks to workers.
• 10:39
  SPEAKER_04: We'll understand, generally speaking, how much CPU and memory particular tasks are taking.
• 10:43
  SPEAKER_04: And say a worker is running low on memory, we may not want to pack a memory-intensive task on said worker and spin up a different worker instead.
• 10:54
  SPEAKER_04: A lot of intelligence coming with task to worker allocation.
• 10:58
  SPEAKER_04: And similarly with worker autoscaling.
• 11:01
  SPEAKER_04: Right now, with salary, it's like a prey and spray model where tasks can land on any running worker.
• 11:08
  SPEAKER_04: And inclusive of termination of workers, it's quite random.
• 11:11
  SPEAKER_04: So we would definitely want to uplevel that and introduce intelligent worker autoscaling and more be able to wake workers from termination.
• 11:18
  SPEAKER_04: Today, with salary, once a worker goes into sort of termination, the termination grace period is respected, of course.
• 11:25
  SPEAKER_04: But you can't say more tasks come in, you can't assign them to that worker.
• 11:30
  SPEAKER_04: That worker will just run through whatever task has remaining and it'll terminate.
• 11:34
  SPEAKER_04: A lot of innovation coming down the pipe in the not too distant future, which we're super excited about.
• 11:41
  SPEAKER_01: This episode is brought to you by Astronomer, creators of Astro, the leading fully managed airflow service designed for the modern data stack.
• 11:50
  SPEAKER_01: Whether you're orchestrating large language models, implementing a data mesh architecture, or streamlining complex workflows, Astro empowers your team to focus on innovation, not infrastructure.
• 12:02
  SPEAKER_01: To see how you can maximize Airflow's potential within your data ecosystem, visit www.astonomer.io.
• 12:11
  SPEAKER_03: I want to touch on the remote execution piece a little bit as well.
• 12:13
  SPEAKER_03: So, Ian, you mentioned earlier that, of course, the Astro Executor works for both hosted tasks that are running in the same place as your Airflow instance, which is how they traditionally have always worked, but also works for remote execution, which I think was one of the most exciting features released in Airflow 3 was this ability to run tasks in totally separate infrastructure from where your Airflow deployment is living.
• 12:38
  SPEAKER_03: And that's a feature of open source Airflow as well.
• 12:40
  SPEAKER_03: But tell me a little bit about why we thought it was important to build remote execution into Astro as a first class feature.
• 12:48
  SPEAKER_04: Yeah, I'll take that one.
• 12:49
  SPEAKER_04: That's a great question.
• 12:50
  SPEAKER_04: So we truly believe remote execution is a major and differentiated architectural enhancements to orchestration, particularly with Airflow on Astro, really solving primarily for critical information security needs, but also unleashes a lot of operational and computing flexibility.
• 13:08
  SPEAKER_04: And the reason we went down this path is we've, of course, had a ton of interest in our fully managed product, right?
• 13:15
  SPEAKER_04: And we frankly have customers across all industries running on Astro.
• 13:20
  SPEAKER_04: On the flip side, we also have our self-managed product, Astro Private Cloud, where those customers are fully managing an astronomer install.
• 13:29
  SPEAKER_04: And again, that's due to mostly due to critical information security requirements where everything needs to remain in their environment, no third-party access, so on and so forth.
• 13:39
  SPEAKER_04: Now, we've seen over the past few years with running Astro a need from our existing customers and prospects that they very much want a managed solution, but at the same time, they very much need to solve for these information security requirements, whether they're regulatory, contractual, compliance, what have you, and not have a third party have any mechanism of connecting into their environment, having any access to their data, whether that be the data they're orchestrating or the DAG code, or frankly their task logs.
• 14:13
  SPEAKER_04: So with remote execution, we're solving for critical security needs to address these sort of compliance and then regulatory concerns and so on and so forth, by ensuring that the components that touch any customer data or require any customer data as part of orchestration are all running within their environment.
• 14:32
  SPEAKER_04: But the orchestration capabilities that Astro provides, as well as all of our other capabilities around alerting, observability, so on and so forth, that's still all accessible and available as part of the Astro platform.
• 14:44
  SPEAKER_04: So this split of orchestration and execution has really hit home with a lot of our current prospects and really opening up opportunity for new customers.
• 14:53
  SPEAKER_03: Yeah, definitely.
• 14:54
  SPEAKER_03: So many different use cases that are enabled by this feature from everything you touched on security and sensitive data and resource utilization and all of that stuff.
• 15:04
  SPEAKER_03: It opens up uh a lot of doors there.
• 15:07
  SPEAKER_03: And then I'm also curious to learn more just about how remote execution works on Astro and how it's different from the remote execution that of course exists in open source airflow with the edge executor.
• 15:19
  SPEAKER_02: Sure, that's a great question.
• 15:20
  SPEAKER_02: So winding back to the Astro Executor discussion we had earlier, we said how we designed it from the ground up to be compatible with hosted and remote.
• 15:29
  SPEAKER_02: So essentially what's happening is that we've built this new what we call an agent-based architecture, where in hosted we just run Astro Exec worker.
• 15:37
  SPEAKER_02: But in the remote execution case, as Peter said, anything that touches user code or produces logs or sensitive task logs or touches customer data sources has to run in there.
• 15:49
  SPEAKER_02: The sort of the design goal is that must run in the customer infrastructure and be isolated from astronomer managed components.
• 15:56
  SPEAKER_02: So essentially this new Astro Agent architecture can run one of three flavors of process.
• 16:02
  SPEAKER_02: It can run the worker, just as in hosted, but it can also run the DAG processor, touching customer code, and it can also run the triggerer.
• 16:10
  SPEAKER_02: So for the deferred operators, the sort of asynchronous tasks.
• 16:13
  SPEAKER_02: And we built this is kind of a two-plane architecture.
• 16:15
  SPEAKER_02: So we have the orchestration plane, as Peter mentioned, and that runs the scheduler and the API server.
• 16:21
  SPEAKER_02: So that's the orchestration parts where there's no sensitive customer information.
• 16:25
  SPEAKER_02: And then there's the execution plane, which is everything else.
• 16:27
  SPEAKER_02: So the workers are the triggerer and the DAG processor.
• 16:31
  SPEAKER_02: And we built the Astro Agent from the ground up to have a protocol that can interact with the API server to make sure that all those three components work seamlessly in a remote context.
• 16:42
  SPEAKER_02: Yeah, so that's like how it works in terms of how it's differentiated from OSS offering, I think is the sort of the same answer, really, that OSS has the edge executor, but that's already focused on workers.
• 16:53
  SPEAKER_02: But we wanted to bring in this sort of triggerer and DAG processor part of it as well.
• 16:57
  SPEAKER_02: So customers could have like full confidence that nothing other than redacted or metadata was available in the orchestration plane.
• 17:05
  SPEAKER_03: Interesting.
• 17:06
  SPEAKER_03: That's really cool.
• 17:07
  SPEAKER_03: I have just using it myself for demos and stuff, been just struck by the idea of uh running a task and then I can't see the code in my UI, which is exactly what it was designed for.
• 17:19
  SPEAKER_03: But it's weird when you're used to being able to see everything, like, oh, this is just my airflow environment and it has all the stuff.
• 17:25
  SPEAKER_03: And actually, now in this case, it doesn't.
• 17:27
  SPEAKER_03: I can run the task.
• 17:28
  SPEAKER_03: I have no idea what code was executed.
• 17:30
  SPEAKER_03: Yeah, yeah.
• 17:31
  SPEAKER_04: That's really I will add also a key differentiator of our remote execution solution over the edge executor is you have an entire RD team that is consistently developing, innovating, and enhancing this offering that we're speaking about.
• 17:47
  SPEAKER_04: And this is, of course, a key focus of ours.
• 17:50
  SPEAKER_04: As I said, it presents a huge opportunity.
• 17:52
  SPEAKER_04: Our customers can be rest assured that they'll get all the necessary support and future capabilities unlocked for them.
• 17:59
  SPEAKER_03: So that's great.
• 18:00
  SPEAKER_03: Maybe this is just my own personal bias of coming from a data engineering background and not a software one and not a networking one.
• 18:08
  SPEAKER_03: But I feel that in my own experience, setting up remote execution can be a little tricky in some cases, just to get everything wired up correctly with a lot of different infrastructure.
• 18:18
  SPEAKER_03: So for me personally, I would definitely rest assured knowing that I had some extra help with making that all work.
• 18:24
  SPEAKER_02: Definitely.
• 18:24
  SPEAKER_02: And we're doing more and more.
• 18:25
  SPEAKER_02: One of our key focuses this quarter is to make the experience of managing remote execution much better as well.
• 18:31
  SPEAKER_02: So we're constantly pushing on that as a sort of delivery goal.
• 18:34
  SPEAKER_03: Yeah, that's great.
• 18:36
  SPEAKER_03: And Peter, from your perspective, you touched on this a little bit earlier, but I'm curious, since this has been rolled out to the extent that you can share, are there any particular industries or types of companies that you've seen benefit from this feature already?
• 18:51
  SPEAKER_04: Yeah, we already have a number of customers leveraging remote execution.
• 18:54
  SPEAKER_04: It's only been a couple months now, but everything from financial services to automotive to gaming we've seen across.
• 19:04
  SPEAKER_04: Again, the adoption here is certainly not limited to any industry.
• 19:08
  SPEAKER_04: Everyone's dealing with all sorts of critical information security requirements, or they may need computing flexibility that Astro does not offer today.
• 19:17
  SPEAKER_04: So it's amazing to have a product that doesn't have a particular niche or industry that it applies to.
• 19:23
  SPEAKER_04: So we're super excited to see further adoption and uh and what that all brings.
• 19:27
  SPEAKER_03: Like I said before, I think one of the most exciting features in Airflow 3 from an open source perspective, and also certainly from the Astro perspective and how we've implemented this, just as you said, very widely applicable to a lot of Airflow users, which is really cool.
• 19:42
  SPEAKER_03: Appreciate you both going through all of the context there.
• 19:46
  SPEAKER_03: This has been super interesting.
• 19:48
  SPEAKER_03: Given our time, I'll go ahead and move forward.
• 19:50
  SPEAKER_03: And I always like to ask guests a standard final question, and that is what would you most like to see from the Airflow project in the near future?
• 19:57
  SPEAKER_02: I personally am really excited about, and I know it's happening, is the multi-language support that's coming in Airflow.
• 20:04
  SPEAKER_02: So obviously, Airflow is flexible enough for you to be able to utilize any language today.
• 20:09
  SPEAKER_02: You can make Airflow do that, but it's not a sort of a first-class citizen, if you like, so to use like Golang or Java to write your tasks.
• 20:16
  SPEAKER_02: So I'm really looking forward to what we can do as a community to bring Airflow into that kind of multi-language world.
• 20:22
  SPEAKER_02: I think that'll open up a whole extra set of possibilities.
• 20:26
  SPEAKER_02: Yeah, it should be exciting.
• 20:27
  SPEAKER_03: Yeah, definitely.
• 20:28
  SPEAKER_03: And of course, they think that Golang support is in development right now, if I'm not wrong.
• 20:33
  SPEAKER_03: So yeah, it's already well on its way to being available to folks.
• 20:38
  SPEAKER_03: Yeah, that's great.
• 20:39
  SPEAKER_04: And I'll say true multi-tenancy support down to like really the execution layer.
• 20:43
  SPEAKER_04: And what I mean by that is everything from like the images, the dependencies, the connections, the secrets, environment variables, up to, of course, who can see particular DAGs, who can trigger particular DAGs, which sort of exists today.
• 20:58
  SPEAKER_04: I think that would very much lend itself to, again, that enterprise type customer.
• 21:03
  SPEAKER_04: I'm coming in from that angle as that's who I interface a lot with.
• 21:07
  SPEAKER_04: Now, some of these capabilities are actually available through remote execution today, just due to the nature of that architectural split that Ian was describing.
• 21:15
  SPEAKER_04: But I think it would very much benefit the Airflow project on a whole if that could just be natively available as well.
• 21:23
  SPEAKER_03: I love that too.
• 21:24
  SPEAKER_03: Okay, great.
• 21:25
  SPEAKER_03: Thank you both so much for coming on.
• 21:27
  SPEAKER_03: It's been really great speaking with you.
• 21:29
  SPEAKER_03: Just to close out, what is the best way for folks to get in touch with you?
• 21:33
  SPEAKER_02: I don't really act like a social media person, but I am on LinkedIn.
• 21:35
  SPEAKER_02: So if you search for EMBus astronomer on LinkedIn, I'm sure I'm sure I'll pop up.
• 21:39
  SPEAKER_02: That's probably the best way.
• 21:40
  SPEAKER_04: I'm on LinkedIn as well, but I'm also part of the Airflow Astronomer channel in the Apache Airflow Community Slack workspace.
• 21:48
  SPEAKER_04: So feel free, reach out.
• 21:49
  SPEAKER_03: Plug for Airflow Slack.
• 21:50
  SPEAKER_03: If you're not already on there and you're listening, please join.
• 21:53
  SPEAKER_03: I'd love to have you on there.
• 21:54
  SPEAKER_03: Okay, cool.
• 21:55
  SPEAKER_03: Thank you both again.
• 21:56
  SPEAKER_03: This has been really interesting conversation.
• 21:59
  SPEAKER_03: And yeah, I appreciate you both coming on.
• 22:01
  SPEAKER_04: Thank you.
• 22:02
  SPEAKER_04: Thank you.
• 22:02
  SPEAKER_04: It's been great.
• 22:06
  SPEAKER_01: Thanks for joining us for the latest episode of the Data Flowcast, mastering airflow for data engineering and AI.
• 22:12
  SPEAKER_01: You'll find links in the show notes to any resources mentioned in today's show.
• 22:16
  SPEAKER_01: If you're enjoying our podcast, please subscribe so you never miss an episode and leave us a five star review.