Latest / The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI / Building a custom Tableau provider for Airflow at JLR

Transcript

Building a custom Tableau provider for Airflow at JLR · The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI

• 0:00
  SPEAKER_01: It's it's a gate, it's it's a git system, so you cannot move your DAC you know across the gate unless the gate is open for you.
• 0:07
  SPEAKER_01: So it catches and not just it blocks issues from getting to the production environment.
• 0:12
  SPEAKER_00: So you're listening to the Data Flowcast, the podcast about Apache Airflow and the world of data and AI around it.
• 0:20
  SPEAKER_00: Let's get into it.
• 0:22
  SPEAKER_03: Okay, hi everyone.
• 0:24
  SPEAKER_03: Welcome to the Data Flowcast.
• 0:25
  SPEAKER_03: I'm your host, Kenton Danis, and today I'm joined by Najeeb Sulaiman, who is a senior data engineer at JLR.
• 0:32
  SPEAKER_03: Uh Najeeb, how are you?
• 0:34
  SPEAKER_01: Yeah, Kenton.
• 0:35
  SPEAKER_01: I'm fine.
• 0:36
  SPEAKER_01: And you?
• 0:37
  SPEAKER_03: I'm good.
• 0:38
  SPEAKER_03: Good.
• 0:39
  SPEAKER_03: Thanks for joining.
• 0:40
  SPEAKER_03: Okay.
• 0:41
  SPEAKER_03: Okay, well, let's go ahead and dive in.
• 0:43
  SPEAKER_03: So for today's podcast, we're going to talk about how you use Airflow to power your data pipelines at JLR.
• 0:49
  SPEAKER_03: But before we get into that, I want to start with some background.
• 0:52
  SPEAKER_03: Can you tell me for folks who might not be familiar, what does JLR do as a company?
• 0:57
  SPEAKER_01: Yeah, thank you.
• 0:58
  SPEAKER_01: I mean, first of all, thank you for vitamin.
• 1:01
  SPEAKER_01: It's a privilege to be here.
• 1:03
  SPEAKER_01: So JLR, I want to assume a lot of people would have seen JLR products, even if they don't know, you know, they are actually produced by JLR.
• 1:11
  SPEAKER_01: So JLR is a luxury automotive manufacturer.
• 1:15
  SPEAKER_01: JLR is the company behind luxury brands, vehicle brands like Range Rover, DeVenda, Jaguar, and Discovery.
• 1:23
  SPEAKER_01: So if you've ever seen any Range Rover pass you by on the streets, you know, that's JLR products.
• 1:29
  SPEAKER_01: And we design these vehicles across different plants all over the world.
• 1:34
  SPEAKER_01: And we also sell these vehicles globally as well.
• 1:36
  SPEAKER_01: So that's what Jaguar does.
• 1:39
  SPEAKER_03: Okay, great.
• 1:40
  SPEAKER_03: Yeah, as you said, I think everybody is familiar with Range Rover, Jaguar, all of those, all of those car brands.
• 1:46
  SPEAKER_03: So it was certainly new information to me about the parent company.
• 1:50
  SPEAKER_03: But yeah, that's cool.
• 1:51
  SPEAKER_03: And I'm I'm interested to hear more, you know, about your team and what you do.
• 1:56
  SPEAKER_03: So what what kind of data problems are you solving at JLR?
• 2:00
  SPEAKER_01: Yeah, so JLR is a very interesting organization.
• 2:04
  SPEAKER_01: From what I've just said, from data perspective, it's just about the sheer breadth of what we need to make JLR a data-led organization.
• 2:16
  SPEAKER_01: I mentioned dislostery brands.
• 2:18
  SPEAKER_01: You know, there is high expectation of dislosure brands, different plants all over the world.
• 2:23
  SPEAKER_01: We saw these vehicles all over the world as well.
• 2:25
  SPEAKER_01: So you could imagine the amount of data across different domains, manufacturing, finance, supply chain, e-commerce, etc.
• 2:34
  SPEAKER_01: So to ensure JLR continues to be a data-led organization, I work in a team called Data and Artificial Intelligence Office.
• 2:43
  SPEAKER_01: We do a lot of things, and among what we do is ensuring that JLR continue to be data-led.
• 2:49
  SPEAKER_01: So we provide capabilities for JLR across the different domains, manufacturing, supply chain, et cetera, to ensure we annex this data, to ensure that we continue to produce uh vehicles that delight our customers, and also ensure that we continue to improve the higher eye, reduce costs, improve maintenance, et cetera, to ensure that JLR is data-driven.
• 3:13
  SPEAKER_01: So we do this across um different domains and also using different technologies and stocks.
• 3:17
  SPEAKER_03: Yeah.
• 3:18
  SPEAKER_03: Okay, great.
• 3:18
  SPEAKER_03: And that makes sense.
• 3:20
  SPEAKER_03: And where does airflow fit into that?
• 3:22
  SPEAKER_03: What sort of orchestration needs do you have within those use cases?
• 3:26
  SPEAKER_01: Yeah, so we use Airflow a lot.
• 3:29
  SPEAKER_01: It's main orchestrator in our data stacks.
• 3:32
  SPEAKER_01: So I would say it's like the central nervous system of our data stack is the two that connect everything.
• 3:41
  SPEAKER_01: Because we get data across different sources.
• 3:44
  SPEAKER_01: You know, imagine data gets coming in milliseconds, you know, from telemetry data from the manufacturing plant, etc.
• 3:51
  SPEAKER_01: So within our data stack itself, Airflow sits at the center of it.
• 3:56
  SPEAKER_01: It helps us to connect the dots, third-party systems from source destination, and even around data as well.
• 4:03
  SPEAKER_01: We use it every little, like I said, and I would say it's it helps us to orchestrate things just like you already know.
• 4:10
  SPEAKER_01: Airflow is an orchestrator.
• 4:12
  SPEAKER_01: So it's scheduling, automation, ensure there's no manual intervention, ensure our systems are reliable, ensure we continue to support the business with data.
• 4:22
  SPEAKER_01: So it's it's the integral part, the central nervous system.
• 4:25
  SPEAKER_01: That's how we summarize it for our data stock in JLO.
• 4:28
  SPEAKER_03: Okay, very cool.
• 4:29
  SPEAKER_03: I love that description of Airflow as the central nervous system.
• 4:33
  SPEAKER_03: I think that's a really good way of putting it.
• 4:35
  SPEAKER_03: And then yeah, yeah, that's great.
• 4:39
  SPEAKER_03: And then I'd love to dig into you know some of your use cases in a little bit more depth.
• 4:45
  SPEAKER_03: One of the things that I think you had mentioned to me before was that you have a tableau integration.
• 4:51
  SPEAKER_03: So I would imagine your team is providing some sort of like uh dashboards for internal users or something like that.
• 4:58
  SPEAKER_03: Tell me more about that and how Airflow fits in there.
• 5:01
  SPEAKER_01: Yeah, so that's a very interesting problem we're trying to solve.
• 5:07
  SPEAKER_01: And to give more context, because I know a lot of people listen to this, uh, context is very important as well.
• 5:12
  SPEAKER_01: Tableau is one of our visualizations tools.
• 5:15
  SPEAKER_01: Um basically, when you want to get data into Tableau, there are usually two major modes.
• 5:21
  SPEAKER_01: We have what we call extract refresh or live connection.
• 5:24
  SPEAKER_01: So the live connection means every time you run a visualization on Tableau, it goes to your data warehouse and then pull the data every time.
• 5:33
  SPEAKER_01: So if you go to the visualization 10 times, it goes back to the data warehouse to pull the data every time you visualize your data.
• 5:40
  SPEAKER_01: The other approach is what we call extract refresh.
• 5:43
  SPEAKER_01: So extract refresh is more like batch processing.
• 5:46
  SPEAKER_01: So it goes into your data warehouse, pulling the data, and if you visualize your data 100 times, it doesn't go back.
• 5:53
  SPEAKER_01: You know, it just uses the data that's already been extracted, and that way you can use your data without going back to your data warehouse.
• 5:59
  SPEAKER_01: That is trade-off, right?
• 6:01
  SPEAKER_01: But those are available.
• 6:02
  SPEAKER_01: So now we use extract refresh a lot.
• 6:06
  SPEAKER_01: So what that means is it's scheduled.
• 6:08
  SPEAKER_01: For example, you have a dashboard that refreshes 6 a.m.
• 6:11
  SPEAKER_01: in the morning.
• 6:12
  SPEAKER_01: So that means every morning at 6 a.m., you expect Tableau to go to your data warehouse, put the data that's available there, and then make it available.
• 6:19
  SPEAKER_01: So when you come into the morning, you already have like data to visualize.
• 6:24
  SPEAKER_01: Now, there's a problem with that.
• 6:26
  SPEAKER_01: The problem, which is a major problem, is that it's not data pipeline aware because remember, you schedule 6 a.m.
• 6:33
  SPEAKER_01: in the morning, so it is not aware if the data has already refreshed in your digital warehouse or you're still extracting the still data.
• 6:40
  SPEAKER_01: You know, uh, maybe something happened and your data pipeline has not refreshed your data warehouse, for example.
• 6:45
  SPEAKER_01: So that means you could be extracting still data because the table is not data pipeline aware.
• 6:51
  SPEAKER_01: It just goes to the data warehouse, anyways.
• 6:54
  SPEAKER_01: The other thing is that if your pipeline refreshes data warehouse earlier than 6 a.m., Tableau has to wait till 6 a.m.
• 7:01
  SPEAKER_01: before it goes and pulling the fresh data.
• 7:03
  SPEAKER_01: So that means it's also not um live.
• 7:06
  SPEAKER_01: Uh we can not need it live, but we cannot get the fresh data as that when it's already available.
• 7:11
  SPEAKER_01: So that was the problem statement.
• 7:13
  SPEAKER_01: So and what we set out to build is how can we make our dashboard pipeline away.
• 7:19
  SPEAKER_01: So there's a connection and natural integration between pipeline, uh, data pipeline refreshes the data warehouse, table is aware, tableau goes to the data warehouse, pulling the data, and anytime you are viewing data, you're always seeing the data as that's when the fresh data that is available in the data wheels, basically.
• 7:36
  SPEAKER_01: So we wanted to solve that problem, you know.
• 7:40
  SPEAKER_01: And luckily, interestingly, Tableau has a REST API.
• 7:44
  SPEAKER_01: So what that means is we can programmatically interact with Tableau.
• 7:49
  SPEAKER_01: Uh you know, engineers who love APIs.
• 7:53
  SPEAKER_01: So what we ended up started with is we uh started with building a Python module that just interact with this API uh because there are available endpoints, so there's an endpoint to trigger a refresh using the API.
• 8:06
  SPEAKER_01: There's an endpoint to do a lot of other things.
• 8:08
  SPEAKER_01: So we use those endpoints made available by Tableau to trigger these refreshes.
• 8:13
  SPEAKER_01: So that worked.
• 8:14
  SPEAKER_01: We're able to just use Python, just write the scripts, and we're able to refresh Tableau dashboards when pipeline finished running.
• 8:21
  SPEAKER_01: However, that is not scalable because what that means is everybody that needs to do this in every project needs to then copy this code across those data pipeline.
• 8:32
  SPEAKER_01: It's not scalable.
• 8:34
  SPEAKER_01: It's also not maintainable because when you distribute code like that, it means people will be running on stale version because everybody just duplicates the code, which is not a very good uh practice, but we it proved that we can solve the problem, basically.
• 8:50
  SPEAKER_01: So I don't know if you want me to just continue to explain the story or you want me to stop there and you know on the problem statement.
• 8:57
  SPEAKER_03: Yeah, I think that's a great overview of you know what you're dealing with and kind of why Airflow is involved.
• 9:04
  SPEAKER_03: I I think you may have mentioned that you have a provider that you've built to do this within Airflow.
• 9:11
  SPEAKER_03: So tell me a little bit more about that because I think the community, there is like an open source provider for Tableau, but it sounds like that's not what you're using.
• 9:21
  SPEAKER_03: So what was the what was the decision there?
• 9:24
  SPEAKER_01: Oh very interesting one as well.
• 9:26
  SPEAKER_01: So, like I said, we built something, we evaluated that you worked.
• 9:30
  SPEAKER_01: But we're like, wait, we use Airflow a lot in most of our data pipeline.
• 9:36
  SPEAKER_01: Why not just use the Airflow Tableau provider to interact with Tableau?
• 9:41
  SPEAKER_01: Why use our so yeah, we then try to use the Airflow Tableau provider.
• 9:48
  SPEAKER_01: So naturally, because it's already available, um, I mean, kudos to the community, very, very strong community that Airflow has.
• 9:54
  SPEAKER_01: We were able to uh test this, integrate this, see how it works.
• 9:58
  SPEAKER_01: However, we uh it's a roadblock, not that it didn't work, but it worked even do it.
• 10:05
  SPEAKER_01: It was a little bit of you know trial, but the roadblock is uh tabloo cloud that allows uh differ as different modes of authentication basically.
• 10:17
  SPEAKER_01: So it has username and password, PAT, which is personal access token, and also JWT, which is Java Web Token.
• 10:24
  SPEAKER_01: So it has this mode of authentication to Tableau.
• 10:26
  SPEAKER_01: However, what happened with the F flow community provider, uh Tableau provider is uh it's supported the PAT, uh, which is personal access token, username and password initially, so that was fine.
• 10:41
  SPEAKER_01: However, there were a lot of issues with the personal access token because the way Tableau made that available is you cannot have two connections running at the same time with the same PAT.
• 10:52
  SPEAKER_01: So what that means is that you cannot exactly, you cannot do you cannot run things simultaneously using the PAT.
• 10:59
  SPEAKER_01: But we love parallelism as an engineers, so we it was causing a lot of issues, and the maintainers of the provider or Airflow generally, they were having to like respond to a lot of career from users of this provider.
• 11:12
  SPEAKER_01: So I read the issues because I actually followed the GitHub issues, and I felt I saw the conversation, and at some point there has to be like a decision to make to made do we keep having headaches and people keep coming, you know, that there's an issue with this, and we cannot do it without tablet getting involved, or we just remove that particular feature and support the one that is uh easier.
• 11:33
  SPEAKER_01: So, yeah, at some point it was deprecated.
• 11:36
  SPEAKER_01: So PHT was no longer available for authentication, so it was just user-new and password.
• 11:40
  SPEAKER_01: So, and that was the limitation for us um initially because PHT over user animal password.
• 11:46
  SPEAKER_01: I mean, we don't want to use animal password, it's not recommended.
• 11:50
  SPEAKER_01: So, however, the community provider doesn't allow us to support PAT any longer.
• 11:56
  SPEAKER_01: So we're now in the mid-crossroad.
• 11:58
  SPEAKER_01: How do we proceed?
• 12:00
  SPEAKER_01: So that led us to now going the route, okay, fine.
• 12:02
  SPEAKER_01: We will start from what the community provider already gives and build atop of that to build our own custom provider that then allows us to use our mode of authentication, which is PAT.
• 12:12
  SPEAKER_01: So that was what led us to building our custom provider because the one available by the community has you know duplicated the PAT approach of authenticating to tablet cloud.
• 12:22
  SPEAKER_03: Yeah.
• 12:22
  SPEAKER_03: Okay, very cool.
• 12:23
  SPEAKER_03: That totally makes sense.
• 12:25
  SPEAKER_03: Those limitations on authentication can be really tricky to work around, but certainly a benefit of Airflow that you can just build your own provider and sounds like that's working pretty well for you.
• 12:35
  SPEAKER_00: This episode is brought to you by Astronomer, the team behind Astro, a managed Airflow platform built for data engineering teams.
• 12:44
  SPEAKER_00: From batch pipelines to training models and wiring up agents.
• 12:48
  SPEAKER_00: Astro handles the infrastructure, upgrades, and scaling so you can focus on building.
• 12:54
  SPEAKER_00: Check it out at astronomer.io Yeah, cool.
• 13:00
  SPEAKER_03: Well, that's yeah, a really interesting use case.
• 13:02
  SPEAKER_03: Appreciate you sharing that.
• 13:03
  SPEAKER_03: One of the other things that I wanted to dig into, a bit of a separate topic, was you mentioned using CICD pipelines and how you validate DAGs before they're deployed to production.
• 13:16
  SPEAKER_03: This is, of course, a really important topic, especially, well, maybe for all companies, but especially for really big ones like a JLR where you've got a lot of airflow usage going on and your production pipelines are really critical.
• 13:29
  SPEAKER_03: Tell me about what that process looks like and you know how how well it works in the day-to-day in terms of testing your DAGs.
• 13:36
  SPEAKER_01: Yeah.
• 13:37
  SPEAKER_01: Again, we there's also always a problem statement, and you already mentioned one of them.
• 13:43
  SPEAKER_01: One of them is that GLI is a very big organization.
• 13:47
  SPEAKER_01: So that means, you know, you can imagine the number of engineers, number of teams that runs data pipeline.
• 13:55
  SPEAKER_01: So one of our best practices is to use CICD to ensure we follow a very solid pattern across the teams.
• 14:04
  SPEAKER_01: And the problem statement is this you can develop Airflow locally or in your dev environment, but when you move your data to production, is that it's not business as usual.
• 14:16
  SPEAKER_01: You don't want things to break, you don't want to cause, you don't want you know your downstream users to uh be frustrated, basically.
• 14:24
  SPEAKER_01: So the idea is this how can we ensure that before things get to the production, we do our sanity checks to ensure we check for you know errors that could happen so that when someone is trying to push their DAG into production, we already catch common issues that would have affected the production environment.
• 14:44
  SPEAKER_01: So the way it works is this we have in our CSD pipeline, we have a couple of checks.
• 14:50
  SPEAKER_01: In fact, we have uh what we call DAG validation, you know, and to talk about the DAT validation specifically uh in our CSD pipeline, it mutually checks for things like I'll give you some examples.
• 15:01
  SPEAKER_01: You know, for example, in Flo there's what we call top-level codes.
• 15:05
  SPEAKER_01: And top-level code is more like you can have within your DAG, you can have a script or a function that is making external calls to a database or to somewhere a third-party system.
• 15:15
  SPEAKER_01: And that is very bad for Airflow because FLO, the scheduler, is going to pass a DAG every couple of seconds.
• 15:22
  SPEAKER_01: So, what that means is every time FLO is passing the DAG, you are making connections to a third-party system, external connection.
• 15:28
  SPEAKER_01: So that's very bad.
• 15:29
  SPEAKER_01: So DAG will not essentially break your DAG, but it's not an efficient way to write a DAG.
• 15:34
  SPEAKER_01: So um, because you know, engineers we develop and want to push the DAG.
• 15:39
  SPEAKER_01: So we check for things like this.
• 15:40
  SPEAKER_01: Even though it works, is it working appropriately?
• 15:43
  SPEAKER_01: Is it following the standard?
• 15:44
  SPEAKER_01: Is it following breakfast?
• 15:45
  SPEAKER_01: So we catch something like that before it gets to the production so it doesn't overwhelm the resources that is meant to be used by other DACs.
• 15:52
  SPEAKER_01: Another example is import errors.
• 15:55
  SPEAKER_01: You know, we want to also check because that's one of the most common failure in DAX, import errors.
• 16:01
  SPEAKER_01: Maybe you are relying on a package on your local environment that is not already available in production.
• 16:05
  SPEAKER_01: So you want to cache that before it's getting deployed so it doesn't cause issues in production.
• 16:09
  SPEAKER_01: And recently as well, like Airflow 3 compatibility, there are a lot of breaking changes from F-flow 2 to F flow 3.
• 16:16
  SPEAKER_01: So things like that, you don't want to leave it to you know guesswork or you know, to vibes.
• 16:21
  SPEAKER_01: So you want to cache that before it becomes an issue.
• 16:23
  SPEAKER_01: So we have things like that as well, whereby we check if your DAG is using F flow 3.
• 16:27
  SPEAKER_01: Do you have import issues?
• 16:28
  SPEAKER_01: So if you have import issues, we let you know uh before it gets to production.
• 16:32
  SPEAKER_01: So you can fix it right there and there.
• 16:34
  SPEAKER_01: And we uh, you know, so that's how we kind of just use CICD pipeline, Flood validation, a couple of things to ensure we sanity checks everything that is leaving the developer environment to production environment in Flu.
• 16:46
  SPEAKER_03: Yeah, totally makes sense.
• 16:48
  SPEAKER_03: I can see where that would be really important.
• 16:50
  SPEAKER_03: And do you feel like that's working really well overall?
• 16:53
  SPEAKER_03: Do you catch most issues before they make it to production?
• 16:56
  SPEAKER_01: Yes, yes, it helps a lot.
• 16:59
  SPEAKER_01: I think one thing that yeah, I mean it's it's very useful uh for for a fact because again, it's it's a git, it's it's a git system.
• 17:10
  SPEAKER_01: So you cannot move your DAG across the gate unless the gate is open for you.
• 17:15
  SPEAKER_01: So it catches and not just catches, it blocks issues from getting to the production environment.
• 17:20
  SPEAKER_01: So yeah, it's a very, very liable and um important part of our orchestration and also like deployment process.
• 17:27
  SPEAKER_03: Yeah, definitely good to have those guardrails, both for just the general errors and the best practices, as you mentioned.
• 17:35
  SPEAKER_03: So yeah, very cool.
• 17:36
  SPEAKER_03: Well, thank you for going through that.
• 17:38
  SPEAKER_03: Uh those were both really interesting use cases.
• 17:41
  SPEAKER_03: Just in the interest of time, I always ask my guests a standard final question, and that is what would you most like to see from the Airflow project in the near future?
• 17:49
  SPEAKER_01: Um, AirFlow.
• 17:51
  SPEAKER_01: I'm a very I'm a fan of Airflow.
• 17:54
  SPEAKER_01: I I don't just use Airflow, I teach Airflow a lot.
• 17:58
  SPEAKER_01: So you can very deep in Airflow.
• 18:02
  SPEAKER_01: So one thing that I would like to see is based on what I've just said as well regarding testing and validation.
• 18:10
  SPEAKER_01: There is nothing existing for now in terms of native integration testing for Airflow.
• 18:17
  SPEAKER_01: So what I'm by integration testing is you can do all those sanity checks, but it would be nice to have like a native integration testing.
• 18:24
  SPEAKER_01: So whereby you can already flag dependency issues, you can flag if your pipeline is going to work well before you actually deploy.
• 18:31
  SPEAKER_01: Because sometimes what happens is yes, you can check for import errors, you can check for missing data, but sometimes it's not just that.
• 18:37
  SPEAKER_01: It could be dependency issues, you know, and you don't get to cache that until you know it hits an environment because you have to run it to be able to know existing working work.
• 18:46
  SPEAKER_01: So I think it's very nice to have a native integration testing support in Airflow.
• 18:53
  SPEAKER_01: So that's one.
• 18:54
  SPEAKER_01: The other one on top of my head now is the one I encountered recently as well.
• 18:58
  SPEAKER_01: So I was building like an observability platform to observe Airflow infrastructure.
• 19:04
  SPEAKER_01: So not just the DAG to see how the worker is behaving, the scheduler, the metadatabase, etc.
• 19:11
  SPEAKER_01: Then I was trying to use OTL, which is OpenTelemetry, which is more like it's it's now the next best thing in terms of observability.
• 19:20
  SPEAKER_01: So that's it, that's the future of observability.
• 19:23
  SPEAKER_01: FLO has done a good job there uh because I found out that there is already like instrumentation for Otel.
• 19:28
  SPEAKER_01: However, it's still very, very lacking behind compared to the native or the default or the one, the legacy, let me say legacy, Stat D.
• 19:38
  SPEAKER_01: Um, Stat D is legacy and it's already very matured, but hotel is still maturing.
• 19:42
  SPEAKER_01: So I had a couple of roadblocks.
• 19:45
  SPEAKER_01: I remember how to like build my own Grafana dashboard from scratch because nothing existing, nothing's existing already in the community.
• 19:52
  SPEAKER_01: I'm even thinking of open sourcing that.
• 19:54
  SPEAKER_01: I'll think about it.
• 19:55
  SPEAKER_01: Uh, but yeah, it would be very nice to have better observability, uh, especially native hotel observability in airflow, and also lineage as well.
• 20:03
  SPEAKER_01: So airflow can tell you your DAGs, it can show you your DAGs, you know, dependencies, but it doesn't give you the lineage.
• 20:11
  SPEAKER_01: For example, dependency and also like what depends on your uh your DAGs.
• 20:16
  SPEAKER_01: So you can remember, you can imagine the issue of tablet I mentioned.
• 20:19
  SPEAKER_01: So there's nothing to tell this particular dashboard is depending on your pipeline.
• 20:24
  SPEAKER_01: So it's also be nice to have like native lineage and you know into in Airflow.
• 20:28
  SPEAKER_01: That's very, very good.
• 20:29
  SPEAKER_03: Yeah, absolutely.
• 20:31
  SPEAKER_03: Those are all great answers.
• 20:32
  SPEAKER_03: And you're not the first guest I've had on to talk about hotel support within Airflow as well as lineage.
• 20:38
  SPEAKER_03: So yeah, common issues for a lot of data engineers out there.
• 20:41
  SPEAKER_03: Well, Najib, thank you so much for coming on.
• 20:44
  SPEAKER_03: It's been really great speaking with you.
• 20:45
  SPEAKER_03: Just to close us out, what is the best way for folks to get in touch with you?
• 20:49
  unknown: Yeah.
• 20:49
  SPEAKER_01: The best way is on LinkedIn, Najib Sliman, that's my name.
• 20:52
  SPEAKER_01: Also on X as well, Najib Slimer.
• 20:55
  SPEAKER_03: Okay, great.
• 20:56
  SPEAKER_03: We'll throw a link to that stuff in the show notes.
• 20:58
  SPEAKER_03: Well, thank you again, Najem.
• 20:59
  SPEAKER_03: This has been a great conversation.
• 21:01
  SPEAKER_01: Yeah, thank you so much for Kinsang.
• 21:03
  SPEAKER_01: Thank you.
• 21:04
  SPEAKER_00: Thank you for listening to the data flowcast.
• 21:08
  SPEAKER_00: Check the show notes for links to everything mentioned today.
• 21:11
  SPEAKER_00: If you're enjoying the podcast, subscribe so you never miss an episode.
• 21:16
  SPEAKER_00: And leave us a five star review.