Latest / The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI / Data Quality and Observability at Tekmetric with Ipsa Trivedi

Transcript

Data Quality and Observability at Tekmetric with Ipsa Trivedi · The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI

• 0:00
  SPEAKER_03: Some people use Azure, some people just want plain email, some people have SFTPs and AWS and, you know, all of those different things.
• 0:08
  SPEAKER_03: And on a high level, if I think from an Airflow perspective, it would look like it is standard that you have some sources of data and then it's a data lake and then you transform and send it out to all these different people.
• 0:20
  SPEAKER_03: We are able to set our own standards of how our pipelines are supposed to look like.
• 0:26
  SPEAKER_03: But at the same time, we're also able to fit all these different nuances and different customizations that every single user wants.
• 0:33
  SPEAKER_03: And we're not worried about how we're going to achieve that.
• 0:42
  SPEAKER_00: You're listening to the Data Flowcast, where we unlock the secrets of mastering Airflow for data engineering and AI.
• 0:52
  SPEAKER_01: Hey Airflow community, Airflow Theory is here and we are taking it on the road.
• 0:57
  SPEAKER_01: Join the Astronomer Roadshow for an exclusive deep dive into the latest features, hands-on demos, and expert insights, all in a city near you.
• 1:05
  SPEAKER_01: We'll be in London in May, New York City in June, Sydney in July, San Francisco in July, and Chicago in August.
• 1:14
  SPEAKER_01: Don't miss your chance to connect with the Airflow community and get the inside scoop on what's next.
• 1:19
  SPEAKER_01: Register now.
• 1:20
  SPEAKER_01: The link is in the show notes.
• 1:23
  SPEAKER_02: Hi everyone.
• 1:24
  SPEAKER_02: Welcome to the Data Flowcast.
• 1:26
  SPEAKER_02: I'm your host, Kenton Danis, and today I'm joined by Ipsa Trevedi, who is a lead data engineer at Techmetric.
• 1:33
  SPEAKER_02: Ipsa, how are you today?
• 1:34
  SPEAKER_03: I'm good.
• 1:35
  SPEAKER_03: How about you?
• 1:36
  SPEAKER_02: Good.
• 1:37
  SPEAKER_02: Good.
• 1:37
  SPEAKER_02: Are you having a good spring?
• 1:40
  SPEAKER_03: I am so far here.
• 1:41
  SPEAKER_03: Today is Houston Rodeo.
• 1:43
  SPEAKER_03: So I'm going for that.
• 1:44
  SPEAKER_03: So I'm so excited.
• 1:46
  SPEAKER_02: Fun.
• 1:46
  SPEAKER_02: That sounds really interesting.
• 1:48
  SPEAKER_02: I have never been to anything like that.
• 1:50
  SPEAKER_03: You have great games and awesome food.
• 1:54
  SPEAKER_03: So much fried food, so many desserts.
• 1:57
  SPEAKER_03: It's the best Friday.
• 1:58
  SPEAKER_03: It's going to be the best Friday.
• 2:00
  SPEAKER_02: Yeah, yeah.
• 2:01
  SPEAKER_02: That's really great.
• 2:02
  SPEAKER_02: We love recording these right before people start on their fun weekends.
• 2:06
  SPEAKER_02: So yeah, that's awesome.
• 2:08
  SPEAKER_02: Cool.
• 2:08
  SPEAKER_02: I won't keep you too long from all of your fun plans.
• 2:11
  SPEAKER_02: So let's go ahead and dive in.
• 2:13
  SPEAKER_02: For today's podcast, we're going to discuss why Techmetric, which is a growing startup, chose Airflow as their central orchestrator.
• 2:21
  SPEAKER_02: So to start with some background, what does Techmetric do?
• 2:25
  SPEAKER_03: So Techmetric is the largest cloud-based shock management system in the United States.
• 2:31
  SPEAKER_03: We are about 10 years young, I want to say.
• 2:35
  SPEAKER_03: And so diving a little bit about how auto repair shops function and where the shock management system comes into play here.
• 2:43
  SPEAKER_03: So, say for instance, you have a car, you have a problem with your car, and then you call an autorepair shop, and then you book an appointment, you go in, and then you talk to a service advisor who will take your concerns in a ticket.
• 2:57
  SPEAKER_03: A ticket is created.
• 2:59
  SPEAKER_03: Somebody's going to inspect your car.
• 3:00
  SPEAKER_03: They're going to give you like seven things that are wrong with your car.
• 3:03
  SPEAKER_03: I hope not, but seven things wrong with your car.
• 3:06
  SPEAKER_03: And then they send it to you for approval and you say, okay, let's fix the five things.
• 3:10
  SPEAKER_03: I don't want to do the other two things now.
• 3:13
  SPEAKER_03: So the technician will take it in, they will work on your car, you get notification for payment, you pay it, you take your car away back home, healthy, happy.
• 3:22
  SPEAKER_03: Now, this is the customer-facing side of an autorepair shop.
• 3:26
  SPEAKER_03: But behind that table, there is a whole team that's functioning and working very hard to give you the best customer experience that they can.
• 3:33
  SPEAKER_03: And this team is going to do digital inspections on your car.
• 3:37
  SPEAKER_03: There are a team of technicians who are working on your car.
• 3:40
  SPEAKER_03: Before they start working on your car, they want to probably order parts for a car, like new parts.
• 3:45
  SPEAKER_03: Then once the parts arrive, they work on your car and then sending payments and again the same thing till you take your car back.
• 3:51
  SPEAKER_03: So that is what happens on the other side of the table.
• 3:54
  SPEAKER_03: But then there is also a third side to this where shop owners come into play.
• 4:00
  SPEAKER_03: These are the owners of, say, individual shops or these are organizations that hold many hundreds of shops.
• 4:07
  SPEAKER_03: And these people want to make sure that the business is running well day in and day out.
• 4:11
  SPEAKER_03: You're getting the best customer experience that you possibly can.
• 4:13
  SPEAKER_03: They want to remove any hurdles in this whole experience.
• 4:17
  SPEAKER_03: So all of this is going to happen when one customer comes in with a problem.
• 4:22
  SPEAKER_03: Now, this entire life cycle that we just discussed in the last couple of minutes, that is what a shop management system strives to accomplish.
• 4:32
  SPEAKER_03: So what we're building is a one-stop solution so that the shops don't have to go across applications and they can function and do everything, all their day in and day out businesses with just one app.
• 4:44
  SPEAKER_03: So that is what Techmetric does.
• 4:47
  SPEAKER_02: That's so interesting.
• 4:48
  SPEAKER_02: I think auto repair is something that probably everybody, at least in the US, where we're so car-centric, has had to encounter.
• 4:56
  SPEAKER_02: I know I myself have and never gave a second thought to how all of it is coordinated behind the scenes.
• 5:01
  SPEAKER_02: So one of those things that I think just how just works for the end user, which is the ideal state, but takes a lot of effort behind the scenes to make that work.
• 5:12
  SPEAKER_02: Yeah, that's really cool.
• 5:14
  SPEAKER_02: So what does your team do?
• 5:16
  SPEAKER_03: So my team is the data team, which is responsible for managing, centrally managing all the data that goes in and out of our systems.
• 5:26
  SPEAKER_03: Now, this team is fairly young.
• 5:27
  SPEAKER_03: It was started in 2024.
• 5:29
  SPEAKER_03: That was early 2024, about maybe five or six months before I joined the team.
• 5:34
  SPEAKER_03: So if we're very new.
• 5:35
  SPEAKER_03: But then with this shop management system, we have a lot of data and data work that was previously done by other people.
• 5:42
  SPEAKER_03: Now we're just moving some of the responsibilities to the data team.
• 5:46
  SPEAKER_03: And we're also starting like new projects with the data team.
• 5:49
  SPEAKER_03: So what we really want to do is centralize the data that powers our reporting dashboards and our internal operations that take care of how our business is doing, our business success, driving our customer success.
• 6:01
  SPEAKER_03: So we want to power all the dashboards and all the applications and analytical use cases and data warehouses, all of those things that power all of these different use cases.
• 6:11
  SPEAKER_03: One of the main ones that we're focused on right now is that we want to provide data services to our customers.
• 6:17
  SPEAKER_03: So for like a customer has a point B that they want to reach, where B is a business question to which they're looking for an answer.
• 6:26
  SPEAKER_03: And the point A being, okay, we have this data.
• 6:29
  SPEAKER_03: Now, how do we make our customers or help our customers go from point A, which is we have all this data for you, and point B, which is the question they're trying to answer before they take a decision.
• 6:40
  SPEAKER_03: So everything that needs to happen in between sending them raw data to like coordinating if they have data teams, coordinating with the data teams, or just building reports and sending reports out to them.
• 6:50
  SPEAKER_03: All of those things is what data team is doing.
• 6:53
  SPEAKER_03: Now, this is again like it is a new use case we're working on, but we're very excited and we're already seeing some success, all thanks to Airflow.
• 7:00
  SPEAKER_02: Yeah, yeah, that's great.
• 7:02
  SPEAKER_02: So let's dig into that airflow piece a little bit more.
• 7:06
  SPEAKER_02: I think as a new team, as you said, you're in what I would consider both the enviable and also very difficult position of having to build everything out from scratch.
• 7:16
  SPEAKER_02: But at least then you get to make a lot of choices about which tools you're using.
• 7:21
  SPEAKER_02: So as you're focusing on building out that tech stack for your team, why did you choose Airflow as your central orchestrator?
• 7:29
  SPEAKER_03: So one of the main reasons of choosing Airflow is that it is the kind of tool that you can bend and mold in different ways, and you don't have to worry about certain features that it may not offer.
• 7:42
  SPEAKER_03: Because at least till this point, we haven't faced a scenario like that, minus a couple of times where, you know, we we thought that a feature was missing, and this engineer from our team, Ovi, he actually submitted PRs to the Airflow repository, and then we could get our feature released, and then we were able to use it.
• 8:01
  SPEAKER_03: So the customizability and the usability of Airflow is so flexible that it keeps our tech stack simple.
• 8:08
  SPEAKER_03: Like that, like we discussed all these different things that Data Team does.
• 8:11
  SPEAKER_03: There are 1500 tools out there that you can use and combine and make it work.
• 8:15
  SPEAKER_03: But then, do you really want to use 1500 tools and then worry about security, infrastructure, take all that headache, or have one tool that's going to achieve all of that for you?
• 8:25
  SPEAKER_03: So that's where Airflow was really like a no-brainer for us.
• 8:28
  SPEAKER_03: Now, this decision was taken before I joined the team, but I would imagine these were the thought process that they used to decide on Airflow.
• 8:37
  SPEAKER_03: And I did speak with them, the people who decided that Airflow should be taken, adopted in the team.
• 8:43
  SPEAKER_03: And another reason for that was also that it is open source, which means that it is very easy to see what all functionalities it offers beyond the documentation.
• 8:53
  SPEAKER_03: And sometimes it's just easy to find that kind of functionality inside the repo, inside the actual code, and just understand how Airflow really works.
• 9:03
  SPEAKER_03: So it does not abstract away that component of, okay, what's happening in the background?
• 9:07
  SPEAKER_03: What's happening in the background?
• 9:08
  SPEAKER_03: So it makes it easy to easy to trust and easy to learn, which is all great things.
• 9:13
  SPEAKER_03: The UI is so intuitive.
• 9:15
  SPEAKER_03: I have worked on Airflow in my previous experience where our use case was very complex and it was a much bigger company with a lot of data and a lot of complexities.
• 9:26
  SPEAKER_03: But then Airflow fit perfectly in that scenario.
• 9:29
  SPEAKER_03: And now I'm here where we're starting out.
• 9:31
  SPEAKER_03: It's a new team, it's new use cases.
• 9:34
  SPEAKER_03: We're still building our frameworks and fundamentals.
• 9:36
  SPEAKER_03: And yet, Airflow is still the best piece for it.
• 9:39
  SPEAKER_03: So whether you're looking at like small use cases or more standard use cases, or you're looking to bend Airflow to its limits, everywhere it fits in really well.
• 9:49
  SPEAKER_03: And that's what I like the most about this tool.
• 9:52
  SPEAKER_02: Yeah, yeah, I love to hear that.
• 9:54
  SPEAKER_02: And I think it's so consistent with what we hear with a lot of companies that we talk to, is that as you said, Airflow is so flexible and so made to be a general orchestrator that it really works for any different use case.
• 10:08
  SPEAKER_02: So that's why you see it across so many different companies.
• 10:11
  SPEAKER_02: And I would imagine for you, you said you had used Airflow in your previous job, which was a very different use case.
• 10:17
  SPEAKER_02: But that's another reason that companies often choose to use Airflow, is that most data engineers have experience with it because you're going to encounter it in almost every organization.
• 10:28
  SPEAKER_02: So it's easy to hire people and onboard them where as you're helping to build that out, you've already used Airflow and you already know a lot of things.
• 10:36
  SPEAKER_03: So there's a lot of background knowledge that comes with it.
• 10:38
  SPEAKER_02: Yeah.
• 10:38
  SPEAKER_02: Yeah.
• 10:39
  SPEAKER_02: That's great.
• 10:40
  SPEAKER_02: As you've implemented it for your use cases or are starting off doing that, are there any particular pieces of Airflow that you think are most helpful?
• 10:49
  SPEAKER_02: Or you've mentioned its flexibility.
• 10:51
  SPEAKER_02: Are you guys looking at custom how your Airflow works for your use case or anything like that?
• 10:56
  SPEAKER_02: Wow.
• 10:56
  SPEAKER_03: Yeah.
• 10:57
  SPEAKER_03: While I was talking about all these different use cases, I was thinking about how we have so many different data sources from where our data comes in.
• 11:05
  SPEAKER_03: That's transactional data, non-transactional data.
• 11:08
  SPEAKER_03: It comes from places like emails, all the way to AWS database service called RDS.
• 11:13
  SPEAKER_03: So we have all these varieties of different data sources.
• 11:16
  SPEAKER_03: And then we want to collect all of that into a data lake, which is going to act as a raw data repository.
• 11:22
  SPEAKER_03: And then on top of it, we want to fit all our different use cases, like raw data, moving to data warehouse, sending it out to customers, and of course, before that, do some transformations.
• 11:32
  SPEAKER_03: Now, all our customers are going to have different kinds of setups.
• 11:36
  SPEAKER_03: Some people use Azure, some people just want plain email, some people have SFTPs and AWS and all of those different things.
• 11:44
  SPEAKER_03: And on a high level, if I think from an Airflow perspective, it would look like it is standard that you have some sources of data and then it's a data lake, and then you transform and send it out to all these different people.
• 11:57
  SPEAKER_03: We are able to set our own standards of how our pipelines are supposed to look like.
• 12:02
  SPEAKER_03: But at the same time, we're also able to fit all these different nuances and different customizations that every single user wants.
• 12:09
  SPEAKER_03: And we're not worried about how we're going to achieve that.
• 12:13
  SPEAKER_03: Like we know how to use, because it is a Python-based tool, you can use all these oops concepts and you can create your DAGs and they're just very easy, reusable, but also you're able to configure them in different ways.
• 12:28
  SPEAKER_03: So I feel like that is one of the best things that I've found with Airflow till now.
• 12:34
  SPEAKER_03: And it's also a small nuance.
• 12:36
  SPEAKER_03: If I, this probably may not be the reason anybody would choose Airflow, but I find it very convenient that very recently I was going through Airflow repository and I was just looking for some information that I cannot recall right now.
• 12:47
  SPEAKER_03: But I found this dry run method that you can use on operators.
• 12:52
  SPEAKER_03: And we had a use case where we don't want to send emails from our lower environments, but we still want to see if the setup is exactly how it's supposed to be before we release it into production.
• 13:03
  SPEAKER_03: And this dry run that I found, it is a minor thing, but putting it on a task instance and running it in lower environments and not in higher environments.
• 13:12
  SPEAKER_03: Like these are minor things, but they add so much to your convenience.
• 13:16
  SPEAKER_03: And you always know there's going to be something that's probably not in documentation, but you'll definitely find in the repository that you can use.
• 13:23
  SPEAKER_03: So those are the kind of awesome things about AirTru, I would say.
• 13:27
  SPEAKER_02: Yeah, that's so interesting.
• 13:28
  SPEAKER_02: So you said the feature is called dry run.
• 13:32
  SPEAKER_02: Yeah.
• 13:35
  SPEAKER_00: This episode is brought to you by Astronomer, creators of Astro, the leading fully managed airflow service designed for the modern data stack.
• 13:44
  SPEAKER_00: Whether you're orchestrating large language models, implementing a data mesh architecture, or streamlining complex workflows, Astro empowers your team to focus on innovation, not infrastructure.
• 13:56
  SPEAKER_00: To see how you can maximize Airflow's potential within your data ecosystem, visit www.astronomer.io.
• 14:04
  SPEAKER_02: It is not every day that I learn about a new Airflow feature that I have never heard about before.
• 14:10
  SPEAKER_02: I totally see where that would make sense.
• 14:13
  SPEAKER_02: It made me think of this.
• 14:15
  SPEAKER_02: You can do this kind of in a manual way with open source airflow, but on our managed Airflow Service, Astro, we have a feature for connections management where we allow users to override different pieces of the connection for different environments so that when you're switching between like dev and prod, you're testing against your dev database or your prod database without having to update the connection name or any of your code.
• 14:36
  SPEAKER_02: So what you're describing is just like that before the actual operator, which I think is so interesting and I'm frankly a little mind blown that I have never heard about this before.
• 14:46
  SPEAKER_02: That sounds great.
• 14:48
  SPEAKER_03: Yeah, it's just these nut tiny things that you find.
• 14:51
  SPEAKER_03: And then this was one thing that I found previously was setting task.
• 14:55
  SPEAKER_03: You can set task instances to different task status outside from Airflow.
• 15:00
  SPEAKER_03: So if you have a process that you want to, I don't know, start your airflow deck for or externally mark something success from outside, and you can easily do those kind of things.
• 15:11
  SPEAKER_03: So it's just, it just makes it so easy.
• 15:14
  SPEAKER_02: Yeah, yeah, I love that.
• 15:15
  SPEAKER_02: Definitely I am also somebody who is liable to just start going through the airflow code when I'm looking for something.
• 15:22
  SPEAKER_02: Obviously, we my team tries to make sure that there's lots of documentation and resources available for folks.
• 15:28
  SPEAKER_02: But sometimes it is really nice, just as you said, having all of that open source code available so you can just dig in and see how it works.
• 15:34
  SPEAKER_02: And obviously, sometimes you find features that you didn't know about, which is great.
• 15:38
  SPEAKER_03: Exactly, exactly.
• 15:40
  SPEAKER_03: And even just being able to submit that code, like a new feature.
• 15:44
  SPEAKER_03: Now, Airflow has so many out-of-the-box operators, right?
• 15:47
  SPEAKER_03: And at least till now for all our exporting use cases where we're connecting to all these different providers, we haven't faced a problem yet where we've had to worry about, okay, this operator is not available out of the box with Airflow.
• 16:01
  SPEAKER_03: But if there were, we could just create a custom operator of our own.
• 16:04
  SPEAKER_03: There's no problem with that.
• 16:05
  SPEAKER_03: If we feel like that's the kind of use case generally everybody will really benefit from, you submit a PR and you get it in the Airflow repository.
• 16:14
  SPEAKER_03: So it's very easy to just make those kind of changes and being open source is very hurtful.
• 16:20
  SPEAKER_02: Yeah, absolutely.
• 16:22
  SPEAKER_02: It is a self-reinforcing positive cycle, right?
• 16:25
  SPEAKER_02: Where more and more companies and people use Airflow and contribute back to it, and more and more people benefit from it and then can use that, which is which is great.
• 16:34
  SPEAKER_02: Yeah.
• 16:35
  SPEAKER_02: So you folks are in kind of the early stages of your Airflow implementation.
• 16:42
  SPEAKER_02: And I'm curious where you're looking to go forward with it.
• 16:46
  SPEAKER_02: Do you envision once you have everything implemented, you'll use Airflow for more use cases, or what are you looking forward to?
• 16:54
  SPEAKER_03: Yeah.
• 16:55
  SPEAKER_03: So we have a lot of next steps.
• 16:57
  SPEAKER_03: The biggest next goal that we want to achieve is have a solid design for data quality and data observability.
• 17:05
  SPEAKER_03: Because of course, if this data is going to external clients who want to drive their decisions on top of it, we have to make sure the data we're giving them is of the most optimal quality.
• 17:15
  SPEAKER_03: So we want to achieve that in a phase-wise manner where initially we just want to use the core Airflow functionalities and have custom rules and checks for all our data sets between like sequential pipelines and before the data goes out to our clients.
• 17:30
  SPEAKER_03: And then we also want to think about the concept of self-healing pipelines with Airflow.
• 17:35
  SPEAKER_03: I've done that in the past and it works pretty well for, well, I think it'll work very well for the use case that we're talking about for our data team.
• 17:43
  SPEAKER_03: And then we want to think about more advanced data quality tools that we can fit and integrate with Airflow.
• 17:48
  SPEAKER_03: I did see Airflow offers.
• 17:50
  SPEAKER_03: Of course, it offers more connections and integrations with more data quality tools, et cetera.
• 17:54
  SPEAKER_03: So that is one thing, but I think eventually, because we are building our fundamentals and framework right now, eventually we do want to expand it to all these different teams in our company as well, because not every team is going to have data teams.
• 18:10
  SPEAKER_03: And we're fairly young, so there's one data team right now.
• 18:13
  SPEAKER_03: But we want to democratize our data as well.
• 18:15
  SPEAKER_03: And we want to build something like a self-serving data platform kind of scenario where the data team owns the left side of the pipeline, the extraction kind of type of pipeline.
• 18:27
  SPEAKER_03: And then other teams can easily extend the functionalities that we give on the data platform, and they can build their own databases or data sets or reports on top of it, which kind of makes it like a data lake plus data mesh kind of architecture.
• 18:41
  SPEAKER_03: And that's where I think I would assume that we would go with extending our capabilities to all these different teams as well.
• 18:48
  SPEAKER_02: Yeah, I love both of those plans.
• 18:50
  SPEAKER_02: I mean, data quality and observability is so important, especially for those like end user-facing data products that you're powering, where there's a huge impact to, in this case, anybody getting their car repaired, which is like everybody, if things don't work.
• 19:06
  SPEAKER_02: So, yeah, that data quality and observability is something that Astronomer is heavily focused on and investing in ourselves as well, because we've seen it's so important to most of the use cases out there.
• 19:18
  SPEAKER_02: So that's that's great that you're already thinking about that, even in kind of these early stages.
• 19:23
  SPEAKER_02: And I think the idea of a democratized central platform is so interesting.
• 19:27
  SPEAKER_02: I've talked to lots of people on this podcast who work for these giant companies that have had data teams for many years, and that's where they've gone with it.
• 19:36
  SPEAKER_02: It's a really common pattern that we see because it works really well and because airflow works really well for it, right?
• 19:43
  SPEAKER_03: Exactly.
• 19:44
  SPEAKER_03: You can just sequence your pipelines and it just makes it very easy.
• 19:48
  SPEAKER_02: Yeah, yeah, totally.
• 19:50
  SPEAKER_02: That I think is a good lead in to my standard final question as you're looking forward and have all these grand plans for where your team is going.
• 20:00
  SPEAKER_02: And my standard final question is what would you most like to see from the Airflow project in the near future?
• 20:05
  SPEAKER_03: No, see, that's a question where I had an answer for it, I guess four or five, four, maybe three years back, because our use cases were complex that we were able to find things that we felt like I wish Airflow had that.
• 20:21
  SPEAKER_03: And I remember it was like a right before Data Factory came out, we were in this position where I wish Airflow had that.
• 20:29
  SPEAKER_03: And then just maybe I think a couple of months later, Data Factory was released.
• 20:32
  SPEAKER_03: So, like those kind of things, I think back then we did have more things, but right now, because our use cases are so standard, I feel like at this point Airflow is really fitting into all the difference that we want to make in the industry.
• 20:47
  SPEAKER_02: Yeah, yeah, that's great.
• 20:49
  SPEAKER_02: Did you mean DAG Factory?
• 20:50
  SPEAKER_02: Was that is that what you said?
• 20:51
  SPEAKER_02: DAG Factory.
• 20:52
  SPEAKER_02: Yeah, that is a great project for sure.
• 20:54
  SPEAKER_02: And I think also when this podcast comes out, I believe Airflow 3 will have recently been released.
• 21:00
  SPEAKER_02: So there will be some big new changes.
• 21:02
  SPEAKER_02: But we've talked about this a lot in the lead up to Airflow 3 is that when, you know, features are needed by a lot of the community, they get developed.
• 21:12
  SPEAKER_02: And that's where, you know, a lot of the things that are coming in major and minor releases are coming from like the Airflow survey, where users say, Hey, this is what I need.
• 21:22
  SPEAKER_02: So I I think your experience of like, well, I did need this, but now it exists is common.
• 21:28
  SPEAKER_02: Yeah, which is we love to see that from the project for sure.
• 21:31
  SPEAKER_03: Yeah.
• 21:32
  SPEAKER_03: It's really making a difference.
• 21:33
  SPEAKER_03: I want to say Airflow is in my 10 years of data engineering experience.
• 21:38
  SPEAKER_03: I want to say by far Airflow has been my best experience.
• 21:42
  SPEAKER_02: Yeah, yeah.
• 21:43
  SPEAKER_02: I love that.
• 21:44
  SPEAKER_02: That's great.
• 21:45
  SPEAKER_02: I think that's a good place to close out.
• 21:48
  SPEAKER_02: So, Ipsa, it's been really wonderful having you on and chatting with you about your Airflow experience and where techmetric is going.
• 21:55
  SPEAKER_02: So, what is the best way for folks to get in touch with you?
• 21:58
  SPEAKER_03: So you can reach out to me at my email ID.
• 22:01
  SPEAKER_03: That's my first name ipsa.mylastname Trivedi at gmail.com.
• 22:07
  SPEAKER_03: Or you can just Google me on not Google me, but find me on LinkedIn with my name Ipsa Trivedi and you see me.
• 22:15
  SPEAKER_02: Yeah, of course.
• 22:16
  SPEAKER_02: And we will put all of that in the show notes for folks so they don't have to look for it.
• 22:20
  SPEAKER_02: Okay, well, again, it's been wonderful talking to you, and we really appreciate you coming on the podcast.
• 22:25
  SPEAKER_03: Thank you so much.
• 22:26
  SPEAKER_03: Thank you for your time, and this was very fun.
• 22:30
  SPEAKER_00: Thanks for joining us for the latest episode of the Data Flowcast, mastering airflow for data engineering and AI.
• 22:37
  SPEAKER_00: You'll find links in the show notes to any resources mentioned in today's show.
• 22:41
  SPEAKER_00: If you're enjoying our podcast, please subscribe so you never miss an episode and leave us a five star review.