Latest / Agentic Conversations (formally mlops.community) / The Rise of Modern Data Management // Chad Sanderson // #226

Transcript

The Rise of Modern Data Management // Chad Sanderson // #226 · Agentic Conversations (formally mlops.community)

• 0:00
  My name is Chad Sanderson. My title is Chief Executive
• 0:03
  Officer. My company is gabled AI.
• 0:07
  Well, I drink tea. Does that count instead of
• 0:09
  coffee? Welcome back everyone.
• 0:11
  We are in another ML OPS community podcast session.
• 0:15
  I am your host Dimitrios. And today talking with Chad
• 0:17
  Sanderson, he dropped a few bombs on me and this is probably
• 0:21
  I think this is the third time that he's been back.
• 0:24
  And to be honest, he does not disappoint anytime.
• 0:27
  He is one of the greatest thought leaders in this space of
• 0:32
  data platforms, data quality, data engineering, you name it.
• 0:39
  He has strong stories. And I think the reason that I'm
• 0:44
  able to latch on to the stories that he tells is that he
• 0:48
  encompasses and articulates what people are feeling and talking
• 0:52
  about in the community, on LinkedIn, just in general so
• 0:57
  well. And he harps on why it doesn't
• 1:02
  need to be like this. First of all, he talks about the
• 1:04
  problem. He has a deep understanding of
• 1:07
  the problem from his experiences as a data engineer, and he knows
• 1:11
  that data engineers can be a bottleneck sometimes when you
• 1:14
  have these different roles, like a data scientist or a data
• 1:18
  analyst are asking a data engineer to go and figure out,
• 1:22
  go debug why my data isn't coming in the same format that
• 1:26
  it was a day ago or why my machine learning model all of a
• 1:28
  sudden is going haywire. And I think it's a data quality
• 1:31
  issue. He's able to talk and harp on
• 1:35
  why data engineering does not need to be 90% debugging.
• 1:40
  He also goes through what it looks like, what a traditional
• 1:45
  analytics system looks like, what a traditional data platform
• 1:48
  system looks like, and how these systems currently break down and
• 1:54
  why he has set out to fix it. What he's doing at his new
• 1:58
  company, Gable was really cool to see because I knew him when
• 2:03
  he was at Convoy and he was talking a lot about the problems
• 2:06
  that they were having there and also the issues that they were
• 2:09
  trying to solve. And he mentioned that.
• 2:12
  And then he went out and for the last year and a half, he's been
• 2:15
  talking to just about everyone in the data space, talking to
• 2:18
  companies small and large about how they are running their
• 2:22
  systems. And he's in a lucky position
• 2:24
  because he is so well known, he gets to have access to so many
• 2:29
  different people in different sized companies, different
• 2:32
  maturity levels and different quality of their data
• 2:37
  infrastructure. And he can show what it looks
• 2:43
  like in all of these phases and where there are common problems
• 2:48
  across all company sizes, maturity levels, you name it.
• 2:53
  Let's get into this conversation with my man Chad.
• 2:58
  And as a little aside, a little secret, he's going to be
• 3:02
  speaking at the AI Quality Conference.
• 3:05
  We've got a few tickets left, so come join us June 25th in San
• 3:10
  Francisco. Ohh.
• 3:12
  Yeah. And of course, if you liked it,
• 3:14
  give us a share. Let us know what you thought of
• 3:16
  the episode. Rate us on Spotify, subscribe on
• 3:21
  YouTube, all that fun stuff. We will talk to you on the other
• 3:23
  side, Chad Sanderson. It has been a year since we
• 3:33
  talked, I think, and when you came on last time, we were
• 3:37
  beating the drum about data quality.
• 3:39
  It's probably more pertinent than ever before.
• 3:44
  I want to know though. It's been a year, man, what has
• 3:48
  changed? Well, in the last year that
• 3:51
  we've talked, there have been some meaningful, meaningful
• 3:56
  changes. One, I got married so, so
• 4:00
  congratulations, That was great. That happened in October and
• 4:04
  then #2 it started a company left Convoy.
• 4:07
  I now have a a business called Gable dot AI and we are, we are,
• 4:15
  we are off to the races building founders journey is very
• 4:20
  interesting and different than what I've done before, but
• 4:23
  pretty rewarding as well. I believe it.
• 4:26
  So I want to get into Gable a tonne because you pretty much
• 4:31
  created this term data contracts and I know you are thinking
• 4:37
  deeply about data contracts and how to help productize that
• 4:42
  idea. Before we go into that rabbit
• 4:46
  hole though, you've been talking with people for a year, maybe
• 4:50
  over a year on how they're implementing their data
• 4:53
  pipelines, what their data architecture looks like.
• 4:56
  Do you have insights for us and anything that you've learned
• 5:01
  when it comes to setting up proper data infrastructure?
• 5:06
  Yeah, that's a good question. So I think there's a few major
• 5:10
  trends that are starting to emerge that are different from
• 5:15
  the last few years, especially the the sort of data bubble,
• 5:20
  I'll call it of the early twenty 20s where it was the midst of
• 5:26
  COVID, the interest rates were still low and so growing
• 5:31
  startups could afford a pretty large bills for their data
• 5:36
  infrastructure, Snowflake, Databricks, a whole host of
• 5:41
  tools, even if they didn't have that much to show for it.
• 5:45
  That's something that we're seeing is starting to change.
• 5:48
  Less companies are starting to assume that the data will
• 5:52
  inherently be valuable and that teams will just be able to find
• 5:58
  value within it. And more companies are starting
• 6:01
  to demand from the beginning that data teams have a good idea
• 6:06
  of how that data will be used and how it will generate ROI
• 6:10
  before the infrastructure exists.
• 6:13
  And this is actually a callback to the way the data
• 6:17
  infrastructure used to be set up and created in the 80s and 90s,
• 6:22
  before storage and compute was decoupled.
• 6:25
  If you're a company like Ford and you wanted to invest in it
• 6:29
  in a data warehouse, this was an incredibly expensive task.
• 6:33
  And so you had to think very carefully, not just from a cost
• 6:38
  perspective, a literal, you know, storage and compute cost
• 6:41
  perspective, but from a human capital cost.
• 6:44
  How many people are we going to be deploying to answer some data
• 6:47
  question? How many data architects?
• 6:50
  We'll, we'll need to have some tool that's handling all the
• 6:54
  ETL, doing the transformation in some centralised place, what we
• 6:57
  do with the data. And so this is where you you see
• 7:01
  the data warehouse becoming a very popular friend, not because
• 7:05
  it was like a generic system that sat on top of all data, but
• 7:10
  because it was very, very useful for answering a specific set of
• 7:13
  questions that drive that we're driving business value.
• 7:16
  And So what we're seeing is teams starting to think a little
• 7:20
  bit less about building U an end to end stack from day one, or
• 7:26
  primarily thinking about analytics as the main use case
• 7:30
  for data. The lateral.
• 7:32
  The latter is actually a pretty important point because
• 7:36
  analytics for a very long time in a company's life cycle is not
• 7:40
  going to deliver massive business value.
• 7:42
  It's more so getting you information about the product.
• 7:45
  And so it makes sense why product teams really want
• 7:48
  analytics. But this is very different from
• 7:51
  something like invoice reconciliation, which has an
• 7:55
  actual cost to it. And if you don't do it well, it
• 7:58
  can impact the business negatively.
• 8:00
  Same thing with machine learning.
• 8:01
  And we're beginning to see a rise in more of those, what I
• 8:04
  would call operational use cases for data as the primary use
• 8:09
  cases of data infrastructure. And that requires the
• 8:11
  infrastructure to be built a little bit differently.
• 8:14
  Have you seen people do this wrong?
• 8:17
  Well, I think it ties back to to the analytics use case, because
• 8:21
  oftentimes the impetus for our data investment comes from
• 8:26
  product teams who want analytics for their their product
• 8:30
  Organisations can build their data infrastructure in service
• 8:35
  of analytics and not in service of operational systems.
• 8:39
  So as an example, a very common analytics data pipeline is
• 8:45
  you've got Snowflake for storage, you've got a data lake
• 8:48
  and you can just throw all the data that you care about in
• 8:51
  there. You've got something like
• 8:53
  Fivetran to just quickly plug into these services or, or
• 8:57
  databases and move the data into something like Snowflake.
• 9:01
  And you've got debt sitting on top of it to very quickly and
• 9:04
  rapidly transform it. And then you have an analytics
• 9:07
  tool like Looker. This is a very common setup for
• 9:10
  early stage companies and teams. Basically thousands of teams
• 9:15
  have have been using this. The problem is, is like that
• 9:18
  setup does not actually scale nor does IT support the
• 9:22
  operational use cases. One, there's a quality and
• 9:25
  governance issue there, which is if we are extracting data from
• 9:29
  the data producers using a tool like Fivetran, and the producers
• 9:33
  haven't taken any level of ownership over that data,
• 9:35
  meaning they're not treating it as a product, they're not
• 9:38
  treating it as an API, They're not, you know, thinking about,
• 9:43
  they don't have any accountability to it.
• 9:45
  Then when something changes, it means it is 100% the
• 9:49
  responsibility of the data teams to deal with that change.
• 9:54
  Now, in an analytics world, that's OK because if the change
• 9:58
  affects your dashboard, the company doesn't lose any money.
• 10:01
  You just go in and and fix whatever the issue is.
• 10:04
  But if it's a machine learning model or it's an AI system, it
• 10:08
  actually does have an impact. You can't just go back
• 10:11
  retroactively and fix things like, especially if you don't
• 10:14
  know about the change. Like if you've got a, let's say
• 10:17
  that your model is training every night and you have a data
• 10:21
  quality issue that maybe you're test didn't catch for some
• 10:24
  reason, then your model is going to be retrained on, on bad data
• 10:27
  and the predictions will start having a real financial impact
• 10:30
  on the company, right? That's one instance, but invoice
• 10:33
  reconciliation is another. Like if we, if we don't catch
• 10:36
  that problem in a certain time window and deal with it, there
• 10:40
  are real, you know, physical consequences.
• 10:44
  And that means there has to be ownership, there has to be
• 10:47
  visibility of the data as it flows to the system.
• 10:50
  There has to be very clear expectations that are divine
• 10:53
  defined. There has to be some feedback
• 10:57
  loop from the, from the person that's using the data to the
• 11:00
  person who's producing the data. There's a whole sort of set of
• 11:04
  features and functionality that we could talk about.
• 11:06
  But I think the best way of summarising it is in, in the old
• 11:09
  world 80s and 90s, you had centralised data management.
• 11:13
  And you can think about centralised data management in
• 11:15
  the same way you can think about the management of like a, like a
• 11:18
  library, right, where you've got, you've got books coming in,
• 11:21
  they get stored in a catalogue, you've got a librarian and
• 11:24
  they're responsible for what comes in.
• 11:26
  They know everything that goes out.
• 11:28
  They're, they're checking what comes in and what goes out.
• 11:31
  They're organising the books in a certain way so that they're
• 11:34
  easily findable and discoverable.
• 11:36
  And, and you have a staff of librarians and this is their job
• 11:39
  and it's what they do all day. And I think I read that the
• 11:42
  average number of books was like, you know, 20,000, like
• 11:45
  20,000 books in a library, which is not all that money.
• 11:49
  Many, if you're thinking in terms of like data sets, right?
• 11:51
  Some companies have hundreds of thousands or millions of, of, of
• 11:54
  data sets. And what that was sort of the
• 11:58
  old way. And what we're moving to is, is,
• 12:01
  is the Federated way. It's like, well, we don't have
• 12:04
  these librarians anymore that are these centralised kind of
• 12:07
  people managing all the data in the company in determining who
• 12:10
  gets to have access and who who doesn't.
• 12:12
  So how do you do this in more of a Federated environment where
• 12:17
  the people who are actually writing the books are making
• 12:21
  those books available to the people who who want the books?
• 12:23
  What does that system have to look like?
• 12:26
  And you'll find that it looks a lot more like like a bookstore
• 12:30
  or like Amazon, right? Something like that.
• 12:32
  And I think that's the change we're undergoing in in data
• 12:34
  right now. The question that came to my
• 12:37
  mind directly, as you were saying that is just how can we
• 12:41
  make sure that if we are writing a book, other people come, and
• 12:46
  in this metaphor, they come and buy our books.
• 12:51
  And it's not like I just write a book for nobody to read.
• 12:55
  Yeah. So that is 1 area where the
• 12:57
  metaphor slightly falls down because in Amazon, right, the
• 13:01
  purpose of writing the book is to sell it.
• 13:03
  But in data, the purpose of producing data may not be for
• 13:08
  other people to use it. I might produce data as an
• 13:11
  engineer because I need that data to run my application,
• 13:14
  right? Like if I'm, if I'm collecting
• 13:17
  data about a customer goes to my website, I ask them to fill out
• 13:21
  a form and includes their, their name, their age, you know,
• 13:26
  whether or not they've bought any new socks in the last three
• 13:28
  days. I will use all of that
• 13:30
  information as the owner of that service to do interesting things
• 13:34
  like maybe I want to send them a gift card for their birthday.
• 13:37
  Maybe I want to show their name on their profile page.
• 13:41
  If I know they haven't bought any new socks in the last three
• 13:43
  days, maybe I'm going to show socks as like the number one
• 13:46
  item for them to buy in their list of referred items.
• 13:50
  I like I'm actually using that data for something.
• 13:52
  And so I'm not thinking about how other people in the company
• 13:56
  might be using that data as well, right?
• 13:59
  Like if, if I am, if I'm saying, hey, this is useful to me.
• 14:02
  The way most organisations work is that the data teams will come
• 14:05
  in and look at all of this data that's stored in transactional
• 14:08
  system. And they will say, well, this is
• 14:10
  actually really important data for me too.
• 14:12
  Like we would like to know what the average age of all of our
• 14:17
  customers are. Maybe there's something
• 14:19
  meaningful we want to do with that.
• 14:20
  Or we want to know like whether or not, you know, people have
• 14:24
  bought socks in the last three days, Like, is that a relevant
• 14:26
  question for us to continue asking?
• 14:29
  What should we be showing them in those cases?
• 14:32
  And so they will use tools like 5 Trend, these like data
• 14:36
  extraction tools, to kind of forcibly rip the data out of the
• 14:40
  hands of the engineers that are producing them and put that data
• 14:43
  into their own environments where they can analyse it.
• 14:47
  And now you have this problem, right, where the engineering
• 14:50
  team is kind of making the changes to the data that they
• 14:54
  really care about that makes sense for them.
• 14:56
  And that might mean changing stuff, right?
• 14:57
  They might decide, hey, I've got today in my database, I've got
• 15:01
  sort of a first name field and a last name field, but I don't
• 15:04
  think that's relevant, you know, we're wasting space or whatever.
• 15:07
  So I'm gonna create a single column that's just called name.
• 15:10
  Now, as long as it's as long as it is only affecting me, that's
• 15:15
  no big deal, right? It's just my service.
• 15:17
  I've built my service in a way that can handle that change.
• 15:19
  But if these downstream teams are extracting data and they're
• 15:23
  expecting, you know, maybe they're putting into some
• 15:26
  marketing automation or something and they're always
• 15:28
  expecting first name, last name to be in different fields.
• 15:31
  And I've just broken that downstream marketing automation.
• 15:34
  The problem I feel is not that these product engineers don't
• 15:38
  care, it's that they don't know, right?
• 15:41
  Like if you go to any application developer and you
• 15:45
  ask them do you know how your data is being used and you know
• 15:50
  where dependencies on you exist and and what it's being used for
• 15:55
  and how valuable it is to the company.
• 15:57
  Most of these developers are going to say no, I, I, I have no
• 15:59
  clue. And if they don't have any clue,
• 16:02
  then to your point, they are not able to even start thinking of
• 16:06
  how do I make this data available in a safe structured
• 16:09
  way to make it accessible and to make it fit for purpose in the
• 16:12
  application world. It makes total sense to try to
• 16:15
  build in, you know, in this very decoupled way where I'm not
• 16:20
  stepping on anyone's toes. I can move as fast as I want
• 16:23
  within my own service. I don't have to depend on or
• 16:26
  rely on anyone else. I can and and the whole purpose
• 16:29
  of that is is speed. And the reason speed is
• 16:32
  important is because this is the age of rapid iteration and rapid
• 16:37
  experimentation. We know that some features are
• 16:39
  not going to work with customers.
• 16:41
  We know some will. The faster that we can ship
• 16:43
  things and we can try things out, the more that we could, you
• 16:46
  know, kind of hill climb our way to to some sort of of meaningful
• 16:51
  success with that product. But data is not like that.
• 16:56
  If I am computing a metric called profit, by definition
• 17:03
  that metric will take a dependency on some other metric
• 17:07
  called revenue and another metric called cost, and the
• 17:11
  revenue metric will itself take a dependency on the variety of
• 17:16
  sources of revenue within a company.
• 17:19
  When I worked at Microsoft, you know, one source was Bing Ads
• 17:23
  revenue. And then you also had all the
• 17:25
  revenue you got from selling Windows.
• 17:29
  And then you got the revenue you had from the the Microsoft
• 17:32
  Store, right? These are all kind of sources of
• 17:34
  revenue. And then if you trace that line
• 17:36
  all the way back, then you can split it out into the
• 17:38
  individual, you know, line items.
• 17:41
  And so you could there, there are these chains, these trees of
• 17:45
  dependencies that form when you're trying to answer
• 17:49
  questions on data. And that cannot be decoupled,
• 17:53
  right? It's impossible to decouple
• 17:54
  that. And so and, and when you, when
• 17:57
  you step back and you actually look at how the data flows
• 18:00
  throughout a company, it does look more like a supply chain
• 18:03
  where you've got a producer of kind of the raw materials.
• 18:07
  And these might be the the logs that are being emitted from some
• 18:11
  service. And then that data gets
• 18:14
  transformed in some raw layer or silver layer of a data warehouse
• 18:21
  into a bit more normalised, a bit more kind of aggregated.
• 18:24
  And then they get transformed yet again into a domain.
• 18:29
  And so here's all the events for a particular business unit.
• 18:34
  And then from there, they're transformed again into a
• 18:38
  particular metric. And then it's sort of on, on and
• 18:40
  on it goes. And if there's any break in the
• 18:43
  chain at any point, then it has a ripple effect to everyone
• 18:48
  downstream that is leveraging that data.
• 18:50
  And so like one of the things we've kind of been dancing
• 18:53
  around while we're having this conversation is this idea of
• 18:57
  change management, right? If I am a software engineer and
• 19:01
  I'm trying to make changes to my code, there are very well
• 19:05
  understood systems on how to do that without causing damage to
• 19:10
  everyone else that is dependent on that code.
• 19:12
  And we call this DevOps, right? So if I'm, if I'm an engineer
• 19:17
  and I'm adding a new feature that goes through a process,
• 19:21
  there's unit tests and there's integration tests and there's
• 19:25
  pull requests. So humans look at these changes
• 19:29
  and then I get a diff so I can there's things that they could
• 19:32
  easier for the humans to actually review the changes.
• 19:35
  And then there's a whole process that sort of runs to merge the
• 19:38
  branch, the the back into sort of the main trunk of the code
• 19:43
  base. And then there's a whole process
• 19:45
  of ensuring that deployment is safe.
• 19:47
  And then there's tools like Launch Darkly that allow you to
• 19:51
  to toggle that feature on and off.
• 19:53
  And then you've got tools like Amplitude and Mixed Panel that
• 19:56
  actually track how the business users or how the users that
• 20:01
  engage with that feature. All of this is change
• 20:03
  management, right? I've made a change.
• 20:05
  I want to make sure that that change was safe and it had the
• 20:08
  intended effect. But if you look at the data
• 20:12
  world, none of that exists, right?
• 20:15
  None of that infrastructure I just talked about exists.
• 20:17
  So and data changes just as frequently as as software
• 20:21
  changes. If, if, if I'm an engineer and
• 20:23
  I'm making a change to my database, that's data changing.
• 20:27
  If I'm adding a new event that is emitted every time a user
• 20:31
  interacts with the feature, that's, that's, that's data
• 20:34
  changing. If I'm, if I'm removing an
• 20:36
  event, that's, that's data changing.
• 20:38
  So data is changing all the time, yet there is no system for
• 20:43
  the teams actually leveraging that data to give any sort of
• 20:46
  human in the loop review or, or feedback.
• 20:50
  There's no sort of integration testing for data.
• 20:54
  There's no way to actually check to make sure that the data is
• 20:56
  flowing the, the way that it is supposed to.
• 20:59
  These are all things that you detect after the fact, right?
• 21:03
  Like way after the changes have already happened.
• 21:05
  And generally you're looking at the, the, the outcomes, the use
• 21:08
  case like you're looking at, you're saying your machine
• 21:10
  learning model has failed. And that's how I know that a
• 21:14
  data issue has occurred. Yeah, all of a sudden the
• 21:18
  dashboards are starting to go wild, and it's like, ohh, maybe
• 21:21
  that was because my data was a little bit dirtier than I had
• 21:23
  thought, and it exchanged one too many hands, or somebody did
• 21:29
  something upstream that I have no idea how or what.
• 21:33
  And then I have to go and be a detective and figure out what
• 21:38
  exactly who was changing what and where did they change.
• 21:42
  It ohh, yeah, well, I think it's on it's on both sides of the of
• 21:46
  the workflow as well. There's definitely the workflow
• 21:48
  of the a thing has changed. I I've been impacted.
• 21:52
  How do I root cause this and what do I do moving forward?
• 21:56
  Usually that work is put onto the data engineer, because if
• 22:00
  you're a data scientist or an analyst, you don't know that
• 22:04
  it's it's very. Hard just saying hey, it stopped
• 22:07
  working. What's going on?
• 22:08
  Why is this like going all haywire?
• 22:10
  Why did this happen? I have no idea.
• 22:12
  All I know is that it's impacting me.
• 22:14
  So I'm gonna escalate it to my data engineering team.
• 22:16
  And then data engineering is usually the one that has to
• 22:19
  trace the lineage and say, OK, well where is this data actually
• 22:23
  coming from? I can try to manually figure out
• 22:26
  the services that that may have changed.
• 22:28
  I can look at the pull requests over time.
• 22:30
  Then I have to go and talk to the owners of those PR's and
• 22:33
  say, did you make a change? Did did it impact this?
• 22:36
  I have to manually investigate it myself.
• 22:38
  And then I have to convince them that they need to, to roll that
• 22:41
  change back or put in a fix into their PR.
• 22:44
  And that takes time, right? Because they don't really care.
• 22:47
  It doesn't affect them. So it goes on to their backlog.
• 22:49
  It may take weeks, it may take months.
• 22:51
  And then once the fix is actually put into place, now I
• 22:53
  need to do a backfill. And, and that's the data
• 22:56
  engineering team. And usually there are way fewer
• 22:59
  data engineers in a company than there are data scientists or
• 23:02
  analysts. So they become a bottleneck,
• 23:05
  right? They're getting all of these
• 23:06
  requests. They're kind of like stacking,
• 23:08
  and it's very hard to address them all.
• 23:10
  That's sort of one side of the problem.
• 23:12
  And then the other side of the problem is, well, if you don't
• 23:16
  trust the data to be accurate and if there is no clear
• 23:21
  ownership of the data, that means it is your responsibility
• 23:25
  as a data scientist to make sure with, you know, 100% confidence
• 23:31
  that this data means semantically what you expect it
• 23:34
  to mean. That there is validation rules
• 23:36
  in place, That, you know, if anything changes, you are going
• 23:40
  to at least know about it. And hopefully it doesn't totally
• 23:43
  blow your model up. And that takes a lot of time.
• 23:46
  And we're in work too. In fact, I think I've read like
• 23:48
  75% of data science time, it's been on validation.
• 23:51
  And that might seem like, oh, well, that's just kind of the
• 23:53
  state of the world. But this is not how software
• 23:56
  engineers work. They don't spend 75% of their
• 23:59
  time trying to figure out if the API that they're ingesting is
• 24:03
  giving them the right thing or not.
• 24:04
  Like they trust that API because it is an API, right?
• 24:09
  There is like it's very clearly documented exactly what this
• 24:14
  thing is that they're consuming, right?
• 24:17
  What what it means, how to use it.
• 24:19
  And and because that API exists, it means they can take a
• 24:22
  dependency on it and justice refer back to the documentation
• 24:26
  and the usage and the spec on how to manage it properly.
• 24:29
  And when you think, you kind of think about data in that way,
• 24:33
  data is an API too, right? When when I'm taking a
• 24:36
  dependency on data coming from some database, it's just, it's
• 24:41
  just another interface. And yet I'm doing that without
• 24:45
  the contract, right? There's that word that a
• 24:49
  software engineer gets when they are leveraging an API for their
• 24:53
  services. Like I'm taking the dependency,
• 24:56
  but I'm not getting any of the guarantees.
• 24:58
  I'm not getting any of the ownership, no, none of the
• 25:01
  trust, none of the documentation.
• 25:03
  And I think like this is one of the big things that that needs
• 25:05
  to change. All right, real fast, I want to
• 25:08
  tell you about the sponsor of today's episode, AWS Tranium and
• 25:11
  Influencia. Are you stuck in the performance
• 25:14
  cost trade off when training and deploying your generative AI
• 25:17
  applications? Well, good news is you're not
• 25:21
  alone. Look no further than Aws's
• 25:24
  Tranium and Influencia, the optimal infrastructure for
• 25:28
  generative AI. AWS, Tranium and Influencia
• 25:32
  provide high performance compute infrastructure for large scale
• 25:38
  training and inference with Elms and diffusion models.
• 25:42
  And here's the kicker, you can save up to 50% on training costs
• 25:48
  and up to 40% on inference cost. That is 50 with A5 and A0.
• 25:54
  Whoo. That's kind of a big number.
• 25:57
  Get started today by using your existing code in frameworks such
• 26:00
  as Pytorch and Tensorflow. That is AWS, Tranium and
• 26:05
  Influencia. Check it out.
• 26:07
  And now let's get back into the show.
• 26:09
  So how are you doing it? Are you basically in, I'm
• 26:13
  thinking about those tests that you can get when you go to the
• 26:16
  doctor and they check to see if your veins are clogged or not.
• 26:21
  And they put something inside your veins and then the blood
• 26:24
  pumps all through the body and you can see where there are
• 26:28
  places where you have like friend clots or parts of the
• 26:34
  veins that aren't working as well because it's been tracked,
• 26:38
  right? And I think it's like blue dye
• 26:40
  or something. And so under the special
• 26:42
  machine, you can see like, OK, these veins are not working that
• 26:46
  well and you have a clear picture of it.
• 26:49
  I almost feel like you need to do that where you put a stamp on
• 26:53
  the data. And if it comes out of this API,
• 26:57
  then it is with this like pedigree and you know these
• 27:01
  things. But then if anybody takes it and
• 27:03
  then they use it and you then have that data downstream from
• 27:09
  three or four different transformations or uses or
• 27:13
  people have touched it in many different ways, you can still
• 27:18
  retrace it because it has almost like all the different pedigrees
• 27:22
  as it's gone through this supply chain.
• 27:26
  So the the way, the way that we think about So what you're
• 27:29
  describing right now is sort of a, I think a more advanced form
• 27:32
  of of data lineage. And if folks in the audience are
• 27:35
  not familiar with data lineage, it's essentially the process of,
• 27:39
  of starting from the that either the, the looking at the
• 27:45
  generally it's sort of looking at an analytical environment and
• 27:49
  parsing through SQL and basically stepping through each
• 27:53
  query and understanding how that query was built.
• 27:58
  What table does it rely on and what table does it produce.
• 28:01
  And when you can answer sort of all of those questions for every
• 28:05
  query in your data environment, then you effectively have this
• 28:08
  big grass. And the graph shows you how one
• 28:13
  column flows to all these other different tables and views in in
• 28:18
  the company. And this is really great if
• 28:21
  you're using a tool like Snowflake or Databricks.
• 28:25
  And the reason why is because these companies have effectively
• 28:30
  built abstraction layers on top of data.
• 28:32
  They have created the interface, which as an example would be
• 28:35
  Snowflake SQL. And everybody knows how to
• 28:38
  interact with Snowflake SQL. And you also have tools like DBT
• 28:41
  that sort of write the transforms.
• 28:42
  And DBT also provides an interface that's even easier to
• 28:45
  use than Snowflake SQL. And they actually sort of manage
• 28:48
  a lot of this lineage work for you.
• 28:51
  The problem is generally not downstream, right?
• 28:54
  So, and by downstream, what I mean is what happens to the data
• 28:59
  after it arrives in some environment that a consumer of
• 29:03
  the data can do something with it.
• 29:06
  So it could be a notebook, it could be snowflake, it could be
• 29:09
  data bricks, right? It's somewhere.
• 29:11
  As long as I can access the data as like a data scientist, the
• 29:15
  problem is not really there. It's everything prior to that
• 29:20
  because where that data is coming from doesn't have nice
• 29:25
  clear abstractions. It's the Wild West, right?
• 29:29
  Like the data could be coming from.
• 29:32
  We could, we could have a Python script that's pulling, pulling
• 29:37
  out Excel spreadsheets and dumping these like CCVS into a
• 29:41
  data lake and then pushing that on some cadence into our
• 29:45
  analytical database. And anyone at any point time can
• 29:49
  go in and make a change to the structure of that file.
• 29:52
  And we would never know about it until it actually reaches us,
• 29:56
  which is too late. It could be a software engineer
• 29:59
  that is adding a new, you know, adding new logging to some
• 30:03
  feature that they've they've created.
• 30:05
  And then we're pushing that event via Kafka into a into a
• 30:09
  database. And then from the database,
• 30:10
  we're going into the analytical environment.
• 30:13
  And so if I don't know that, if I don't have the link between
• 30:18
  the analytical environment in the event code, I will always
• 30:22
  have to, that will always be a manual slog for me because I
• 30:26
  will have to make that connection myself.
• 30:28
  And this is, this is sort of the state of the world for a lot of
• 30:31
  data scientists. It's like there is that gap
• 30:34
  between when the data actually lands in some storage system and
• 30:39
  what's going on before that. And so when people talk about
• 30:43
  like a source of truth, that's what they're talking about,
• 30:46
  right? When someone says, Hey, I want
• 30:48
  to know the source of truth of these transactions, they're not
• 30:51
  talking about the table and snowflake.
• 30:53
  They're talking about the actual moment that the invent of a
• 30:57
  transaction occurs. And what does that mean, right?
• 31:02
  What is the underlying code that is producing that interaction
• 31:06
  and, and where can they find it? So in terms of in terms of what,
• 31:11
  what Gable does, there's a few things.
• 31:16
  Gabel is a, is, is kind of a, a meaty platform.
• 31:19
  We tackle this problem and in modular pieces, people can use
• 31:24
  any number of the modules that we produce or they can tie them
• 31:28
  together into a more seamless experience, which is where it
• 31:30
  really gets magical and powerful.
• 31:32
  But the core of the platform is what I would call our our our
• 31:36
  API's. Like we can build a consistent
• 31:42
  abstraction on top of many different types of data sources
• 31:46
  automatically. So if you think about what
• 31:51
  actually makes up any data source, there's really only two
• 31:54
  components. There's the structure of the
• 31:56
  data. So that would be the schema, the
• 31:59
  data types, column names, stuff like that.
• 32:03
  And the and the semantics like what?
• 32:04
  What does that structure actually mean?
• 32:06
  What does any particular column or row mean in the real world?
• 32:10
  And then there is the contents of the data itself.
• 32:13
  So those values that I would expect from that data, the if,
• 32:20
  if I have 5 categorical values in some particular field, I
• 32:25
  would never expect there to be a six or or 7th suddenly appear
• 32:28
  like maybe my system is not built in such a way to to deal
• 32:31
  with that. Or if someone, if I have an age
• 32:34
  field, I would probably not expect their age to ever be less
• 32:37
  than 0 or greater than, you know, like 300, unless they're,
• 32:42
  you know, you've. Got putting conditions and
• 32:45
  almost like guardrails onto the data so that people understand
• 32:49
  OK, no matter what and how tainted things can get, the data
• 32:54
  is not going to come out like this.
• 32:56
  And if it does, we need to figure something out.
• 32:58
  Are you alerting people if it does show up like that?
• 33:01
  Or is it just not allowed and you get like a null?
• 33:05
  Or it's it's a bit of both. So the, so the there's sort of
• 33:08
  a, a flow that happens the, so there's really like 4 main steps
• 33:12
  in Gable. The first step is the
• 33:13
  identification of these data sources.
• 33:15
  And we have some magical AI driven mechanisms to scan a code
• 33:21
  base and to automatically figure out where in code is data being
• 33:26
  produced. And so that could be a, you
• 33:31
  know, we can figure out here is all the database, all the
• 33:34
  database tables that are being produced, that are being all,
• 33:37
  all the structures that exist in your code.
• 33:39
  Here's all the event code that's being produced.
• 33:41
  We could connect to some file structure like S3, scan through
• 33:47
  S3 and then figure out for each file, is this file related to
• 33:52
  data? Does it have a schema and stuff
• 33:55
  like that? Or is this more of like a non
• 33:57
  data related file? Is it just like a Word document
• 34:00
  or something like that? Once we have all of those, and
• 34:03
  we could do this for 3rd party stuff too, Salesforce, SAP,
• 34:06
  whatever, it doesn't really matter.
• 34:07
  But once you kind of have all of those identified, then we can
• 34:12
  extract structure and we can extract contents of the data and
• 34:17
  we can represent those in a standard format.
• 34:20
  So it doesn't matter if it's a Postgres transactional database,
• 34:25
  if it is a, a segment event, a Python dictionary, you know,
• 34:30
  a.net executable, a Java whatever.
• 34:35
  All of that data can be extracted and and sort of
• 34:38
  organised and formatted and in in the exact same way.
• 34:41
  Once that's in place, we can then automatically enrich that
• 34:46
  data with lots of useful information O because we're
• 34:50
  injecting ourselves into the Git sort of version control platform
• 34:54
  of choice. So GitLab, GitHub, whatever, you
• 34:58
  can extract the ownership on the repo level, right?
• 35:03
  Like who is the person that actually owns the code that is
• 35:06
  producing this data? That is true ownership.
• 35:09
  You can extract how that data asset has changed overtime and
• 35:15
  then you can also extract how are these assets actually
• 35:18
  connected to each other. Like where do we see the same
• 35:22
  information in in different assets?
• 35:25
  And that's sort of the lineage graph that I mentioned before.
• 35:30
  Once you have that, and that all happens sort of automatically
• 35:33
  now. Once you have that, now you can
• 35:35
  start to have humans come into the loop and a human can say,
• 35:39
  OK, well, I have some expectations on this particular
• 35:43
  data. And those expectations might be
• 35:47
  the constraints you mentioned. So there might be some specific
• 35:51
  constraints that you have in mind that a machine would never
• 35:54
  really be able to figure out. Or maybe machine could figure it
• 35:57
  out by doing some profiling, but we're not, we're not there yet.
• 36:01
  Or it might be something like an SLA like hey I you know I always
• 36:04
  expect there to be 1000 events emitted over this time period
• 36:09
  and if that number ever drops it means something is probably
• 36:13
  wrong. What we can do then is based on
• 36:16
  the expectation, we can alert when we detect a change is going
• 36:22
  to happen or has happened. So if you're in CICD, for
• 36:26
  example, we will know is the structure of this code event
• 36:31
  going to change based on a PR? Hey, a software engineer has
• 36:35
  decided to get rid of a column that you, the data scientists,
• 36:40
  are using. We know that before that
• 36:42
  happens, same thing for thing new things being added.
• 36:46
  The other option is to look at the contents of the data and to
• 36:49
  say, hey, the contents of the data have changed in a way that
• 36:52
  are going that's going to affect your machine learning model.
• 36:55
  We're going to tell you about that before the pipeline is
• 36:58
  actually run and cause all that damage.
• 37:00
  Which I imagine just seems like magic for anybody that you
• 37:04
  talked to or that you're working with, because they're like, Oh
• 37:06
  my God, that is going to be a lifesaver for me.
• 37:09
  I've been spending 75% of my time trying to figure this out
• 37:12
  and now you're just telling me it's going to work.
• 37:14
  That's incredible. Well, there's a lot of things
• 37:17
  that you get for free with a system like this that I think
• 37:20
  are really cool. I mean, one thing is because
• 37:23
  we're registering the source data, then you get a nice sort
• 37:29
  of catalogue of all of your data sources, which is the things
• 37:32
  that most data people actually want, right?
• 37:35
  Like I said before, they don't want, they don't care as much
• 37:38
  about the tables. And then from there they can,
• 37:40
  they have to go back to the sources anyway.
• 37:42
  They just want the sources. And that's something that we we
• 37:45
  get, we get that for free because of this registration
• 37:48
  process. Same thing with the change
• 37:50
  management. Live change detection is a big
• 37:54
  part of our platform. We have a change log, so it
• 37:57
  shows over time, how has a data asset changed?
• 38:00
  Who actually made the change, What was the change?
• 38:03
  It's all the context that's very relevant for the teams that are
• 38:07
  consuming all this information. And there's a bunch of other
• 38:10
  things too around like data contracts and how do we agree on
• 38:15
  them and how do we collaborate as the data evolves.
• 38:18
  The thing that was most meaningful to us when we built
• 38:21
  this is how do we help the data producer O the engineer or the
• 38:27
  salesperson or whoever is generating this data understand
• 38:30
  that they are not in an isolated environment anymore?
• 38:34
  And as long as we can integrate with GitHub or GitLab, because
• 38:40
  we're automatically scanning and profiling this code and figuring
• 38:45
  out what it's connected to and who is dependent on it.
• 38:48
  We can provide that feedback without them actually having to
• 38:52
  do anything. Meaning if if I'm a software
• 38:55
  engineer, I'm just kind of going about my day-to-day work and I'm
• 38:58
  shipping features and I decide to change some SQL code that
• 39:03
  exists somewhere in my system, I will be told by Gable, hey,
• 39:07
  you're about to change records that other people in the company
• 39:11
  are using. Here are the people in the
• 39:13
  company who are using these records.
• 39:15
  Here is what they are using those records for.
• 39:18
  You need to go and speak to them before you make this change.
• 39:22
  Or at the very least, you should write up a summary of what this
• 39:26
  change is going to be. And Gable will manage
• 39:29
  communicating that to the right people, right?
• 39:31
  So it's not about like applying AI magic to turn the data,
• 39:39
  resolve all data quality issues or something like that.
• 39:42
  It's applying the AI magic to bring the people together and
• 39:45
  have closer connectivity when it comes to managing and creating
• 39:49
  data in the first place. Yeah, just that awareness is
• 39:52
  gotta be so brilliant for teams. I'm curious about you a year,
• 39:59
  year and a half ago left Convoy with ideas on what was going to
• 40:04
  be the most valuable. It seems like you had a pretty
• 40:06
  clear vision. But it's been polished over the
• 40:11
  last year and a half. What is one or two features that
• 40:17
  people have been asking for nonstop and you're you're almost
• 40:22
  like are surprised by these features because for you that
• 40:27
  was a non thing, but now you're talking to a lot of different
• 40:30
  companies. I think something I wasn't
• 40:33
  expecting is how much of an end to end system teams really want.
• 40:39
  My initial goal when I set out was can we build something that
• 40:44
  by pressing a button can scan all this code and then and
• 40:51
  figure out, you know, where, who, who, who owns that data?
• 40:56
  What does this actually mean? And put the constraints like the
• 41:00
  CID constraints in there and really, really start to treat
• 41:03
  data as, as, as another step in the DevOps process.
• 41:07
  That was my main goal. It's like bringing DevOps to to
• 41:10
  data and using data contracts as the vehicle to do that.
• 41:14
  What I found though is that people have a much broader
• 41:17
  vision of what data contracts are.
• 41:19
  So there's the CID approach that I mentioned, but people also
• 41:23
  want contracts for cataloguing purposes, right?
• 41:25
  Like I get all of the source data in one place and I can
• 41:28
  actually, you know, sort through that source data and sort of
• 41:32
  segmented by domains. People also want the live change
• 41:37
  detection on their data sources. So I wanna see how you know, I
• 41:43
  want to know when someone makes an update in Excel in real time
• 41:48
  so that I can either protect myself or protect my data
• 41:52
  consumers on a on a platform team.
• 41:54
  And I, I never want these issues to actually reach the analytical
• 41:58
  database in the 1st place right. So there, there was a lot more
• 42:01
  interest in that than we originally thought.
• 42:04
  And I would say I've also been surprised at ohh, Sorry, go
• 42:08
  ahead. I was just looking at how far
• 42:12
  upstream you are going. You're thinking like very far
• 42:18
  upstream. And if you can catch this in the
• 42:22
  moment this data is created, I think that's where you want to
• 42:26
  live. Yeah, our goal is to go, go all
• 42:29
  the way, all the way upstream. I I think that there.
• 42:33
  All the work maybe? All the way I I think that there
• 42:35
  are. Some places.
• 42:36
  Where you just you, you can't do that or there are problems that
• 42:40
  are that are introduced outside of that sort of core Data
• 42:44
  source. So, so for example, you know,
• 42:46
  you may have a, a human that is entering some data into a text
• 42:51
  field and they just enter in something that should never be
• 42:54
  entered. Like it doesn't make any sense
• 42:56
  and doesn't work with your, with your pipelines or your, your
• 43:02
  dashboards. And you, you can't really catch
• 43:04
  that during the CI CD workflow because there's no code being
• 43:07
  changed. But you would be able to catch
• 43:09
  that, you know, basically as as soon as that data becomes
• 43:14
  accessible through some system from moving it from that like
• 43:20
  front end interaction to to either the database or the
• 43:24
  analytical database. And so you want to try to catch
• 43:26
  that as soon as humanly possible.
• 43:29
  But really I, I think that the future is going to be all about
• 43:33
  code and also direct integration with any platforms or, or source
• 43:39
  systems that modify data like Excel or Google Sheets or
• 43:43
  Salesforce or SAP. Like a lot of data work happens
• 43:46
  in these products and there really is no oversight for data
• 43:52
  quality or data governance. You know, if I'm a salesperson,
• 43:55
  I'm going to change around my Salesforce schema in a way that
• 43:58
  makes sense for me and my team in the way that I want it.
• 44:01
  But if that sales data is really critical to downstream teams,
• 44:05
  like there's no real way of them knowing that or, or
• 44:07
  understanding that. And so all of that like change
• 44:11
  management and change detection CID, these are all places we
• 44:15
  want to insert ourselves. I want to just look at the other
• 44:18
  side of this, which is if you didn't have that, it's it's like
• 44:24
  you have to then go and manually and each one of these systems
• 44:29
  try and create something that says, hey, Alert me if this ever
• 44:33
  gets changed. And just thinking about that
• 44:38
  almost gives me a headache on all these different data sources
• 44:41
  on all these different people that you're interacting with.
• 44:44
  Maybe for a small team at a startup, you're running fast.
• 44:47
  It doesn't matter that much. And you can deal with that
• 44:50
  technical debt once you find product market fit or you start
• 44:53
  like taking off. But if you're at a gigantic
• 44:56
  company, just the amount of people that are working at that
• 45:00
  company that you would have to interface with to try and figure
• 45:03
  out like how in am I going to protect my assets, which are
• 45:11
  these things that I've been working on for the past whatever
• 45:14
  three to six months. And I thought I was having
• 45:16
  success with them. But little did I know somebody
• 45:20
  that I have no idea in some random department changed
• 45:25
  everything and now I don't know why but I gotta go debug it or I
• 45:29
  got asked by data engineer to debug it.
• 45:32
  Yeah, exactly. There's right at these big
• 45:35
  companies, there are, are thousands of people potentially
• 45:38
  who all touch different parts of the data that you're using.
• 45:42
  And, and you have no clue. A lot of these teams have, you
• 45:46
  know, thousands of, of microservices, 10s of thousands
• 45:50
  of of data sets of hundreds of thousands of CVS.
• 45:57
  You know, there's just enormous, enormous amounts of data and
• 46:00
  it's very, very unclear where it all comes from and, and what it
• 46:04
  means and why it exists and, and when it changes.
• 46:07
  And so any, it's not just any change that happens, it's, it's
• 46:12
  anytime you want to use that data, you have to go.
• 46:14
  There's actually, there's sort of this relatively narrow safe
• 46:18
  zone where you've already gone out and done all the work to
• 46:22
  understand what the data is and where it's coming from and what
• 46:24
  it means and all that. And that takes a lot of time.
• 46:26
  And then there's, you know, all the changes that could
• 46:29
  potentially happen to that data because it's plugged into so
• 46:32
  many sources. So you've kind of got this like
• 46:35
  relatively infrequent window where everything's OK,
• 46:40
  everything is safe and you've just kind of like perpetually
• 46:43
  got your fingers crossed that that, yeah, like today is not
• 46:46
  going to be the day. But but it always is the day at
• 46:50
  at some point, right? At some point the the day comes
• 46:53
  and and when it does come, it's it's very scary and it's a it's
• 46:57
  a big problem. That's why I've been championing
• 47:00
  for the Data Engineering Appreciation Day.
• 47:03
  Yes, they did. Engineers definitely need that.
• 47:06
  So before we jump, what was the other feature that surprised you
• 47:10
  that people were asking for a bunch?
• 47:12
  Yeah. Well, I, I think the other thing
• 47:14
  that was surprising to me was that how excited data producers
• 47:21
  and software engineers were about this.
• 47:24
  I think that data teams really, really underestimate is a bit of
• 47:28
  a like Stockholm syndrome going on.
• 47:30
  I think where they've probably tried in the past to go to their
• 47:34
  engineers and say, Hey, I need you to own this data or I want
• 47:36
  you to be more thoughtful about me.
• 47:38
  And they kind of got brushed to the side or the engineering team
• 47:40
  said, hey, this doesn't really matter.
• 47:43
  But when, when I I think that may just be the perception.
• 47:46
  When I've talked to data producers, they all seem to
• 47:50
  acknowledge the problem. They're like, yeah, we
• 47:52
  understand that this is an issue because we get asked about it
• 47:55
  all the time. The the issue is like, I, as a
• 47:59
  software engineer don't have a mechanism to help them with
• 48:03
  this, right? All of the solutions that I'm
• 48:06
  given is like, well, you need to be very thoughtful about what
• 48:09
  data lands in this table. But I don't know anything about
• 48:12
  Snowflake. I don't know anything about
• 48:14
  Databricks. I don't know anything about the
• 48:16
  data lake. I don't know about the ETL
• 48:18
  systems that you're using. I don't know anything about
• 48:20
  debt. So it's like, in order for me to
• 48:22
  help you with this problem, I have to learn like four or five
• 48:27
  different tools. I also don't have any way of
• 48:30
  like, if something does go wrong, I don't know how to
• 48:32
  resolve that. I, I don't, I don't know how to
• 48:34
  root cause it. I, I don't even know how to
• 48:37
  figure out if it was a big deal or not.
• 48:39
  I don't know what right or wrong even looks like.
• 48:41
  So there's like this whole list of requirements that need to be
• 48:46
  met in order for a producer can take ownership.
• 48:50
  And if those, if those questions are not answered, then this just
• 48:53
  seems like a very murky, unclear, difficult problem.
• 48:57
  And they, they don't know how big of an investment it's going
• 49:00
  to be. And so they don't do it.
• 49:02
  And so when they see that the way the way that we tackle this
• 49:05
  is, ohh, this is all DevOps. This is just another step in the
• 49:09
  workflow that I already run where we're accounting for data
• 49:13
  the same way that we account for security or the same way that we
• 49:16
  account for code quality. And if I do make a change that's
• 49:20
  not good, I'm going to be told, and I'm going to be given all
• 49:23
  the context on everyone I need to talk to and why I need to
• 49:26
  talk to them and what good actually looks like and what I
• 49:30
  need to do in order to progress this pull request forward.
• 49:34
  That is very exciting. Like it turns this from kind of
• 49:38
  a nebulous, unclear, very confusing problem to something
• 49:42
  that is very clearly scoped, very clearly defined, and fits
• 49:45
  into the workflows that they already use.
• 49:47
  So that is what I think is probably the biggest and most
• 49:52
  interesting thing to me is just how much software engineers have
• 49:55
  gravitated towards these solutions.
• 49:59
  Are you interfacing more with the DevOps teams or is it data
• 50:02
  engineers? Is it a little bit everybody?
• 50:04
  It feels like since the surface area is so big, you have to make
• 50:10
  sure that you are on top of it with each one of these people
• 50:14
  and that you understand their needs just as much as anybody.
• 50:18
  Else, yeah, I mean, I, I think the data engineers are in data
• 50:22
  platform teams are, are probably still the folks that we talked
• 50:26
  to the most because they, they they're impacted the most, they
• 50:30
  have the most. There's the.
• 50:33
  Most they're both for them. If there's data quality, right?
• 50:35
  Exactly for, for the data consumers.
• 50:39
  So the data scientists and the analysts, they're also impacted,
• 50:43
  but they're impacted in a different way.
• 50:45
  They're usually not the ones kind of doing all the root
• 50:47
  causing and, and the lineage, like all the stuff that takes a
• 50:50
  bunch of time. They just want to make sure that
• 50:52
  it doesn't happen and they don't really care how it happens.
• 50:55
  And then on, on the side of the data producers, they don't want
• 50:59
  to cause people pain. They don't want to be seen as
• 51:02
  the bad guy. And they certainly don't want to
• 51:05
  be in a situation where the, the, the whole company knows
• 51:08
  that they were the ones responsible for an outage to
• 51:11
  their machine learning model, right?
• 51:13
  Like that's what they want you want to avoid, but they, they
• 51:16
  just don't have the systems to do it.
• 51:18
  And, and so we, we sort of, we kind of started in the middle
• 51:21
  and then we work our way out. What my guess is, is that over
• 51:24
  time we will go the same route of the dev SEC OPS kind of
• 51:31
  workflow. I think dev data OPS will become
• 51:33
  a term that most DevOps teams and most engineering teams are
• 51:38
  using. And that means like selling,
• 51:41
  mainly selling into the software engineering organisation.
• 51:44
  And like in the same way that software engineers, they,
• 51:46
  they've now started to realise that security is a critical part
• 51:51
  of their workflow. They can't just pass that off to
• 51:53
  someone else in the organisation and kind of wipe their hand
• 51:55
  cleans of it. It's their responsibility too.
• 51:58
  If they don't do security best practises from the beginning,
• 52:02
  then hacking and fraud will inevitably happen and all you
• 52:05
  can do is deal with it. In the same way, if you don't do
• 52:08
  data management from the beginning, data quality issues
• 52:10
  will happen and you just have to deal with it.
• 52:14
  And that's not, that's not acceptable anymore in the age of
• 52:18
  of artificial intelligence. Maybe in the age of analytics it
• 52:20
  was it was acceptable, but but now it's becoming a lot less
• 52:23
  acceptable. It's so funny you mentioned that
• 52:27
  because I was just a cube con 2 weeks ago and every other booth
• 52:31
  was a security booth. And so it is very much
• 52:35
  understood that there are no excuses for any type of security
• 52:40
  breach. So I do see this vision where
• 52:43
  you're like, we cannot use these excuses anymore for poor data
• 52:48
  quality or just for poor data practises because so much of the
• 52:53
  business is relying upon this. You have so many different use
• 52:56
  cases that are coming from the data.
• 52:58
  And if you can quantify that, even better.
• 53:02
  And it feels like when you have things set up and you know where
• 53:08
  the data sources are and you know how the data is being used
• 53:11
  and that rich lineage, you can tell, OK, this data is giving
• 53:16
  us, it's generating a whole tonne of money through these
• 53:19
  machine learning models or through this, these other
• 53:23
  examples that you showed. I think that is another one that
• 53:26
  just like gives it shows the teams the value of the data in a
• 53:32
  way that is irrefutable. Yeah, exactly.
• 53:35
  I I think that we are gonna start to see a big shift over
• 53:39
  the next 5 to 10 years, not just in the tools, but in the
• 53:44
  companies that are using data. And it will split from companies
• 53:49
  that are using data for analytics primarily.
• 53:52
  And those companies will invest in low cost tooling, O very
• 53:57
  cheap storage and compute, very cheap analytic systems, probably
• 54:01
  open source. And then on the other side,
• 54:05
  we'll be companies that are actually using data to make
• 54:07
  money. AI ML is obviously one example,
• 54:11
  but there are other ways that you can monetize data or use
• 54:14
  data to save large costs. Invoice reconciliation is one
• 54:18
  example that I mentioned. And those companies are going to
• 54:22
  have to treat data like a product from the very moment of
• 54:26
  its inception. And that means not just putting
• 54:29
  quality into place downstream, but upstream as well.
• 54:34
  Beautiful man well I always appreciate coming on here I'm
• 54:38
  super excited to get to meet you in person for the conference we
• 54:41
  are going to be talking about this so much we're going to
• 54:44
  spend the whole day talking about quality.
• 54:47
  I had to have you there because when I think about quality, I
• 54:52
  you're like one of the first people that comes to my mind.
• 54:55
  And it's, it's almost like we couldn't have a conference about
• 54:59
  quality, AI quality, data quality without having you
• 55:03
  there. So yeah, this is going to be
• 55:06
  fun. It's going to be a whole lot of
• 55:07
  fun. And we're going to get to talk
• 55:08
  about this stuff all day long. Amazing.
• 55:11
  Well, I can't wait for it and I will.
• 55:14
  I will show up in my Sunday best and I'll I'll cut my beard and
• 55:18
  so I don't look like a wild man out of the case.
• 55:21
  That's it. Actually on a not so serious
• 55:24
  note, I was thinking like should we do some kind of a a theme
• 55:29
  like a Hawaiian shirt or something?
• 55:32
  So if anybody has any ideas, throw him at us because I am
• 55:37
  open to it. Right now.
• 55:38
  It's not too late to figure out what the theme or what the fun
• 55:42
  things that we're gonna do are. We're already going to have a
• 55:44
  comedian there. We're going to have some jam
• 55:47
  sessions happening. So beware of that too.
• 55:52
  I don't know the quality standards on those.
• 55:55
  We may need a tool like Gable to check on the upstream
• 56:00
  possibilities of the quality and given the stamp of approval for
• 56:05
  the musical session qualities, but you'll see when you come.
• 56:12
  I guess that's that's the fun of it.
• 56:15
  Hold up, wait a minute. We gotta talk real fast because
• 56:18
  I am so excited about the MLB's community conference that is
• 56:22
  happening on June 25th in San Francisco.
• 56:25
  It is our first in person conference ever.
• 56:27
  Honestly, I'm shaking in my boots because it's something
• 56:29
  that I've wanted to do for ages. We've been doing the online
• 56:33
  version of this. Hopefully I've gained enough of
• 56:36
  your trust for you to be able to say that.
• 56:38
  I know when this guy has a conference, it's going to be
• 56:42
  quality. Funny enough, we are doing it.
• 56:45
  The whole theme is about AI quality.
• 56:48
  I teamed up with my buddy MO at Colina who knows a thing or two
• 56:53
  about AI quality, and we are going to have some of the most
• 56:58
  impressive speakers that you could think of.
• 57:01
  I'm not going to list them all here because it would probably
• 57:03
  take the next two to 5 minutes, but just know we've got the CTO
• 57:08
  of Cruise coming to give a little keynote.
• 57:11
  We've got the CEO of u.com coming, we've got Chip, we've
• 57:17
  got Linens, we've got the whole crew that you would expect.
• 57:20
  And I am going to be doing all kinds of extracurricular
• 57:24
  activities that will be fun and maybe a little bit cringe.
• 57:30
  You, you may see, hear or see me playing the guitar.
• 57:33
  Just come. It's going to be an awesome
• 57:35
  time. We'd love to have you there.
• 57:37
  And that is again, June 25th in San Francisco.
• 57:41
  See you all there.