Latest / Agentic Conversations (formally mlops.community) / MLOps #22 Feature Stores: An Essential Part of the ML Stack to Build Great Data // Kevin Stumpf - Co-Founder & CTO at Tecton

Transcript

MLOps #22 Feature Stores: An Essential Part of the ML Stack to Build Great Data // Kevin Stumpf - Co-Founder & CTO at Tecton · Agentic Conversations (formally mlops.community)

0:00
It's good to have everybody here.
0:16
I'm going to jump right into it. Get started with our guests to
0:23
is none other than the CTO and co-founder of tecton AI which As
0:30
quite a bit of hype these days because the majority of the team
0:36
comes from, some hard hitters. They've got people like our host
0:41
himself who was an ex kuber X Michelangelo.
0:46
We've also got some x Facebook X, Cora guys, X Twitter, you
0:53
name it. As far as where you spend your
0:56
time on the internet, they probably have someone that used
0:59
to work there. There.
1:01
So, that is what's going on with tecton.
1:04
And for our guests, he is not only a machine when it comes to
1:11
coding and creating ml Ops tools, but he also does some old
1:17
truck Endurance Sports, so he told me he did a little bit of
1:23
Iron Man. He's done some Ultra marathons
1:26
but his favorite hit the most fun.
1:28
He's had was when he ran. 250 kilometers across the Sahara
1:33
Desert, which is mind-blowing. I think for all you that are
1:39
from the US, and don't know, km, that's around like, 150 miles.
1:44
So you get an idea. I can imagine, you're drinking a
1:47
lot of water or did you go with Toph style?
1:51
So, yeah, please give it up big round of applause for our
1:56
guests. We have Kevin stump the CTO of
1:59
tecton. Thank you.
2:00
For coming on here. Kevin, I really appreciate you
2:02
joining us. Thanks for having me.
2:06
Yeah. So before we jump into anything
2:09
about tecton and feature stores, I know that's kind of what we're
2:14
going to be centered around today and we're talking about a
2:17
lot of feature stores and what you all are doing at tecton
2:21
around future stores. I wanted to just start with a
2:23
question that I normally start with, which is how did you end
2:27
up in the place that you're at? Now, I know you're originally
2:30
from Germany and now you're living in the Bay Area.
2:34
You were working at Uber, can you give us a bit of a story
2:37
around that and how it all happened?
2:40
Yeah, definitely. I'll try to keep it brief.
2:43
But I've always been a software engineer.
2:45
And as I mentioned, I'm from Europe from Germany specifically
2:48
and after high school started a software development company
2:51
that builds software for various companies all over Europe and
2:55
the us including things like test but automation software for
3:00
GM in Michigan and through that. Engagement of project, I got to
3:06
really like the US and decided to come over, come over to the
3:08
Bay Area and stop focusing on just service oriented software
3:13
businesses and really focus on product oriented businesses and
3:17
through that thought would be a good idea to layer on some some
3:22
business education to and then went to Stanford, Stanford
3:25
Business School and through that ended up co-founding dispatcher,
3:31
which was an end-to-end Market. Is the connect, the long-haul
3:35
truck drivers to shippers and was really the Uber for
3:38
long-haul truck driving. That's then also how we ended up
3:42
a boober because who burrowes building out its own internal
3:45
over afraid Division and then as soon as I joined Uber, I
3:49
switched over to Michelangelo, the ml platform team and then
3:54
work there as an engineering manager and Tech lead off the
3:56
platform and saw all the potential and all the the impact
4:02
that Michelangelo and then, Centralized and that
4:05
standardized ml platform have a tuber, which I can really think
4:10
about it as leaving to this Cambrian explosion off machine,
4:13
learning applications within Uber, just by providing this
4:17
Center Line standardized platform that all the engineers
4:21
and all the data scientists were able to use to get a male real
4:24
into production, so that had massive impact that the company
4:29
at Alber. And that's when we decided, hey,
4:32
let's, let's Company around and this face.
4:36
So they that we can provide similar tooling similar
4:38
platforms to other Enterprises that are not necessarily only
4:42
the Bay Area. Yeah, so we're you bought by
4:46
Uber and what year was that that was in 2016 as when we joined
4:52
them and it was a yeah our entire company joined them and
4:58
our entire operations and sales and whatnot to him or the
5:01
majority of them then joins Acts.
5:04
Fledgling, Uber Freights division at Edinburgh at the
5:07
exact same time, they acquired Auto which was the self-driving
5:10
trucking company and then through that they then just
5:16
build out that division within Uber.
5:19
Perfect. So, let's jump into feature
5:22
stories because I got a lot of questions to ask and event.
5:27
I want to just start with what a feature is.
5:31
And I, you know, I was reading up a bit on.
5:35
Can you just give us a breakdown of what a feature is?
5:38
What feature engineering is that kind of info?
5:43
Yeah, definitely. Yeah.
5:44
For those of us that are are like me and a little bit slower.
5:49
No, but sure, it's definitely a complicated and interesting and
5:53
new topic and features is really data for machine learning and
5:57
it's typically data or information.
6:01
That's derived from raw data and an organization.
6:04
You can think about it as being highly curated, really high
6:08
quality information, that ml models are able to detect
6:12
patterns. And those features then helped
6:15
the model to predict something like it helps a model to predict
6:20
which restaurant are you most likely to pick from if you open
6:24
up and overeats app or how long is it?
6:26
Most likely going to take you to get an Uber ride from Palo Alto
6:31
up to San Francisco and typically the way you get from
6:36
just raw data to these features. So, this highly curated
6:39
high-quality information is by either, just cleaning the raw
6:42
data. By filtering out all of liars or
6:46
by aggregating information, where maybe you're saying, hey,
6:49
how many times has a certain customer ordered Thai Dish in
6:55
the last 30 days? So it's an aggregate feature.
6:58
Then that would be very helpful for animal model to detect
7:02
patterns. And and then oftentimes, it's
7:04
also human intuition. That is applied to raw
7:08
information where you can imagine that maybe.
7:13
Data scientist has the intuition that the weather outside, we
7:16
have something to do with what people want to eat right now in
7:19
it. So you could have a Boolean
7:20
variable that says, hey, is it like pretty warm right now
7:24
outside? Or is it pretty cold outside
7:26
right now? Which may determine whether
7:28
somebody is more likely to order like a hot soup or a cold ice
7:30
cream and stuff like that. Oh, that's that's such a good
7:36
way of putting it and it makes total sense.
7:38
So now when it comes to features and this idea of taking the
7:45
whatever certain parts of I look at it, like you have your Excel
7:49
sheet, right? And then you take a certain
7:52
piece of the Excel sheet. Maybe you combined it with
7:54
another one and or you say this is the important one.
7:58
Let's look at it that way. I know I'm over simplifying this
8:01
like crazy but But that's how I'm looking at it.
8:06
Now, what is it that Needs to be done when you're trying to
8:12
create like a store where you can.
8:15
Just look at these and say, all right, I want to be able to pull
8:19
from that quite easily. Mmm.
8:22
Yeah, definitely. So there are at least three
8:26
things that you need to do on the one side.
8:28
It's like curation of those features.
8:29
So somehow these features need to be calculated and they need
8:33
to be ingested into this feature store so they need to come from
8:36
somewhere. And again it's the big lie.
8:38
Like you start with the Raw. Which may live in a data link
8:41
that may live in a data warehouse and even Beyond
8:44
streams like Kafka streams or maybe real-time data.
8:48
And that needs to be transformed typically cleaned up aggregated
8:52
and then those feature values need to make it into the feature
8:55
story itself, like the database, if you will where it can then be
8:59
retrieved from later on and then you need to be able to serve
9:04
those feature values. Serve them, typically two or
9:07
three different types of consumers.
9:09
So one is the data scientist who just wants to interactively,
9:14
explore features and maybe solve a new modeling problem and pull
9:19
out historical feature values larger scale.
9:22
Typically in order to be able to train models play around and see
9:25
what works, what doesn't work. Second consumer with typically
9:28
the production training pipe lines that run are going to Cron
9:31
job, naively weekly, however frequently to train uml models,
9:36
that also need to then consume from this feature store.
9:39
Finch the latest set of updated, future values to train on your
9:42
model and then third, and most importantly ends.
9:46
And also the hardest type of consumer is the actual
9:50
production model. That's often times, has the
9:54
requirement of serving predictions at very very low
9:58
latency very, very high scale, like an Uber Eats,
10:02
recommendation model, or Uber ETA prediction model these types
10:06
of models. They do no more than 10 to 20
10:09
Seconds to make a prediction. They need to be able to fetch
10:12
these feature values from the feature store and then really
10:16
low, latency with with production as a lace.
10:19
And then layered on top of that, the last piece of course, is
10:22
also the management of those features somehow need to be able
10:25
to be able to discover and search and find new features.
10:31
Identify what business entities business objects over these and
10:35
these business videos features typically associated with.
10:38
Huzzah, Hook for them. Can I trust a certain feature?
10:42
How much does it cost me? Am I even allowed to use it?
10:45
What purpose am I allowed to use it for?
10:47
And that type of stuff. Hmm, yeah.
10:50
And that gets into something that I want to touch on later on
10:54
about permission and how that would work out.
10:58
But before we go into that, there was something I wanted to
11:01
ask about when do you feel it's right for a company to start
11:07
looking at feature stores? And what is it for right from
11:11
the beginning of their ml journey?
11:13
Is it something that you have to be a bit more mature in order to
11:16
actually make good use of it? Or is it a little bit of both?
11:24
yeah, I mean if you're if you're just one data scientist or two
11:30
playing in jupyter notebooks and you're just running very simple
11:36
offline predictions, literally yourself in the notebook and
11:39
there is not a whole lot of features that you iterate on or
11:43
the doing are sharing with the other data scientists and then
11:46
something like a feature store, maybe an Overkill, but typically
11:50
signals to look out for that are good sign that you do need a
11:54
Your store is, are you wasting a lot of time getting ml model
12:00
into production? And are you spending a lot of
12:02
time there, creating writing you data pipelines that make that
12:07
feature data available in production for serving for the
12:10
model? Do you struggle with training
12:13
serving inconsistencies where features all calculated
12:16
differently for training purposes versus for serving
12:19
purposes? If that's the case.
12:21
That's a problem that a, that a feature story helps you with Do
12:24
you find that data? Scientists are often times.
12:26
Reinventing the wheel implementing the same feature
12:29
over and over again. Maybe even with slightly
12:32
different understandings of. What, a, how a feature should be
12:36
implemented. If you have features that
12:42
consumed raw data from various different types of data sources.
12:46
Say, you want to combine raw data from a data warehouse and
12:50
the data leg and the streaming Source doesn't really hard to
12:54
get We need really sophisticated data pipelines to get to do this
12:58
type of stuff properly. If you face, these types of
13:00
challenges than typical future store, is also a pretty good
13:03
idea for you. Yeah, let's go into that a bit
13:05
more. Like why is it really hard to
13:08
build features? Yeah, so be you can think about
13:14
it this way. I look going again.
13:18
Looking at again, at the raw data sources, the day Warehouse,
13:23
the stream, the data Lake, It's the real time data sources.
13:25
They all have very different characteristics to them, for
13:30
one, you typically cannot run the exact same type of feature
13:36
transformation against those different types of data sources.
13:39
So data warehouse is really, really good at running large
13:43
scale, aggregations and Joints. It takes a long time to execute
13:48
them, but they're really, really good at doing that type of stuff
13:50
whenever you slice and dice and filter out, only a small set up.
13:54
Columns, you will not aggregate over them again, join them.
13:58
That's when, uh, when a data warehouse or a data lake is very
14:01
good. If you look at streams, for
14:04
instance, you cannot do a large-scale aggregation or
14:07
large-scale joint. Very efficiently with a stream
14:10
with a stream and really limited more to running role-based or
14:15
even phase Transformations where you're combining a couple fields
14:18
from from events on a stream or you run temporal aggregations
14:22
over limited time window. Toes, because of the time window
14:25
becomes too large. Then of course you're speaking
14:28
processor can run out of memory and run into all kinds of
14:32
issues. And then if you're looking at
14:35
more of the olltii p-type databases like the mongos of the
14:38
world, the my sequels or postgres has, they are typically
14:43
pretty good at running. Road-based Transformations and
14:47
small-scale aggregations, but they don't have the entire
14:50
historical data, right? They only have typically like
14:53
the Using data Maybe with the last six months, maybe the last
14:56
year, but they don't have all the historical context in which
14:59
you cannot really important. High-quality information that
15:03
may allow the ml model to predict much more accurately.
15:06
What a user is about to do next. And so that's a that's a pretty
15:11
hard challenge to build a system that is able to run
15:14
Transformations against these different types of data sources
15:17
that all have various very, very, very different
15:20
characteristics. Then of course your Features
15:24
also have different freshness requirements.
15:26
So if you think about like an ubereats model, which is trying
15:32
to predict, how long is it likely going to take for a
15:35
restaurant to prepare an order for you?
15:37
If you were to place an order right now they're imagine, there
15:41
are two features that are typically very important here.
15:44
One is how many orders have been placed in the last 30 minutes
15:49
because that's typically a proxy for how busy is the restaurant
15:51
right now and another Future will be here.
15:56
How long does it take on average over the last two weeks, for
15:59
this restaurant to finish an order?
16:02
Both of those have very different freshness requirements
16:05
or you can imagine that the 30-minute average or over the
16:11
30-minute, trailing count of orders, has to be updated like
16:14
every minute, every five minutes, if you updated every
16:16
hour or so, you're, you're missing on the lot of, you know,
16:21
very relevant information because you just don't know
16:23
what's actually currently A happening in the restaurant
16:26
two-week feature, though. You can just update nicely.
16:28
It doesn't really matter that much.
16:29
It doesn't, there's not that much minute-to-minute variance
16:32
in this feature. That the model could pick up
16:33
from then, of course, there's also again, this problem with
16:39
online offline consistency where you find a lot of people just
16:43
implementing feature Transformations differently on
16:46
the online side as well as on the offline side.
16:48
And if you have different implementations for these
16:51
features Transformations, then all kinds of things.
16:54
Go wrong where oftentimes some edge cases are treated
16:57
differently or introduced a bug in one but not in the other and
17:00
then your model picks up on something in the offline world
17:04
that it will never seen the online world.
17:06
And then of course, your model looks great when you do an
17:08
offline back test but it's going to completely fail.
17:11
When you're then later on trying to run it in production, you
17:14
have no idea what's going on and then you need to debug the
17:16
feature transformation code and then you see, oh shit.
17:19
Here I'm handling. A null completely different or
17:21
I'm dealing with a different floating-point, Precision, or Or
17:24
something like that. It's just really tricky to dig
17:27
into and and figure out. And then another thing that
17:33
makes these features of for operational Amell, really hard.
17:36
As that typically these types of ml models, when you generate
17:41
training data, you want to be able to time travel to any point
17:45
in the past, to look at, what does the world look like at any
17:48
given point of time in the past? And then fetch a state of Like
17:55
hundreds of thousands of features and then you generate a
17:57
Time series, over say, the new update hourly or daily and then
18:03
over you could go to like a feature store and say, hey give
18:06
me the value of these 1000 features for every single day in
18:10
the last 365 days, which then produces 365 rows and it's
18:16
really hard actually to get this time travel done, right?
18:20
So anyways, those are just a couple of features so I have one
18:25
request. Can we stop talking about boober
18:27
eats? Because I haven't had dinner yet
18:29
and you're making me hungry. It's so but let's get into Uber
18:36
a bit more because I know you did quite a bit of work there
18:39
and you were messing around on Michelangelo.
18:41
So what made you gravitate towards that project?
18:45
And what exactly were you doing on Michelangelo?
18:50
But I gravitated me towards Michelangelo when I was at over
18:54
rooming. Yeah.
18:55
Yeah exactly. Yeah, it was really, it was
18:58
seeing how on the one side, how much potential there was a tuber
19:03
to apply machine learning for all kinds of different use cases
19:06
because Uber was, the fastest-growing private company
19:10
of all times. It was collecting so much data
19:14
on on rides on what users are doing and what suppose Something
19:19
in the Uber Eats app and it was just, it was just a lot of
19:25
manual work that unnecessarily had to happen, that led to sub
19:30
optimal decisions around. How can you tell us like what
19:34
some of that manual work was? Yeah.
19:36
Like imagine that they're a customer support ticket.
19:42
Request is coming in. And now the first step that it
19:46
had that this ticket goes through, is that Some person has
19:50
to read the ticket, understand what the issue is and then
19:54
figure out. Okay.
19:54
What's the urgency of this request?
19:56
What's the priority of it? Who's what's the department with
19:59
an Uber? That's best suited to respond to
20:03
this customer and then manually routed over that obviously takes
20:06
forever. It's a really poor experience
20:09
for the customer because they're waiting forever.
20:12
The person may actually who's making the wrong decision.
20:14
May make a wrong decision, it's hard to track whether it was a
20:17
good or bad decision. So All just not efficient at all
20:20
and extremely expensive, both to the company and to the customer.
20:24
And so, here is an obvious opportunity to apply in a male
20:27
model that takes care of this routing decision, that can
20:31
actually look at the text, try to understand the context, try
20:34
to predict a, what's the priority of this thing?
20:36
What's actually the issue and then routed to the right
20:39
department and treat it with the right level of urgency.
20:44
And so, so when you were seeing these manual steps, having to be
20:49
done, then Michelangelo came and said, all right, we're going to
20:53
fix that. Or what?
20:55
What did you see and how did that work out?
20:58
Like, what was the fix for that? Yeah, I think this, I was
21:00
actually like many, many different teams and many
21:02
Defenders, smart people and Uber, who did see the potential
21:06
for machine learning and models, a tuber across the company.
21:10
And so, you did have a lot of people who then, Did train
21:15
models and then spent lots of time trying to get those models
21:19
into production. And then there were struggling
21:22
with getting production grade as a lays up and running, where the
21:26
model doesn't just crash and the latency very good and all that
21:29
stuff, and it was just a hodgepodge of different
21:31
Frameworks buried were being used, and it would take a very
21:35
long time for data, scientists work with software Engineers,
21:38
but they do engineers with a male Engineers to over weeks and
21:40
months get them uml model into production.
21:44
And then, of course, various different use cases.
21:48
I do burr, could benefit from the exact same types of
21:51
features. There's a lot of feature sharing
21:53
that could happen between different models and then you
21:55
would see that those features are again implemented over and
21:59
over again, it was very hard for other data, scientists to
22:02
identify, need to know what features had other data
22:05
scientists use before them and then I would be almost
22:08
impossible for them to reuse them.
22:10
And so it just became very clear that there is a lot of D 4 ml,
22:15
people are applying them already.
22:17
It's super expensive to do, it's extremely, are prone to do it.
22:20
So let's build a standardized centralized platform, that
22:24
allows data scientists and Engineers to get to the platform
22:28
and trained animal model and get into production completely end
22:32
to end from selecting the raw data to selecting the features
22:37
doing some additional feature engineering training, the model
22:40
back, testing the model and evaluating it.
22:42
Managing the model are The fact versioning, it deploying it and
22:46
serving it in production and then monitoring the predictions.
22:49
All end to end with one platform.
22:52
Yeah. And so talking about this idea
22:55
of the feature stores or just this whole thing, starting to
23:01
realize, hey, this is going to, this could be very costly.
23:05
I want to jump into, do feature stores enable this, the
23:12
companies that are using them, too.
23:14
Save money on in terms of like their machine learning teams and
23:18
if so, how how does that work? Yeah, definitely.
23:22
There are tons of different opportunities to save money.
23:26
One, is that if you're able to just reuse an existing feature,
23:31
you don't have to implement it again.
23:34
So you're saving time having to re-implement the feature.
23:37
You don't have to wait until the feature is backfilled, if you
23:40
were to implement it from scratch so you can just get
23:43
started. Use it.
23:44
Right away, it's out there in production.
23:48
You don't have to go through the pain of working with a data
23:52
engineer on ML engineer who then re implements your entire
23:55
feature engineering pipeline, that you have that you've
23:58
created in like your jupyter notebook, who has to, then
24:00
understand your code and make it production-ready and
24:03
re-implement it. And then work closely with you
24:05
to make sure that all of you they are shot the assumptions
24:08
that you've made he or she then is able to also extract and
24:12
properly implemented. The production code It's all
24:15
this back and forth is saved because the data scientist can
24:18
just click a button or run a command line and get the feature
24:22
into production without having to go through these extreme with
24:25
friction folk processes then on top of that, Central teacher
24:29
sores also able to give you visibility that you've never had
24:32
before where you know actually no hey what's actually the cost
24:36
the literal cost of these features.
24:38
Like how much am I spending every day on running those
24:42
compute jobs? To turn my raw data into into
24:46
feature values is any model you can consuming these features.
24:50
Or I might just producing these feature values every day.
24:53
Every minute and nobody's using them.
24:54
It's a complete waste of time and money.
24:57
That's all, or if somebody's consuming them, if some arms are
25:00
consuming them, which ones are those Which business use cases?
25:03
Are they attached to how much money is the company making?
25:07
Or how much good? We're all good.
25:08
Will, are they creating with those models?
25:10
And is it worth it? Looking at the cost?
25:13
So those are just some of the immediate and obvious
25:17
cost-saving opportunities that you get by having a features or
25:21
that's perfect. Because one of my next questions
25:23
was, you know, how is their relationship between the future
25:27
store and Technical debt and that kind of makes sense.
25:31
Like if you're just creating a bunch of features, but no model
25:35
is actually using them. What is that for?
25:39
And the other thing I was going to ask is about This can enable,
25:44
you know, accountability and traceability.
25:48
So as perfect now before we jump into what you're doing a tecton,
25:52
I wanted to ask one quick question about Uber because it's
25:57
always been on my mind since I heard about it how I've heard
26:00
Uber's also utilizing Cube flow. So how does Michelangelo and
26:06
Cube flow work together? Or is that something that was
26:09
just like a something that they used to use or or can Do not
26:13
talk about it. That's fine also.
26:16
So cute Flow by itself is a collection of various different
26:21
tools, right? And they make it very easy for
26:25
you to spin up jupyter notebooks they make it easy for you to
26:30
train tensorflow models in a distributed fashion or even on a
26:33
single node. They make it easy to deploy your
26:35
models using Selden and what not, they have q flow pipe lines
26:39
around some data data pipelines and easier way.
26:43
But at the end of the day, it's like an umbrella for many
26:46
different tools that are all related to machine learning and
26:50
that cubicle makes easy for you to run on kubernetes.
26:54
Uber, and with Michelangelo, we were actually not running on
26:57
kubernetes. We did use a lot of these tools
27:01
that are under the umbrella of cube flow where we were using
27:05
tensorflow for training or using jupyter notebooks for
27:09
interactive development. But it wasn't that we were using
27:12
the Q flow project. As is we're much more using the
27:15
open-source components under it and fitting them into the Uber
27:18
stack. Okay?
27:20
Perfect. Yeah, because that was something
27:23
that had been roaming around in my head.
27:26
Since we had Josh button on while ago, talking about Cube
27:31
flow 1.0 and he said, oh yeah, Uber uses this.
27:34
And I was wondering like, how is that working with Michelangelo
27:37
and whatever regards. But if it's just like okay just
27:40
the modules of it that completely Makes sense.
27:44
And so now let's talk about tech time and what you're doing over
27:48
there at tecton Ai, and what exactly are you building and how
27:52
is it different from Michelangelo?
27:55
Yeah, definitely. So tecton is data platform for
28:00
machine learning and you can really think about it as first
28:03
like enterprise-ready feature store that takes the lessons
28:09
that we've learned at Michelangelo.
28:11
I do burr and whereas all the other come Please that named
28:13
earlier to bring my really a best-in-class interest or to
28:19
other companies who really want to use machine learning will be
28:22
used efficiently by using a production before I go into more
28:25
of the specifics off collecting on the capabilities of it.
28:29
One big difference to Michelangelo that's very
28:32
important. Is that Michelangelo was an
28:35
end-to-end ml platform that helped you both with the
28:41
modeling layer. And with a date, Later.
28:44
And you like, you machine learning consists of both, you
28:48
need the model and you need the data.
28:50
If you don't have one or the other, then you're not going to
28:52
be able to get them all into production and so Michelangelo
28:56
handle both of them. It trains the model for you off
29:00
of the data that it provided it managed model artifact deployed,
29:04
the model artifact, it serves the model artifact and then as
29:08
part of the serving it was also serving the features the data
29:11
underneath that you can really look at it.
29:13
The this like to layer stack data and models that have to be
29:17
integrated with each other and they have to go hand in hand.
29:20
In order to be successful with machine learning, was it sorry
29:23
to interrupt, or was it also managing the data?
29:28
Yes, yes, it was. Moving it around from databases
29:31
and all of that good stuff too. So it was connected to the raw
29:36
data sources. And so, Michelangelo itself
29:39
wasn't like providing a data warehouse or an old TP database
29:42
that was connected to those raw data sources in the company, but
29:47
then the features and the featured Transformations that do
29:50
turn the raw data into features that will stored and managed by
29:55
Michelangelo, or less than using.
29:57
Link for streaming applications for streaming Transformations
30:00
and Spark for the batch Transformations.
30:02
But it was always deriving features from the raw data and
30:07
then storing and persisting them in data stores that Michelangelo
30:11
itself, managed and then used to serve those feature values.
30:15
Perfect. Okay, so now jump into tecton,
30:20
Tech tongs focuses. Only on the data layer and
30:22
provides very, very like intuitive and first-class, any
30:27
great. Ian's with tools in the modeling
30:30
layer so we have no intention of providing right now.
30:35
Like the modeling layer on top of our data layer.
30:39
We're really focused on providing the Best in Class
30:42
feature platform that allows you to curate manage and serve those
30:48
features lives again. The what does that mean?
30:51
Again curation is you tell tecton, these are my raw data
30:57
source. This is my S3 bucket or this is
30:59
my cough custody and was my Kinesis dream.
31:02
This is my snowflake or Rich of database and so you configure
31:07
tecton to be able to access those typically using I am
31:10
permissions of your running on AWS.
31:13
Then you also tell tecton what are the features that you want
31:17
it to to run? And how does it transform the
31:22
raw data provided by these raw data sources into features that
31:26
your mental models? Downstream will be able to train
31:29
on and learn from. And the way you do that is by
31:34
like we have this and we can go into more detail there later we
31:37
are big Believers in devops course, we're talking about ml
31:40
Ops here, devops Hormel, those Hormel data and features and so
31:44
everything that you do with tecton, all the configuration
31:47
everything happens in a file system in Python files.
31:51
Were you very declaratively? Lay out?
31:54
What are my features? What are the Transformations?
31:56
Look like and And you hand those over to tecton using a CLI.
32:01
So the tech tone is then able to automatically run.
32:05
These feature pipelines, the turn, your raw data into
32:08
features the tecton then stores and its future store.
32:13
So that later on it can then serve those features to data
32:17
scientists who are working into interactive environments and
32:19
their jupyter notebooks to production training pipelines
32:23
and to production models, who are either making Hi scale, low
32:28
latency predictions and like canceled milliseconds or large
32:32
scale batch predictions. Perfect.
32:36
So I know there has been quite a bit of hype around what you all
32:40
are doing. And first of all, great job on
32:43
that one I think because it was it's something that people
32:47
really enjoy like they need. And it's something that is right
32:52
now really in the Forefront of what people are looking for.
32:57
I think it just came out at the right time and just recently
33:01
when was it a week or two ago as spark, you release more
33:06
information about what you're doing, can you talk to us about
33:09
what what exactly that is and right now like because so
33:13
everyone knows if you just wanted to go out and buy tecton.
33:17
You couldn't write. That's not possible.
33:19
Yet. You'll have to talk to us and
33:24
exactly. But we do have something.
33:27
Special for everybody. That is here today and in the
33:31
community I think we got to, we correct me if I'm wrong on this
33:34
one, I'm not going to just give out a lot of stuff from Tech
33:37
Tandoori, but but I could I've done it before.
33:42
And so we said that if anyone wants to sign up for tecton and
33:47
you just put in the comments of when you're signing up for the
33:50
waiting list, because I think you have a waiting list right
33:53
now, right? Yes, exactly.
33:54
So if you put in the comments of the waiting list, You were part
33:58
of the ml Ops Community or you saw this video or you listen to
34:02
the podcast later on whatever you will get entered into a
34:06
raffle for some gift cards and I don't know what the gift cards
34:11
are, but I got a feeling you're going to make it who buries
34:13
right now. I feel like you've been talking
34:17
to us so much about every time, an amazing gift cards for a
34:20
tech. Some credits.
34:21
There we go. There we go.
34:23
That would probably be more useful and some boober eats, So
34:28
so great. So just yeah, talk to us a bit
34:30
about that. Like, if I wanted to go get
34:32
checked on right now, what's the process look like?
34:35
Yeah, I'm right now, you basically reach out to us.
34:38
It's very easy in our website. You just punch in your email
34:41
address. If I would like to be optional
34:43
forms on your use case, are you running on the cloud or re
34:46
running on Prem? Because right now we're like
34:48
allowed native feature platform very with a first-class
34:54
integration with AWS. So ideally running on AWS.
34:57
The case, then that's great. And then we'll reach out to you
35:01
and discuss how to deploy technology into your
35:05
environment. The quick summary on this is
35:09
tecton, basically supports a sense, deployment version and
35:12
the BBC deployment version. There's some customers who say,
35:15
my data is highly sensitive. I really just want you to deploy
35:20
the full text on stack into my environment.
35:23
Typically, the way that would work is they have an AWS
35:25
account, they create a cross. The account access role for us
35:29
with limited permissions. That's enough for us to deploy
35:32
tecton into their environment. And I get it up and running that
35:35
typically takes less than 24 hours or the alternative is
35:39
where they say, no, just give me the full set a solution.
35:41
And I'll get the oars SAS hosted version of tech non-access to my
35:45
estimate buckets, or to my red shift, my snowflake, my Kinesis
35:48
wherever it is, and then everything just runs in our AWS
35:51
account. Cool?
35:53
So you mentioned that it's it's best.
35:55
If you're on AWS, is there any Like prerequisites.
35:59
Do I need to be running kubernetes?
36:01
Or is there something that specifically I need to be
36:04
running to have it? As long as you're in AWS, for
36:09
will be able to figure it out because we use all the base,
36:13
level infrastructure, Primitives from AWS, Vu Z KS is the
36:17
kubernetes hosting solution. Dynamo for a lot of the key
36:20
Value Store, the low latency surveying S3 for the warehouse
36:25
msk for hazard Kafka provider and stuff like that.
36:28
And so as long as you're running on AWS, it'll be very easy to
36:31
get tecton up and running perfect.
36:34
So now what can you walk us through?
36:36
Like what Does the process of building a feature look like in
36:39
technology? How does that work?
36:42
Yeah, definitely. So you typically.
36:46
So what's a feature a feature consists of two things.
36:49
One is the metadata, it's like just the envelope of the feature
36:52
that describes. Hey, what's the name of it?
36:55
What's the entity that this feature is associated with say
36:58
like a transaction or a writer or driver or restaurant or a
37:03
combination of those is also, of course possible.
37:06
Then Who's the owner? What's the SLA of it?
37:09
Is it like an experimental feature?
37:10
Or is it a production feature? And what are the freshness
37:14
guarantees of this feature and all that type of stuff and then
37:18
decides that? That's because that's not
37:20
enough. That's just, that would only
37:22
give you like a catalog of information of metadata.
37:24
That's nice to have, but that's not enough besides that.
37:28
You also need to Define. Hey, we're how do, how do these
37:31
feature values, get calculated, and you describe a feature
37:35
transformation? That turns the raw data into the
37:40
feature values and we integrate with different data processing
37:44
engines. We have the best integration
37:46
right now, what's Park. And so typically what you would
37:48
do is you just Define the feature transformation using pi
37:52
spark code or scholar code. If you prefer that and you write
37:56
that all in Python files that are again, describing in a
38:02
declarative way, the feature Repository And then you once
38:07
you're happy with your feature, then you run then use our CLI,
38:11
your on tecton, apply for those of you who are familiar with
38:14
terraform, it's the exact same thing where you use our CLI to
38:19
basically ship this feature repository configuration that
38:23
you have locally, that will be get back for full versioning of
38:25
What Not to Tech tons back end and then tecton tells you.
38:29
Oh, this is the Delta. This is a difference that I'm
38:32
noticing, you've created a new feature or you've deleted an
38:35
Bonnie of change the configuration?
38:37
Are you sure you want? Do you want to do this then?
38:39
So yep, I'm sure or your Ci City pipeline says yep, I'm sure and
38:44
then tecton actually Stars spinning up and rolling out
38:47
those data pipelines. And then cure AIDS and
38:51
processes. These new feature values and
38:53
makes them available in the future store.
38:56
Great. And then how much automation is
39:02
involved in this and where does that come in the automation?
39:08
The tecton does for you behind the scenes or yeah it's a bit
39:12
of. Yeah exactly.
39:13
Like where's where is where are things that I would generally
39:19
have to manually. Do where is that being picked up
39:23
from tecton? Yeah.
39:25
Okay where do you think Here, there's so much you'd have to do
39:27
manually. There we go.
39:30
Maybe we need to hold another session on.
39:32
Let me, let me just. I'll walk you through the life
39:37
of a future. Like let's say, you've actually
39:39
just created a snippet which and like, in these, in your feature
39:43
repository, again, which like declaratively defines your
39:46
entire feature repository, all the features.
39:49
You just add a new you, add a new couple of lines here, which
39:53
is really just. Oh, here's a new feature.
39:54
This is my transformation is just some high spark code.
39:57
This is the name of it. Then you, if you are in a
40:01
production environment, you would not typically just get
40:03
committed into your good repository, and your cicp
40:07
pipeline, but to talk to tecton, using Tech Don's CLI, which
40:11
rolls it out. Or if you're just experimenting,
40:14
then you can on your laptop just use the CLI to as soon as you do
40:17
this. A whole bunch of stuff is
40:19
happening at tecton. Where tecton looks at.
40:21
Hey, what's the state of the world right now?
40:24
What's your new goal State? Okay.
40:26
A new feature. This is the raw data from which
40:29
this feature wants to generate feature values.
40:32
This is what the Transformations look like.
40:35
What's the, what's the revision? Think that's necessary in order
40:38
to run these Transformations? What's the, what are the
40:42
instance types for your spark? Cluster for your EMR cluster or
40:45
a database cluster on which you want to run it?
40:47
Then it knows. Okay.
40:48
New feature, this is the this is the provisioning I need for a
40:53
new spark cluster. This is that That time range for
40:58
which the customer or the user wants to backfill the future
41:00
value. And then tecton would say, okay,
41:03
let me first spin up a spark job on, say, EMR or data breaks, we
41:08
support both of those which now takes a feature Transformations
41:14
runs it as a spark job, to do a backfill over historical time
41:19
range that you specified, which then writes those feature
41:22
values, into both Technologies, online feature store.
41:25
Dynamo, if you're running on a Us as three-fourth often feature
41:29
store, then it knows. Okay.
41:31
Now the backhoe is completed, I know that going forward, the
41:35
customer really needs. I freshness for this future
41:37
values. How do I get the highest
41:39
freshness feature values? Well, I do get them by
41:42
connecting at this stream that this feature transformation
41:47
wants to wants to consume from. So then spins up a structured
41:50
streaming job as far as structured, streaming job, which
41:54
now continuously in steady state It keeps transforming feature
41:57
values and writes them into both the offline feature store and
42:00
online feature store. And then of course, all of that
42:03
comes with monitoring where the system notices.
42:06
Oh, like a back full job has failed.
42:08
Let me retry it or let me see what's going on here?
42:10
Why did it fail and also looks at the streaming jobs and chores
42:14
that they're running properly, it even looks at the the
42:19
freshness of those features that screaming jobs are producing
42:24
that continuously monitors them and then in You as well as
42:27
Texans on calling. Hey, something's going wrong.
42:30
And that's just a, like, a some stuff that subset half the stuff
42:35
of the automation that's happening behind the scenes with
42:38
effect on if you create an onboard, a new feature on to the
42:40
platform. So you can imagine it's a lot of
42:44
stuff that you would typically have to do manually and that
42:47
most companies are still doing manually today.
42:48
If you wanted to do this all by yourself without a platform,
42:52
Completely super insightful. So is there a way to when you're
42:58
creating these features? Is there a way to group them
43:02
together? Yes, there is.
43:06
So there, one at the most fundamental, physical level.
43:11
There is the grouping where we were you as a user group
43:14
features into feature packages, is what we call them.
43:17
And typically a feature package, refers back to A feature
43:23
transformation pipeline. Imagine just apply spark job,
43:26
really, and despised Park job. Can produce one or many
43:30
features. Typically it's more efficient to
43:32
compute many features of exact same time.
43:35
And so, all of those features together are now grouped by
43:37
default into one feature package.
43:40
Now feature packages can also be Associated on.
43:44
Should be Associated typically, with entities in your
43:47
organization. Again like that's those are
43:49
typically. The Core Business objects,
43:51
business entity. He's that do or organization
43:54
revolves around customers, Shoppers transactions products,
43:58
that type of stuff. And then you're also able to
44:03
group feature packages and to feature families in case you
44:06
want to have some additional partitioning on a team basis or
44:11
something like that. That's, for instance, something
44:13
that we had to do brr. Uber Eats is one of them.
44:18
Fraud is another one. So now talk to me about like Is
44:25
there some kind of? I obviously there's an I would
44:28
imagine there's some kind of UI that tecton has or is it only
44:32
just like this library that you're declaring with?
44:36
How does that work? And we do come with a with a UI
44:40
there. The the main interface is that
44:42
you're interacting with as a user.
44:44
Is on the one side that you I it just makes it very easy to see
44:47
what are all the features in my organization that makes it easy
44:50
to discover new ones filter them by entity by Department by
44:53
Future Family. Full text search and all the
44:56
stuff that you've imagined. Then there is a python SDK that
44:59
you use that you would use in your jupyter notebook.
45:02
Or um, our notebook or tail breaks, notebook to fetch
45:07
feature values typically, in order to generate your
45:10
historical training data and whatnot.
45:12
Where you say, hey tecton for this set of features, give me a
45:16
feature values or the last 365 days for these two hundred
45:22
business entities, and business entities will be A specific say
45:26
riders or drivers where you pass in their primary key into
45:30
tecton, then text him. Just generates the data frame
45:32
for you with all these historical feature values are
45:35
all these entities or the entire time range of these specified.
45:38
That's the python SDK. And then there is the CLI
45:43
honesty. Lies, what I mentioned a few
45:44
times. In order to enable this fold,
45:47
devops flow for features to tell tecton.
45:51
Hey, here's the new goal State. Look at my feature Repository,
45:54
Aggregation that I as a customer back myself and get look at it
45:59
and roll it out to production and tell me what the Delta is.
46:02
That's what he is to see life or.
46:04
And then, last but not least, there are, of course, we have
46:06
apis for the real-time serving of these features.
46:10
Like it to your PC and rest interface.
46:13
And we provide the client Side Library right now for the jvm to
46:17
communicate with these apis. But you can, of course, any
46:19
grades yourself, as well. Those apis, nice.
46:22
So, one thing that I I talked with Shira Jane a while ago
46:27
about their platform as SurveyMonkey when he was
46:31
creating that feature store and I asked him.
46:35
So do you guys have like a leader board of feature?
46:39
Whose features been used the most and who created it?
46:43
How I'm wondering if you have anything like that.
46:45
It's funny that you say that a little over a year ago, we built
46:52
a mock version of this, it wasn't connected to Real data
46:55
but we just we just put it in the end.
46:57
The product is like an Easter egg that we could talk show to
47:00
people who were demoing the product.
47:02
And so we do have that code in the code base, just to show what
47:06
it could look like. But it's not actually connected
47:08
yet but if there is interest for something like that, it would be
47:12
easy enough to to recycle that all to you.
47:14
I yeah I think I think it would be hilarious to have that
47:18
because it can breed a little friendly competition between 10
47:21
minutes for look at this feature that I created spinning Used
47:25
over x amount of times. And I'm also wondering about
47:28
this monitoring, you were talking earlier about like, how
47:32
feature stores can save money when you're looking at?
47:35
It features are actually being used and if they're not, is that
47:38
part of the monitoring that you were talking about?
47:42
That is one part of? Yeah, that's one part of it.
47:45
It's just the visibility into the cost of features and then
47:48
the ability to even identify, which one of those features is
47:50
actually used by model, but there is additional monitoring
47:53
psych data quality. Ring typically and data drift
47:56
monitoring where typically it was machine learning when things
48:01
go wrong. It's not that the model breaks
48:04
or that the micro servers that host the model breaks.
48:06
Like we've got that under control.
48:08
Now, I think with the best software engineering practices,
48:11
those are not the hard problems anymore.
48:13
What tends to happen though and ml bottles break, is that the
48:16
data breaks? And that could mean one of many
48:20
things. One thing that often times
48:22
happens is that an upstream data Offline is just lagging or it's
48:26
completely broken and so, no, new data is coming in.
48:29
What does that mean for the model?
48:31
The model is now getting poor feature values.
48:34
Either it's getting no feature values at all.
48:36
Are you guys getting completely wrong?
48:37
Feature values, where suddenly the standard deviation is 10 x
48:43
different? What you expect over the meaning
48:44
of shifted by a thousand or something like that and that's
48:47
something that we can detect and that we can alert on sometimes.
48:51
What actually is also, the root causes that.
48:54
There is no outage in an upstream data pipeline.
48:57
The problem is much more that the world has just changed
49:00
feature. Distributions have changed and
49:02
all you need to now do is we train your model and it's
49:05
actually really good to have this type of monitoring in place
49:07
because you'll just want to retrain your model every night
49:11
or every week, if you don't get an A performance boost out of
49:15
it, it will only want to retrain your model when you know, there
49:18
is a reason for it and that could be that your model
49:20
performance is starting to drop or more.
49:23
Interestingly, if you see that the Feature distribution
49:26
starting to change, are starting to deviate from the distribution
49:29
of that you've identified. When you first generated the
49:32
training data set, that's like the groove, the goal state,
49:35
where are you know what the distribution is look like any
49:36
training data set that you train your model on and then in real
49:39
time you can compare those distributions to what you're
49:42
seeing in production and if there is a a maintained
49:47
deviation between those two. It's a good sign that you should
49:49
probably retrain them all. And then the yeah.
49:54
I think it's just the data quality, monitoring that we're
49:56
doing on the feature pipelines itself.
49:59
That's more happening. Upstream when you're on these
50:01
bash and streaming pipelines, we look at the raw data.
50:04
Coming out of the raw data sources.
50:06
Does anything look off here and typically would try to do here
50:10
is just provide you with a good Great Expectations integration.
50:12
If you're familiar with us, So, I want to leave a little bit of
50:17
space in case. Anyone has any questions, feel
50:20
free to jump off of mute or throw it in the chat, if you're
50:24
a little shy. And I because I know there's a
50:26
lot that we've been going over. And so if anyone wants to ask
50:29
Kevin anything, go ahead, jump in.
50:32
Otherwise I have a lightning round.
50:35
I wanted to get to on just to see what your initial responses
50:40
and quick responses would be, but before We do that.
50:46
I wanted to mention that. It's super interesting.
50:50
How you were talking about this idea of just like, okay the
50:53
world may have changed. I know, we used to say that a
50:55
DOT science a lot with like a computer vision model.
50:58
That is a car, the self-driving car, and there is, you know,
51:04
like all of a sudden its nose. And this computer vision model
51:08
is like, is that a stop sign? I can't tell because it's under
51:12
snow, or there's snow on it, so that's that's huge.
51:15
Huge. Now, I see some questions here
51:18
in the Q&A. Let me see here.
51:21
So, Gonzales asking seems that you're focusing on AWS.
51:27
Did you choose AWS? Because your experts in this
51:30
cloud provider or you involved at some level with them is
51:34
they're like partnership going on and that leads to the next
51:38
question. Are you like is there some kind
51:41
of Co selling that's happening with AWS by In companies that
51:46
are using Sage maker or how do you approach prospective
51:51
customers? Yeah, definitely so we are an
51:55
AWS partner now most recently, we've got the we're part of
52:00
their official partnership program, but we chose it's them
52:06
mostly because that's just where most of our customers are on.
52:10
And really, that was just a market demand driven, most folks
52:14
today are still 180. Yes.
52:15
Yes, and that's what we started with, but we build a platform in
52:18
a way of where it would be fairly easy for us to lift and
52:21
shift to a different system. That's why everything is built
52:24
on kubernetes all the other AWS in fracking opponents that were
52:28
using, we typically decouple them and built against a generic
52:32
interface. So that would be easy enough for
52:34
us to switch over and support gcp as well.
52:37
Like we wouldn't be able to support it tomorrow but a couple
52:39
weeks of work and we'd be able to get it running on a juror or
52:42
gcp. If somebody comes to you and
52:44
asks for it, you can make it work.
52:47
Yeah, exactly. So the next question is from
52:50
Alex Panos, a also, a former guest on the Meetup and he's
52:55
asking about feature. Well, first he saying feature
52:58
stores are awesome. Yes they are.
53:00
I agree with that how Alex and he was wondering by focusing
53:04
solely on the data layer. Is there a risk that companies
53:08
will prefer solutions that are more end-to-end adding more
53:12
complexity. To the engineering
53:13
infrastructure is not ideal for smaller companies.
53:19
Yeah, it's a good question and I think it depends on the type of
53:21
company. There may be some companies to
53:23
really just want to buy an end-to-end solution that does
53:26
everything for you and like and the ml workflow consists of many
53:33
many different steps. Like Scala 10, different steps
53:35
that. Maybe you've got an integrated
53:37
end-to-end platform that implements from scratch tools
53:41
that help you with every single one of these.
53:43
Steps by going. So broad.
53:45
You can imagine that the individual components are just
53:50
60, 70 %, good. Implementations of these systems
53:54
because you just spreading your focus.
53:55
And so the intent platform just cannot be as good, a cannot be,
53:58
as flexible as the alternative to this, which we often times
54:02
now see in companies where you've got centralized data
54:07
science, platform teams or organizations that provide a
54:12
reference It's stack and an end-to-end platform to the data
54:17
science teams in the organization, they just say hey
54:20
this is the modeling tool, you should use like always use
54:24
Pythor juice since the floor flow.
54:26
This is the serving system that you should use these Selden.
54:31
This is the feature store that you should use ideally tecton.
54:34
And so they provide this opinionated framework to the
54:37
rest of the organization and typically make it easy for them
54:41
by integrating them with Other. So that the data scientists, the
54:44
engineers don't have to think the whole time.
54:46
Okay. Which component should I not be
54:47
stitching together and be using a concert?
54:50
But they just get this blueprint.
54:53
And this this platform by the central organization and they
54:57
still get the benefit that on every single step of the Amell
55:00
workflow, they're able to use the best in breed class tool for
55:04
that workflow step. And I believe that that's more
55:07
where the future is heading. With these ml platforms or you
55:11
will have these best in breed. Individual platforms that are
55:16
really, really, well integrated to give you a really amazing end
55:19
to end experience rather than having these end-to-end
55:23
platforms that Implement everything from scratch, kind of
55:27
reinvent the wheel to an extent and thereby, just give you a
55:30
much less flexible platform and experience that always Place.
55:35
Catch up with the latest things are happening in Industry.
55:39
Yeah, it's that idea of, you know, jack of all trades, master
55:41
of none. Yeah, exactly that.
55:45
Yeah there's so many so many interesting questions I want to
55:49
ask right now for you. It seems like it's really
55:51
getting good here. Now that we've got one minute
55:53
left. Are you cool to stay five
55:55
minutes over, but you have a hard stop right now.
55:59
One second, I can I can run five minutes over.
56:05
He can do it. All right, so Bobby on is asking
56:08
in the chat. She's asking cheeky, question,
56:10
how does tecton measure the cost of?
56:13
Of not having a feature store for a person perspective.
56:16
Customer like I guess this may or may not be something we need
56:20
to ask sales team but how do you map that out for someone?
56:24
If you're saying, hey, you guys need a feature store, it could
56:27
save you X. Yeah.
56:30
So many tecton is running as fairly easy to measure the cost
56:35
side, like we at the end of the day, on Italy, a specifically,
56:38
we run jobs on EMR. So we know exactly how much they cost.
56:42
How many know Ours are they consuming?
56:44
What costs does that boil down to?
56:46
And so you get extremely clear visibility into that costs of
56:52
not just tacked on as the entire platform.
56:54
But on an individual feature by feature level, you can see,
56:57
okay? And this feature cost me this
56:59
much for the data processing. That's feature cost me.
57:01
This isn't that much for fetching, the feature value for
57:04
training data purposes for training purposes, or this is
57:07
how much it cost me for real time serving.
57:09
So you get a really good clear picture of or where do the
57:14
majority of the costs come from. If you want to predict it for
57:18
predictive prospective customer, then the costs are a function of
57:24
how much data are you processing.
57:28
Is how many rows basically are you processing in order to
57:31
generate your feature values, how frequently do you need to
57:35
reprocess these feature values? And then it gets a little and
57:39
then how frequently are you consuming those feature values?
57:43
There is an additional trickiness of like what's the
57:45
cardinality of like your primary keys off individual features
57:48
that's also as factors in there. But by looking at your existing
57:56
data processing, jobs are looking at the data shape.
58:00
We can typically come up with fairly accurate estimates of how
58:03
much this thing is going to cost you.
58:06
So, another idea on these being able to filter your features,
58:10
can you filter it by the most expensive?
58:12
Features, not today. But we do have that information
58:17
center. I'm just thinking, you know, the
58:20
the people in the organization's who saying like, oh yeah I got
58:25
the badge for most used feature and then the other guys got the
58:28
badge for most expensive features but anyway so as far as
58:36
The question goes about. So when you're talking about,
58:38
all right, we see this happening where a lot of tools are going
58:41
to start integrating really nicely with each other.
58:44
How do you bet? What horse?
58:45
Do you bet on to integrate in Tech, tone and what are you
58:49
betting on right now? I guess, what do you integrate
58:51
really nicely with? As far as the rest of the
58:53
spectrum? Yeah, so we have an extremely
58:57
strong integration with data bricks.
59:00
We also have a very strong integration with AWS
59:04
specifically on AWS, Us with sage maker, which with EMR. And
59:09
with just, I am for ACLS, then we have blueprint, not just not
59:17
really Integrations, it's more like here.
59:18
This is a good stack and documentation on how to best use
59:23
tecton with ML flow. How do you best use tecton with
59:26
Selden or how do you best use technique done with algorithm?
59:29
You Perfect. So right now it's that you're
59:36
making the the bets on those guys and is it just because they
59:40
were Market leaders in a way or was there something else that
59:44
made you want to? We try to provide the
59:48
Integrations with really the best in class tools off the
59:50
other parts that cover data parts of the ml workflow and
59:55
typically it's not an extremely complicated tight integration
59:59
with these systems really Just being able to talk to each
1:00:02
other's apis at a fairly well-defined interface level and
1:00:08
it's typically not that hard to add an additional integration
1:00:11
but those are the core ones that we're coming out of the box
1:00:13
with. But we definitely want to
1:00:15
heavily invest in additional Integrations.
1:00:18
With the ecosystem going forward, Perfect.
1:00:21
All right, lightning round 2 minutes.
1:00:23
Here we go. We got it.
1:00:25
So I just you can give me a yes or no or one word.
1:00:29
Answers feel free. No question actually here on
1:00:34
whether that there is a huge Trend towards open source thing.
1:00:38
Mm improvising Enterprise level service, do we have plans to
1:00:42
open source technology on? So it's an ongoing conversation.
1:00:45
We do want to figure out which pieces what parts.
1:00:49
Does it make sense to open source.
1:00:51
That's an ongoing conversation that we're having with
1:00:53
customers. Where one, understand, hey,
1:00:55
which pieces will be helpful for US.
1:00:56
Open Source. Typically will open source
1:00:58
things where the customer needs. Built customizations or wants to
1:01:03
build customizations on their own or build deeper Integrations
1:01:06
on their own and it definitely will make sense for us to do
1:01:08
something there when, when the need arises.
1:01:11
So it's not off the table but there isn't a clear plan for it
1:01:15
on the roadmap because it's them.
1:01:18
It's an ongoing conversation and then on Enterprise level
1:01:21
service. So tectum is software as a
1:01:23
service, even if we deploy it into your BBC, until your
1:01:26
account, we manage it. We upgraded we security patch.
1:01:30
It we are. Run call of the thing breaks.
1:01:32
And so it's the proper production as well as the way
1:01:36
you'd expect it from any other SAS service.
1:01:40
Really, thank you for that, and thanks for the great questions
1:01:44
consult low. So, now, are we ready for a
1:01:47
lightning round? I just give me one word answers
1:01:51
or quick short to the point Cube flow and Grace.
1:01:59
No, maybe so Lots of components air flow.
1:02:08
Very widespread and great Cube flow and air flow together.
1:02:13
Confusing and Val slow, very good and now drastically
1:02:20
changing our life in the next five years.
1:02:23
Absolutely, coronavirus recession.
1:02:26
I hope not tensorflow or pie. Torch light, orange food you eat
1:02:34
the night before one of your long-ass.
1:02:37
Endurance races. Lots of pasta.
1:02:42
Good ml monitoring best tool. Who still to be seen?
1:02:47
Mmm where to start learning ml Ops?
1:02:53
Emma laughs Community. There we go.
1:02:55
I like that answer. One thing that we should know
1:03:00
about Germany or I should know mainly be excited for next
1:03:07
year's Oktoberfest. Are you guys hiring a tecton?
1:03:12
We are in the its data Engineers.
1:03:14
Ml Engineers data scientists There we go.
1:03:19
To sales Engineers. So last one your personal most
1:03:25
used open source tool. Cornett is hmm.
1:03:33
Alright, that's it. Everybody and best place to
1:03:38
reach out to you. I'm not going to say it's
1:03:39
LinkedIn because I've had a LinkedIn request to connect with
1:03:42
you for a few days. Now, it doesn't get through.
1:03:45
I imagine you're very, very popular, and I got lost in the
1:03:48
sea of people that are trying to get you to Outsource them, some
1:03:51
coding in India or in Eastern Europe, wherever they're coming
1:03:56
from. So are you on your on Twitter,
1:03:59
right? Yeah.
1:04:01
Or tell us, is there another way?
1:04:03
Maybe we can get you into the ml Ops Community chat.
1:04:07
Twitter works with Eminem's dumb or just my email, Kevin and
1:04:10
tecton .ii, perfect. And so for everybody, that
1:04:13
didn't hear it before, we are doing a giveaway for tecton.
1:04:18
We are giving away 50 dollars or Euros.
1:04:22
I can't remember what we said. I'm in Euros these days.
1:04:25
I think it's $50 of tecton credit.
1:04:28
So if you go onto tecton, You have the link, he throw that in
1:04:32
the chat, somebody maybe Kevin if you go to tecton dot Ai and
1:04:38
you sign up for text icon you will get to get some.
1:04:44
Make sure to put that it's Mo Ops you were part of ml Ops
1:04:47
community. So they give you the special
1:04:49
treatment and there it is. I think that's about a D&D.
1:04:54
Hey ya, ya quick clarification to gift card is actually going
1:04:59
to be most likely an Jean gift card.
1:05:01
Not not ticked. On credit.
1:05:03
All right. There we go.
1:05:05
Amazon works. Also, that's all good.
1:05:07
I'm sure everyone is happy to get any kind of gift card.
1:05:11
So, thank you guys, very much for doing this.
1:05:14
This has been really enlightening as far as features
1:05:16
stores go for me. I know I learned a ton.
1:05:19
I imagine everyone else did and Kevin.
1:05:23
You've been an awesome guest. You were very, very kind to
1:05:26
break it down, real slow for me, and make sure that I understood
1:05:29
everything So that's about it. I think we're good here and
1:05:36
please. Yeah, please reach out to Kevin.
1:05:41
If you have any other questions or check out tecton and we will
1:05:44
see you next week. We got to meet ups next week, so
1:05:48
enjoy. See you all later.
1:05:52
Thank you. See y'all?