Latest / Agentic Conversations (formally mlops.community) / MLOps #14 Kubeflow vs MLflow with Byron Allen

Transcript

MLOps #14 Kubeflow vs MLflow with Byron Allen · Agentic Conversations (formally mlops.community)

0:19
So let's go ahead and get started.
0:23
I think we can hit it now. We are having a very special
0:29
meet up to Today, this one is and I want to want to stop
0:35
sharing this screen and share a different screen with you that
0:39
is going to hopefully make you laugh but we managed somehow to
0:48
get a sponsorship from Chow cheese today.
0:54
We are going to be talking about Jeezy and analogy between Cube
1:00
Low and slow. So I got an amazing guest here,
1:08
an expert in Cube flow and ml flow.
1:11
You may have seen some of his writing on this very subject on
1:14
medium or you may just know him on LinkedIn and he is a world
1:20
traveler. Originally from Texas.
1:22
I believe found his way over to Australia and then eventually
1:27
landed in Fancy on the Chatters. The people making jokes, so
1:33
good. He landed in in London.
1:36
And without further Ado, please welcome our newest guest on the
1:41
show. Byron Allen.
1:42
Thank you for coming. How you doing today?
1:44
So having me I'm doing well. Great, great.
1:47
So I'll stop sharing that please everyone support your local
1:51
cheesery and child, cheese is a great one.
1:57
If case you were wondering, we have Special discounts codes for
2:01
anyone who wants it. Just contact us afterwards.
2:05
The other thing, the other thing that I want to mention before we
2:09
get started, I know I already introduced you but forgive me,
2:12
we have a slack community and Chris will put that slap
2:16
Community up in our chat. And so if you are looking to
2:24
deepen your knowledge in everything and anything ml, Ops
2:28
whether you're a beginner to Anst, jump into slack.
2:31
There's people from all over a lot of the past speakers are in
2:34
the slack and they're very generous with their time.
2:37
So it's a great place to ask questions.
2:38
If you get stuck on anything which is devops for machine
2:43
learning or this machine learning operations, we talked
2:46
about all kinds of different platforms that are around and in
2:49
the space I know right now everything is changing so
2:52
quickly so it's one of those things that you want to find a
2:57
place where you can stay up-to-date on the Newest
3:00
technology and the best practices which right now as far
3:04
as ml Ops go, it's all kind of in the air so we're learning
3:07
together as we can say. Now that I got that out of the
3:11
way that plug, let's go ahead and jump into this Byron.
3:15
I wanted to ask you first question, what made you go all
3:20
around the world? How did you go from Texas to
3:23
Australia to London? Give us just a background.
3:26
Is it was a long trip. So, So I grew up in El Paso,
3:31
Texas, and lived there until I was 18.
3:35
I studied Japanese and high school.
3:38
And decided I wanted to do my undergrad in Japan and so, it
3:42
did that, I pursued that and got my undergrad, business degree in
3:48
economics degree. And then from there, I moved it
3:50
to Korea where I worked in, kind of marketing roles for LG
3:55
Electronics as a consultant. And then I came to Australia,
4:00
Bella similar capacity and then that's kind of when I started to
4:05
get more into Tack and realized I had a knack for analytics and
4:10
a love for coding and developing things and I spin a year
4:17
re-skilling and just absolutely head down and just trying to get
4:22
into the space and I did successfully and that's how I
4:25
ended up at where I currently am, which is Serbian.
4:28
So serbians a data analyst, Consultancy headquartered in
4:32
Australia. And the company has since I've
4:36
joined three years ago, expanded a fair bit and has come as far
4:43
as London and I came as one of the first people to London.
4:45
And so, that's how I ended up in London.
4:47
Awesome, awesome. So, you were just enjoying life
4:51
traveling around the world, learning, Japanese learning
4:54
Korean. And do you speak any other
4:56
language? It's mostly Japanese.
4:57
My career is horrible. I've tried.
5:00
Turkish Danish, try to smattering of languages and just
5:04
for fun, but mostly learning British English.
5:07
I presume. Yes.
5:10
So I try not to mix accent. It sounds weird.
5:14
Yeah. For me like you don't want to
5:15
hear me speaking a Texan accent. So let's let's get into a bit of
5:21
this ml Flow versus Cube flow. Why?
5:24
They're not the same. I want to just start maybe for.
5:28
I'm I imagine Everyone here that's with us today.
5:32
Has a good overview of what they both are, but just maybe a quick
5:35
overview of what they both can do.
5:39
Yeah, absolutely. So ml flow can flow.
5:42
It is kind of, an unfair comparison.
5:44
They are very different. And in the article that I wrote
5:48
about the cheesy analogy between the two of them, you know, I
5:52
compare ML flow to cheddar cheese and I can a good flow to
5:57
cacio e Pepe which if you're not like hipster food snob like me,
6:01
then to break it down. That's just kind of
6:03
sophisticated mac and cheese like being the Buffaloes very
6:06
versatile. And it is a pip install away.
6:10
It's primarily built to help practitioners track models in
6:17
addition to artifacts. So that could be files images
6:21
anything associated with the development of a model and
6:24
something that would essentially be a dependency along with
6:26
parameters and metrics. So, the point being that, if you
6:30
look at the Paradigm of Version Control, just code you're not
6:35
able to do that as easily with a melon.
6:37
I think probably a lot of people Want to call appreciate that
6:39
point. That's a big foundational point
6:42
of ml Ops and why we do what we do ml flow is a very lightweight
6:47
package to enable that capability so that you can track
6:51
all the artifacts that are needed in order to Version
6:54
Control your model sake, in addition to that, they've in the
6:58
past year and a half, they've come out with a thing called
7:01
Model registry which is important for the governance of
7:05
it and I can get into that in a bit.
7:08
So It's a lightweight package people on the other hand is a
7:12
collection of several different components that come together to
7:16
make a system that sits on your bananas and it requires
7:20
communities to run where it's Mo.
7:22
Flows is agnostic about a grand brunette easy to run out of VM
7:26
can run locally but for food flow, you need kubernetes and it
7:30
attempts to play the role of an end-to-end pipeline, all the way
7:35
from Discovery using notebooks, Through to pipeline development,
7:41
training, the model, and then putting it into production.
7:44
Which is when I say that, I mean deploying it and serving model
7:48
serving models. And so why do you feel like
7:51
they're so commonly compared against each other?
7:53
Because I've seen that a bunch. Is it?
7:55
Just because Microsoft backs one and Google back to the other.
8:00
And so, it's an easy story to say that they're fighting or
8:03
they're trying to take over the space or is it a bit more in
8:07
depth than I think the the comparison comes more because so
8:13
many people are getting into this Arena and maybe your data
8:16
scientist who's not really familiar with devops or your
8:19
devops person not familiar with it.
8:21
Machine learning and so ml Ops is kind of this space where it's
8:24
a little bit new to a lot of people.
8:27
And so your first question is, well, what are the tools?
8:29
What can I use? What where do I get my where I
8:33
get started? And these are the two tools that
8:36
have Come very popular. And I think they're probably the
8:40
most talked-about tools and that does come from to your point.
8:44
There is a bit of a camp thing going on and they'll flow was,
8:48
is really coming out of data bricks.
8:49
It's an open source tool and you don't have to have data bricks
8:53
to run ml flow but they are a big source of the development of
9:00
ml flow. Whereas when it comes to crude
9:02
flow, there's probably I don't want to see a broader group of
9:07
people but A very broad group of people, but a big backer of the
9:11
could flow project is Google. And so, there is there's two two
9:16
sources of these tools as well. And I think that does Play Into
9:19
The Narrative of why people compare them.
9:21
Yeah, yeah, yeah. So where do you feel like?
9:25
I guess you've explained it a bit there, but Are there are
9:30
overlaps and how did how do they overlap in that sense?
9:36
Yeah. So ml flow really serves as a
9:41
artifact archive if you will it allows you to store all these
9:46
kind of dependencies for models to kind of Version Control your
9:50
model and in a similar way, there are components in
9:55
particular. This one called metadata on Cube
9:58
flow that allows You to do something very similar.
10:01
I would argue that the interface and the way in which Mo flow
10:05
operates is a little bit more user-friendly, a little bit more
10:08
robust live a more developed not to say that metadata isn't good,
10:12
is very good. But there is it seems that Emma
10:16
flow is a little bit more advanced than that.
10:18
And it also offers one key thing which I think does is really set
10:21
it apart from metadata and other offerings on Kook low which is
10:27
the model registry. So If you're familiar with get
10:31
lab or any of the Version Control tools that you would see
10:36
for like, you know, she ICD tools.
10:39
You will when you get a pull request and you need to start to
10:43
approve code in a similar way. You really should be doing that
10:46
as you're pushing models into production and in that way,
10:51
having something that allows you to govern that mechanism is
10:54
really important and model registry enables that
10:57
capability. Yeah, that's something that has
11:00
come up. I remember our second guest on
11:03
this show, Charles Radcliffe. Really talked about that when it
11:07
was taught, when he talked about governance and especially in
11:09
fintech. And in banking, you need that
11:11
stuff in there. Yeah, it's very important,
11:14
especially with the in, if you're in the European Union,
11:17
how the there's new white papers that are coming out and they're
11:21
talking about laws and regulations that are looking to
11:25
say, hey what is your motto? Registry.
11:27
What what kind of Models. Do you have in production even?
11:30
And a lot of places with a lot of different models in
11:34
production or a lot of different things going on.
11:36
Don't know that they can't properly answer that so I can
11:40
see how that is is very vital for a system, absolutely.
11:46
Now let's talk a bit about the engineering work that you need
11:51
for standing both of these up. What is it?
11:54
Because I know and your blog post you mentioned you touched
11:56
on on both of them and What it takes to actually Implement them
12:01
into your company or into your just onto your laptop.
12:05
So can you go into that a bit more?
12:08
Yeah. So ml flow is it's it's a pip
12:13
install a way. It's kind of the way I like to
12:15
phrase. It is very lightweight.
12:17
Essentially, if you were, for example on a cloud provider, you
12:21
would be spending up a VM for example, and it would be it
12:28
would be Of installing enough flow into that VM or you could
12:32
even do it locally. The point being, when you
12:34
install that package, you can then point it to either a remote
12:37
machine or a local machine to store an archive.
12:43
The artifacts the parameters and the metrics that you're going to
12:46
be tracking. Every time you run a new model
12:49
and every time you need to essentially Version Control that
12:51
model in a similar way, you can also point to buckets.
12:57
And so it's very straightforward.
12:58
In terms of just setting it up, its you install any kind of
13:02
pointed to where you want it to store these artifacts?
13:07
It's pretty much a straightforward and then you can
13:09
run a command line to set up the gooey.
13:11
So you can set that up. Very in a very, very quick
13:16
order. But when you look at Q, Chloe,
13:18
it does require kubernetes. And so, if you're looking at a
13:22
non prior moment where you're looking at your local
13:25
installation or if that's within a cloud provider, you're going
13:28
to need to set up a kubernetes cluster or something where it
13:31
can run on communities that for some data science teams, who
13:36
might be a little bit challenging, it might be a lot.
13:38
I mean we all know that data scientists machine learning
13:42
practitioners, there's kind of a spectrum and they fall
13:46
everywhere from like more kind of academic researcher towards.
13:50
You know people who were more like a male engineers and maybe
13:52
have more devops backgrounds and less machine learning
13:55
backgrounds. And so, when you talk about the
14:00
ease of setting something up, I think that really comes down to
14:02
the teens, you have in play and who you have access to, and
14:05
there are people that are going to be more familiar with
14:07
kubernetes, and it's going to be a lot easier for them to set up
14:11
could flow, especially if you're doing it manually.
14:14
And that, you know, that plays into the dynamic that who you
14:20
have in your team and how skilled up.
14:23
They are and ready for this. I think that's kind of one of
14:26
The barriers to entry. We can follow that said you do
14:31
see movements more towards setting up coupla as kind of a,
14:36
as a infrastructure as a service or platform-as-a-service, I
14:40
would suspect in the same way that other open source projects
14:44
have moved on Google Cloud. You'll see a similar thing
14:48
happened with confluent in Google cloud and so that will
14:52
also that itself will lower the barrier to entry.
14:55
Yeah, completely I see something on the chat and I was just
14:58
saying how Chris was mentioning that these little micro cubes
15:02
are an easy way to get up and running with Cube flow.
15:04
That's definitely true. I think one thing is to have
15:09
something that you can play around with and another thing is
15:12
to have something that you can actually put out and have in
15:18
production. Have it going?
15:20
And you are very confident about it.
15:23
So yeah that's when those two end Joseph's asking if Can ask a
15:27
question via voice, feel free Joseph whenever you want to jump
15:30
in now would be a perfect time. I'll tell you the floor so you
15:34
guys can hear me. Okay now, sorry about that.
15:38
My understanding is an ml flow. Basically touches a couple of
15:41
things. Experimentation it's a model
15:43
repo, I think one more thing. How does it compare with some of
15:47
the other options we have out there?
15:49
Like I don't know ml comment from experimentation, you know,
15:55
I'm just thinking you know? Pachyderm, Etc.
15:58
And in terms of like model repo, obviously you've got things like
16:01
data Tron and things like that. How does it compare?
16:03
Why should I use one solution versus three Etc?
16:08
Yeah, I mean I think when you're looking at all of these it is a
16:14
case-by-case basis and I wouldn't Advocate one tool as
16:18
the best solution because this entire industry is moving so
16:22
quickly that you know, the moment you you can very quickly
16:27
invest in one product and all of a sudden something kind of sets
16:30
up. One of the things that I and
16:32
this may not directly answer your question, but I hope it
16:34
kind of explains where I'm going with this.
16:38
Is when you're looking at a male.
16:40
So I find it to be a really good introduction into how the
16:45
process works because it is lightweight, it is very
16:47
accessible. There is minimal overhead in
16:51
terms of getting up and running and using it as as that kind of
16:55
what I've described as an artifact archive compared to
16:59
other products. It does do similar things.
17:02
I find that the ml flow you. I personally I am preference it.
17:08
And for me, if you are working with that a brick, say that can
17:11
be handy because it works kind of seamlessly in that
17:13
environment. I've most often kind of come
17:19
into contact with this comparison of DVC versus enough
17:21
flow, which I find DVC is more kind of geared towards data,
17:25
Version Control, and actually version controlling your data
17:29
that sits behind your model. And whilst that is really
17:33
important, and very valuable and a part of something you would
17:35
have to bake into in Alpha. I wouldn't be able to natively
17:39
do that. You can also do some of that
17:42
kind of stuff with tools that are more specific to certain 11
17:48
data warehouses. So like snowflake helps you do
17:51
data Version Control bigquery kind of offers a very similar
17:54
thing. Does that kind of help I'm going
17:59
to assume? No news is good news.
18:03
I'm sorry, I wasn't cute. I can't figure out these buttons
18:06
now. What what I'm my issue is at our
18:11
company is that we want to focus on reproducibility.
18:14
Obviously you need all these little tools lined up and
18:17
capture all this information in order to then press the magical
18:21
button and automatically train a solution getting all this.
18:26
Tools to work together is a major challenge?
18:29
Yeah it absolutely is. And usually when I'm advising
18:33
clients on this face, I tend to try and assess their maturity
18:37
and see where they're at in their Journey because there are
18:40
some instances where there's a level of maturity that will mean
18:44
they can start cracking with crew flow and they will be fine,
18:49
but sometimes there's not a lot of role of maturity.
18:51
And in that sense I find that, you know, doing some
18:55
exploration. With a very specific use case in
18:59
mind using a tool like an overflow or maybe Pacha, Durham
19:02
or DC can be really valuable just for their own education and
19:06
learning. But you know, if you're if you
19:09
do use them fo yeah you do have to connect it with other things
19:12
to fulfill the entire pipeline. Yeah, and that that brings up
19:19
two questions in my mind, one, just talking the other day.
19:24
With someone about how there's a bit of this philosophy right now
19:28
that we're seeing with vendors, which is people are trying to.
19:32
And I think the cube flow verse ml flow battle is a perfect
19:37
example of this. How it's do one thing well, or
19:41
try and do everything in the whole Pipeline and really what
19:46
it sounds. Sounds to me, like Joseph is
19:47
your, if you want to get everything, you have to patch
19:51
them all together and then making them, work seamlessly is
19:56
where the heavy lifting comes in.
19:58
As opposed to just installing ml flow is fairly seamless, but
20:03
when you need to put ml flow with others, that's where it
20:07
becomes a bit more work intensive, so I could see that
20:11
point for sure. Now, the other thing I wanted to
20:14
ask about you mentioned using Data bricks with this, can you
20:18
go into that a bit more? How ml flow works with data
20:22
bricks and just for further record?
20:24
So everybody knows I should say this, we are not we are not
20:31
being paid by data bricks or Google or anybody to talk good
20:36
about them. So we're going to be very
20:38
non-biased. We're going to talk about the
20:39
good stuff and the bad stuff so yeah give us give us a rundown
20:43
Byron on how that works. Absolutely.
20:46
And yeah. Yeah, to be clear, not not
20:48
preferencing any of these tools, they all all tools have their
20:52
place and purpose and I think it's really right tool for, you
20:57
know, you got to find the right fit for you and for your group,
21:01
so, to how ml flow fits into donnybrooks, it fits in very
21:05
seamlessly, Fabrics has basically hosted notebooks and
21:08
allows you to run spark natively and seamlessly within their
21:11
environment and you're able to kind of out the click of a
21:15
button access. Mel flow.
21:17
It's basically baked into the platform at this point, it's
21:20
very easy to use and run. What the terminology within ml
21:24
flow is called the runs or experiments.
21:26
So you can run models multiple times and see performance change
21:32
as you adjust parameters or as you adjust your Source data or
21:37
whatever it may be. And yeah, it's kind of that
21:41
simple. But that also again like you
21:44
know, if you're if you're on data breaks, you know.
21:46
That kind of keeps you in the data bricks environment and
21:51
there's a lot of benefits to that and that's a whole probably
21:54
other conversation as to whether you want to be in debt of bricks
21:56
or who clue that also revolves a lot around spark and how much
22:01
you need spark. And also for that matter what
22:04
cloud provider you're using. If you're on Google, you're not
22:06
going to use data bricks but it's available on other sources.
22:10
And if you are in Google Cloud, there are other options other
22:13
than data bricks as well. There's not a proc where you can
22:15
run spark jobs So so I see I see a question here in the chat.
22:20
I'll get to that in just a second.
22:22
I wanted to ask what you feel. Some of the downfalls are with
22:26
ML flow before we kind of button up this section and then maybe
22:30
move on to cube flow a bit more. So doubtful downfalls with ML
22:36
flow, it is very limited and this when you one of the things
22:41
and in my opinion that is absolutely Central, I think a
22:45
lot of people practicing now L will have experiences.
22:48
You know, you can change the parameters of a model but
22:51
there's a lot of times when it's adding more data whether that's
22:55
variety or volume to the model, is one of the key things that
23:01
changes it. And so keeping track of your
23:03
data itself is really important to do that with them ml flow.
23:07
You're really going to have to kind of, it's a little clunky
23:11
and you know, you're going to have to take extracts and save
23:14
them into storage, something like that or have some sort.
23:16
Out of reference to the data itself and have those extract
23:20
stored in a different way. And you know, that's not the
23:24
worst thing in the world but to me that was maybe one of the key
23:28
things that was One of the downsides I felt and also it
23:34
doesn't again it doesn't. Its purpose is not to fulfill
23:37
the life cycle. It has a special niche.
23:41
That's something. It makes it a bit harder to
23:44
reproduce What You've created when you don't know exactly what
23:47
data you've trained that model on.
23:50
Yeah. And you can do it but I just
23:52
find it. There's there can be others more
23:55
seamless ways and I think there will be in the future especially
23:57
as if you are leveraging, olap databases in a way to help a
24:04
Version Control. That to me, is the preferred way
24:07
to do that version control of your data.
24:10
Is through the the database itself.
24:11
Alright, so got a great question here from an anonymous attendee.
24:18
If I have ml flow and air flow running on kubernetes, do I
24:24
still need another Model Management experiment tracking
24:27
and orchestration with Cube flow or do ml flow and Cube flow
24:32
complement? Each other, in an ml Ops
24:34
Pipeline and this we kind of touched on before everyone
24:38
jumped on but go ahead. Give it your best shot.
24:42
Yeah, so there is kind of a comment about airflow in there
24:46
as well, and hopefully, I'm heading.
24:49
I'm addressing this in the way they're after.
24:52
So are a lot of times you hear people talk about using maybe ml
24:57
flow coupled with air flow or just air flow independently as a
25:00
way of managing the orchestration of of the ml
25:06
training and development process, the entire life.
25:10
Cycle. And you can certainly do that
25:12
and it becomes a lot more Hands-On.
25:14
If you go down that route, and if you had air flow and if you
25:19
had ml slow on kubernetes, would you still do cupola?
25:23
I think was part of it, you know, you maybe you could argue
25:26
that if you build it well enough that you wouldn't need it.
25:30
But if I was going to go that direction, I would I would kind
25:34
of lean towards maybe just depending on include flow since
25:37
it's already there unless you have some ass.
25:40
Aspect about include flow that in air flow that you can't
25:44
fulfill and could flow which there's definitely foreseeable
25:51
things where that could be the case.
25:54
This is so fun to answer directly but basically, like if
25:58
you're using ml flood to end and airflow combined to fulfill what
26:03
Google does, then, yeah, you can report.
26:05
You could use it instead of could flip.
26:07
But in that instance, that sounds like a lot of work.
26:10
To me and I probably personally would lean more towards people.
26:15
Great. So consult those asking what
26:18
your opinion on Cube flow pipelines, first ml flow
26:22
combined with something like and row, or even when used on top of
26:27
data bricks and how they compare as that whole solution.
26:34
So I am unfortunately not familiar with Tinder.
26:36
Oh, that much. That might kind of limit my
26:40
ability to handsome at it too. So no worries on that.
26:44
But I think the idea. And the question is something
26:47
like ml flow being used with other tools to create more of
26:52
that whole end-to-end pipeline? Yeah.
26:55
I mean so you know for the something you could actually use
26:59
and you could use ml flow inside of people pipelines.
27:04
For example, if you wanted if you preferred, how ml flow
27:08
performed model tracking, that is absolutely something.
27:12
That is technically possible. And I haven't heard of many
27:16
people doing that, because again, you do have that
27:19
capability within Cuoco to kind of purple do model tracking and
27:22
do artifact tracking. So it's not a hard requirement
27:28
to do that. People it does a pretty good job
27:30
of trying to address the end-to-end pipeline that promise
27:34
and it's definitely going into that direction.
27:36
But one of the things that I've experienced and heard other
27:39
practitioners expressed, this is that it's very, very useful, but
27:43
it can be clunky at times, and In particular, I think that was
27:47
more the case probably a year ago especially when kale wasn't
27:53
as prominent because I think a lot of people who do come to
27:56
this, they are coming from more of a data science background and
27:59
KL is a little bit more of a data science friendly tool.
28:02
And if you're not familiar with kale, basically it allows you to
28:06
create Coupe flow pipelines, very easily from a jupyter
28:11
notebook by adding in tax, which is Akin very, very convenient
28:16
but the downside to that is it makes your notebooks a
28:20
dependency to your pipeline. So there's some data Engineers
28:25
that would maybe prefer the fairing approach which allows
28:28
you to develop pipelines through that the API instead of through
28:33
this kind of like tags embedded into the books.
28:36
Hmm. And if anyone wants to know more
28:39
about kill and all that good stuff, we had Josh bottom on of
28:44
probably like a month ago now. So that recording is on our
28:48
YouTube channel. You can go check it out and he
28:51
goes real in-depth into kale and I love the name on Cale, just
28:56
that alone almost had me sold on it.
28:59
But moving on two sides that icy Giants asking, how do you
29:06
compare tube flow with sage maker in your eyes?
29:11
To follow the sage maker, ask your question that kind of
29:14
brings up the whole idea of, you know, do you go for a Canoe
29:20
Club? This kind of Open Source product
29:21
that runs on kubernetes or do you go for more of these Cloud
29:25
native applications? Let's say sage maker.
29:28
Let's say, AI platform or what a the options as you are.
29:35
And I think it is a really good point because if you look at the
29:38
development of stage maker, if you look at the development, But
29:41
they I platforming in Google Cloud.
29:45
They're they're they're all kind of going in the same direction
29:48
really they're trying to offer the same thing further down the
29:52
track, they're all slightly at different points of their
29:56
Journey, but the animal is kind of the same thing.
30:00
And so I think the question really becomes where are you
30:04
currently running? These workflows.
30:08
Are you if you're in an on-prem environment?
30:11
For becomes very attractive. If you're in a multi Cloud
30:14
environment, could flow becomes very, very attractive but not
30:19
everybody is like that. If you're very much tied to one
30:22
cloud service, it could be maybe more useful to lean on.
30:26
For example, if you're on AWS stage maker, if you're on Google
30:29
Cloud mean on a platform because it offers you a lot of similar
30:33
capabilities. There are certain things that
30:36
you're not going to get those platforms are really good.
30:39
Example, in my eyes is Is is Feast, which I think is really
30:42
quite special for those that don't know what Feast is.
30:46
It is a, it is another component of coop flow.
30:51
That is, it's not off, its kind, it's becoming more popular but
30:55
it's brought in to decouple feature engineering.
30:59
So if you're familiar with feature stores, it's used to
31:02
generate feature stores in a way that decouples the data
31:05
warehouse from the feature engineering aspect is that this
31:09
can be a very efficient in terms.
31:11
Generating and also keeping track of those features for
31:14
reproducibility. And so, you know, feast in that
31:18
way, that's maybe one at one. One example of something where I
31:23
I like what Google offers, you know, you can.
31:26
It's it has kind of access to some of these earlier components
31:30
when they're in there more raw State, you know, that, for
31:33
example, is not a component that has like an SLA behind it.
31:37
But if you look at Sage maker, you look at AI platform.
31:39
Those those generally have some sort of SLA behind it.
31:43
So it depends on what your organization's goal is and what
31:45
you need. Yeah, that's really insightful.
31:49
Thank you for that. So now, can you give us a few of
31:53
the downfalls that you feel? Cube flow?
31:55
Has some of the downfalls? I mean it goes back to this idea
32:03
that that I've heard a few times the sentiment that very useful
32:08
but clunky and I think if you asked me a year ago, I probably
32:12
would have said clunky but very useful but I think it is it is
32:16
developing Way that it is becoming very useful.
32:20
And when you start to look at tail and the progress on varying
32:26
over the last year, these kind of tools are do make it a lot
32:31
easier previously, you know, before that it was kind of it
32:35
felt like you really had to stitch these components of
32:37
together and it was, it could feel very difficult and you
32:41
might need people with more capability and knowledge around
32:46
Cooper. Nettie's that you still kind of
32:49
need that especially if you're managing it on Prem.
32:54
Down side of it. You know, it's really this kind
32:58
of it is an ecosystem. It is a fragment that ecosystem.
33:01
So for anybody that was familiar with Hadoop and a lot of the
33:05
components that came along with that, there is a certain
33:08
similarity in that in that you're basically taking a bunch
33:11
of Open Source projects and collating them together into a
33:16
tool and that there's there's issues especially you know the
33:20
earlier on you are in that process of developing that
33:23
System. There is inherently some issues
33:27
with compatibility and being able to seamlessly set things
33:31
up. That said it's become a lot
33:35
more. It's become a much more seamless
33:39
experiment experience and I think it still has room to grow
33:43
like a lot of these tools in the ml up space.
33:47
The reality is a lot of the stuff is new and it's not
33:50
perfect. None of it is.
33:53
Yeah. Yeah.
33:53
And I'm wondering, do you feel that it's realistic for a data
33:57
scientist to say? All right, I want to use Cube
34:00
flow and stand that up by themselves at their company, or
34:05
will they need to bring in Heavy Hitters like someone on a devops
34:10
team or other different kinds of teams?
34:15
Well, I would certainly say if you're setting up poop flow,
34:18
especially if it's on premises, on your own infrastructure, you
34:21
probably do you need somebody who's very familiar with
34:24
communities in order to to get it up and running and not just
34:28
get it up and running but also like maintain it going forward
34:31
and sometimes getting things up and running is very easy, but
34:34
maintaining it as you move forward.
34:37
That is the biggest challenge of all.
34:40
And so, yeah, I would say on Purim that's going to be more of
34:44
a challenge. You're doing on the cloud, it
34:47
becomes less of a challenge. But also there's the trade-off
34:49
that if you are on the cloud, maybe you want to start maybe
34:53
using the inbuilt options like Sage maker like a platform, you
34:59
know, becomes may be equally if not more attractive depending on
35:03
whether you're a multi Cloud environment or not.
35:06
And I see another great question in the chat.
35:09
I can't pronounce your name. Sorry pratik.
35:12
But they are asking if you Feel Cube flows production.
35:16
Ready? Have you seen anyone or any
35:20
organization using it in production?
35:23
Yes. I have seen organizations use it
35:25
in production because of the nature of this, you know, these
35:29
these projects whether it's using control or not the take
35:33
time, you know. And it's not just about setting
35:36
up the tools, either it's about the business problem.
35:39
You need a business, you need to use case, you need to be able to
35:43
articulate your Lunch and to understand how to address it and
35:50
go through the data science development phase in a concise
35:52
manner. And to do that, you know, all of
35:56
this takes time. So the people who I've seen have
36:01
this running, well, you know, they've probably been at it for
36:04
a year or two, you know, but I think you also got a bear in
36:09
mind these tools, they are also advancing at a rate where if you
36:13
were to pick it up. Now, it will be Easier than what
36:16
it was for people. A year ago, for example, sorry
36:23
to cut you off, go ahead. If you have, yes, I hope that
36:25
answered the question. It just reminded me that when
36:29
Josh bottom was on here, he was saying that Cube flow comes out
36:32
with a new view version. I think it's every 60 or 90
36:36
days. So it's very true, that idea
36:39
that it's advancing at such a rapid pace and they're coming
36:43
out with a new version. So quickly that if you wait,
36:48
maybe six months, you're going to have a whole different
36:50
experience than you would have if you picked it up today and
36:53
that's not just came flow. It's a lot of these tools is
36:56
that almost every single one of them.
36:58
I've definitely I won't name it but there was a tool where I had
37:02
an issue with backwards compatibility because they age,
37:04
they change the code, the base and you know, it call it caused
37:10
an errand. We had to go in and amend their.
37:14
Okay. So I'm going to put you on the
37:16
spot a little bit right here in your blog post.
37:20
You talked about this production ml value chain.
37:25
Remember that one? Oh, yeah, definitely.
37:28
So did you create that or is that something that I just am
37:31
first seeing now? That's that's my way of looking
37:34
at it. That is that's my Approach
37:37
towards explaining the life cycle and also a way of trying
37:43
to educate people on On what it takes to to productionize
37:47
machine, learning models. Both from like the conversation,
37:50
we've been having is very much this tool Centric conversation.
37:54
Ml Flow versus cream flow, but it really does.
37:57
It is, it is not just that it is not that.
38:00
That simple of, you know, which tool is best, it's not because
38:05
it depends on you as an organization, you know, maybe
38:08
you're not even really an organization.
38:09
Maybe, you're just a ragtag band of to data scientists, it's
38:12
looking to do something really special and that's It's going to
38:15
be very different, from a team of 20 engineers.
38:19
And so you your tool choices will be dependent on your your
38:26
team, the skills they have. And also, you know, there you
38:30
have to bear in mind cost. Like I think that is a, that
38:33
does become a factor. And so the value chain is geared
38:38
at trying to educate people what it takes to care and I usually
38:43
have a conversation around maturity.
38:45
So like where are you at this at each of these stages and that
38:48
kind of helps me assess what, what they need to get to reach
38:54
the goal that we've kind of uncovered together or we talked
38:57
about man. Yeah, I love this value chain
39:02
and the way that you break it down, I thought it was so
39:05
insightful. How it's and for those that
39:07
aren't looking at it and you're listening in the future in
39:10
podcast land, he on the left, he has collaborate and then moving
39:15
towards the right. It goes from collaborate to data
39:18
science data pipe management Model Management hosting and
39:25
down each one of these columns he just goes into that a bit
39:29
more in depth and it's very much a way to self-analyze.
39:33
Hey, where are we at right now with In each one of these.
39:37
So, I was wondering first off, what made you choose each one of
39:42
these topics is this just things that you've been seeing time and
39:47
time again, as you go around and you're Consulting with different
39:50
groups. Yeah.
39:52
Actually let me share it and see if.
39:56
Yeah, that could be everybody. Can you see it?
39:58
Yeah, all right great. So why did I why did I choose
40:03
this? It came from a lot of experience
40:06
working with different clients in various different capacities.
40:11
So my role at Serbian has spanned a variety of different
40:16
things. There's there's been points
40:18
where I was very much focused on deploying and installing Hadoop
40:24
systems like hortonworks Cloudera and spinning up.
40:28
Those ecosystems in order for people to perform analytics and
40:32
data science activities and that whole experience Perience was
40:35
very informative around. These like the step of
40:39
collaboration and like, how data science as an activity unfolds
40:43
because you could, I would kind of sat to the side of analytics
40:46
teams as more in an infrastructure team at that
40:48
time. But you cannot kind of see how
40:51
like you know, infrastructure choices really impact analytics
40:56
workflows. And so that coupled with
41:00
experience in doing data warehousing along with
41:04
developing. Ml models and putting them into
41:08
production with other clients. Yeah.
41:10
Is it was kind of a it was that experience that helped me form
41:14
this and I guess my formative years and and and business
41:19
school kind of made me think about value, chains and data and
41:23
its kind of came together. It seems like a useful tool to
41:26
talk about the stuff. Yeah.
41:28
And the way your mind thinks in Japanese probably also helped
41:31
that may be so So the when I looked at it it really reminded
41:39
me of another guest we had on here Phil winter and she talked
41:43
about the hierarchy of ml Ops needs this.
41:47
Obviously it goes a bit more from end-to-end like ml Ops is
41:52
just one of these columns it jumps in there.
41:55
On the I think it's the Model Management portion of it and
42:00
that. But yeah I thought it was it was
42:03
really insightful and I encourage anyone who is Out
42:05
there listening to check it out. It's either in the chat right
42:09
now, or in the show notes, if you're listening in the future.
42:13
So, the other thing that I wanted to ask about when it
42:18
comes to this actually, before I jump to that, I know there's a
42:21
question in the chat that I wanted to get to, and it's from
42:24
Jose. He's asking, how do you handle
42:27
code reproducibility including dependencies when it comes to ml
42:33
flow? And He said if need be, he can
42:36
talk about his specific use case, if you need a bit more
42:39
color around that. Yeah, sure.
42:42
So when it's specifically, when you're talking about the code
42:44
itself ml flow, it doesn't handle the code.
42:48
It is very much separate and divorce from that.
42:51
With that in mind, I'll always share my screen, but I'll take
42:55
you to the different slide, which I think is better for
42:59
addressing this point. So, again, not Seeing any kind
43:06
of tools are. It's just a slide that I had to
43:08
kind of explain some of the work last year, but you talked about
43:14
code reproducibility, you know, it kind of sits in parallel to
43:19
the what I would call model reproducibility or Model
43:23
Management. And you know Model Management is
43:26
going to largely depend on, you know, your data your model that
43:30
you train the code that you push and with ML flow.
43:33
It look it works like logging so So you integrate ml flow into
43:38
your code and you log your models, your metrics are
43:42
parameters back to the ml flow server and from their stores.
43:47
Some of those artifacts especially if it's if it's big
43:49
files in cloud storage for example and so now I have to
43:55
have more color yet. So thanks iron.
43:58
Yeah I mean I love these details so basically I have a
44:03
scikit-learn rate so What I story is a pickle model and then
44:08
I also add the logic itself, I used to generate that model.
44:14
So in up the prediction stage I can just let's say up and that
44:21
code to the python path. So it's available however they
44:25
what happens with dependencies because seems that everything is
44:31
assigned to be to be having like one Environment per model and I
44:38
share an environment across many different models.
44:42
So if they different models have different dependencies like
44:46
scikit-learn, no point, nineteen.
44:49
And another one, no point with the eye, then I have to resort
44:53
to these condoms that you can save with Emma flow.
44:57
But then again, I didn't need to have one environment for that
45:01
model specifically. And I find it is a challenge for
45:05
People like in my case that have one environment and many
45:10
different models and cannot resort to let's say one
45:14
container where you can load any environment per model.
45:19
All right, so in the case of your dependencies, you're
45:27
talking about, could you give me a little bit more detail and
45:30
like what dependency might be? For example, s yarn, right?
45:33
So excited learning is used to generate the model because we
45:37
can use their, it's a algorithms.
45:43
However when you need to let's say make a prediction, right?
45:48
You You need to make use of that code, that you're used to
45:53
generate the model that also this stuff dependencies like the
45:58
second learner algorithms. So if you load a model
46:03
applicable file, right? That used scikit-learn no point
46:11
nineteen. But the, the scikit-learn that
46:14
you have in this system is an 0.23.
46:18
You have the issue of not being able to run that model because
46:21
the dependencies have gone too far.
46:27
I'm, I don't know, ya know. So, you need to be able to
46:31
replicate your environment that you train on needs to be
46:34
essentially the same or compatible with the one that you
46:37
use for prediction. And so, yeah, when you're using
46:42
a mail flow, one way of doing that so prediction and you can,
46:46
you can create containers and that Integrates with them L
46:49
flow, that's one way of encouraging.
46:51
It is to do containers or if you're in a situation where you
46:55
can use services like Sage maker or as yours ml service.
47:01
You can you can push models from ml flow to those services and
47:06
that puts them behind apis that you can query.
47:09
Or, you know, you could kind of do a more ad hoc way, which is
47:13
what I had it up doing when I was in, when I was using gcp.
47:17
And I just pushed the My code. I ran my code using AI platform
47:23
and then that logs artifacts in parameters, back to ml flow.
47:28
And then I was able to the model that I saved in ml flow and push
47:33
that onto a, i platform predict. Yeah.
47:37
And yea, but yeah, I'll to hijack the the whole thing but
47:42
it's tricky problem to make is then you need to keep like even
47:46
if we push it to search me. Anchor for prediction, you then
47:51
need to keep track of each of these end points per customer
47:55
for example. Yeah, it becomes a bit tricky to
48:00
manage, let's say, yes, your dependency managing
48:03
dependencies. At if in this space it is it's
48:06
critically important because it's a difference between like
48:09
your model being put in production and breaking or not
48:12
and it is tricky and you know, you can see instances where
48:18
Certain Services have been upgraded and that can kind of
48:22
cause cause breakage as well, you know.
48:25
So it's a, it's a game that we all have to play in ml Ops.
48:30
You gotta stuff to stay on top of it, I guess.
48:33
Yeah, it was. Thank you guys so much and God
48:36
because now something that I think you're definitely not the
48:39
only one who wants to know about that and it's something that I
48:43
had thought about, but couldn't properly articulate.
48:46
So I appreciate you jumping in and asking that question.
48:48
Question. I see there's a few more
48:50
questions in the chat and I know that we don't have that much
48:54
time left. So I'm going to Rapid Fire, here
48:56
we go. How much responsibility does an
49:00
ml engineer have when set with setting up ML flow for
49:04
experiment? Tracking it seems like a data
49:06
scientist can already use it easily enough.
49:11
Yeah, that actually points to one of the things that I really
49:14
love about ml flow. Is it is simple enough, that an
49:17
individual data scientist could use it.
49:19
And I've definitely talk to friends who, you know, they're
49:23
in Academia and they work by themselves.
49:25
And I've introduced it to him and I love it.
49:27
And, you know, they're not, they're not working as an
49:29
organization. But I would say, the value of
49:32
like an ml engineer. In the case of, for example, ml
49:36
flow, you know, they can set it up into an environment.
49:41
Are it's it's more robust. Its robust to change, its robust
49:47
to the dependencies that you might need.
49:49
So back to the last question, you know, dependencies and
49:53
environments and consistency in that manner is really important
49:56
and ml engineers and devops Engineers can help in that
50:01
regard. They can be very familiar in
50:04
that space when it comes to ml flow, you know, if especially if
50:08
you're using in the cloud and you're storing artifacts.
50:11
In cloud storage and blob storage.
50:14
You'll have an ml flows server that sits on like compute engine
50:17
or easy to instance. And when you're, when it's
50:21
sitting there on that instance, you want to make sure that it's
50:24
up and running and it's maintainable and so intelligent.
50:30
That's another rule for ML engineer or another task, G ml
50:33
engineer, devops person awesome. So to meet those asking, great
50:38
question here, They're saying I've got a question about
50:43
maturity of platforms like ml flow, which they use a lot.
50:49
There's a lot of new and interesting stuff out there.
50:52
How can we judge the maturity level of an ml Ops product?
50:57
That would like to put into production outside of the value
51:00
chain, that Byron talks about, what does it mean for a product
51:03
to be stable enough to consider To be stable enough to consider.
51:09
I think this goes back to understanding your use case and
51:12
understanding what you need from the product itself and then you
51:17
go out and you try and break it. I think is what it really gets
51:21
down to if you're there's definitely for.
51:26
For example there's a product at a certain point in time this was
51:30
a while back and I would need to Circle back to it to know if
51:32
it's still the case, but there were certain issues.
51:37
Are in ml flow. For example, there was some
51:42
level of setting up containers and in particular, in my
51:47
instance, using it on Google Cloud platform where it was
51:51
listed as experimental and I would have to go back to
51:53
remember the exact details of this.
51:55
So it's just anecdotal in that instance, you know, I needed
52:02
something and and I went out and tried to see if it would work
52:06
and you Just test and see if I um, it's mature enough because a
52:10
lot of these products, they're they're running very quickly,
52:13
they're developing very quickly. And so there's there, there are
52:16
aspects of them that their, you're lacking in some way.
52:19
Or maybe they don't have what you need.
52:21
So, that goes back down to to us as practitioners to know, our
52:26
use case inside and out and understand what their
52:28
requirements are. And then go out and validate
52:32
what is the right case and be prepared to build.
52:34
I think you should always be prepared to build because The
52:37
reality is ml, op spaces is still.
52:41
I wouldn't describe it in its infancy.
52:43
Maybe that's not exactly correct.
52:46
But maybe that is different. Different.
52:48
People will have different opinions.
52:50
I think it's still young enough that it's not.
52:53
You're not going to get everything you need out of the
52:55
box and so, yeah, be prepared to build.
52:58
Yeah, it's by no means Plug and Play quite yeah.
53:01
No, no. And, well, depends on the use
53:04
case, you know, Yeah. So Chris is asking one here.
53:08
He's saying, could you run ml flow on something like Cloud
53:12
run? Continue a cloud run, you should
53:17
be able to. Yeah, I mean anywhere you can
53:20
execute Python and if that environment can reach your ml
53:28
flow server, you should be able to run a mail flow.
53:33
Awesome. So if anyone has any last
53:35
comments or questions now, is the time it's now or never.
53:40
I could sit here and talk to you all day Byron.
53:43
You're very informative and you have helped us all so much.
53:47
I know that people loved it. Thank you so much.
53:50
This has Ramen me. Yeah, this has been great.
53:53
Another shout-out to our cheese sponsors.
53:56
This is brought to you by was If you're in the supermarket, Chow
54:07
cheese will be supporting ml Ops and everything.
54:11
Ml flow verse Cube flow and all jokes aside, I really appreciate
54:17
this. And if you all have any other
54:21
questions Byron, best place to reach out and connect this
54:25
place? Yeah, best place to reach.
54:26
Me is on LinkedIn. I post on medium every once in a
54:30
while. But yeah, if you have questions,
54:31
I'm happy to take them. And we've done.
54:34
Yeah. Great.
54:35
And maybe we can convince you to jump into our own.
54:39
Oh you're already in it? Your yeah I am an actress that
54:41
you are and yeah and happy to answer questions there as well.
54:45
That's your preferred way of reaching out to me.
54:47
Very cool. Yeah.
54:48
So if anyone is not in it, jump into our slack.
54:51
We've got all kinds of good stuff going on there and we I'll
54:55
send you a follow-up. Everyone that came here a
54:58
follow-up email with all the highlights and the URL to the
55:04
recording. So if you want to go back and
55:06
check this out, you can. And other than that, this has
55:09
been awesome. Thank you so much.
55:11
I'll see you in slack. Okay everyone, thank you guys.