Latest / Agentic Conversations (formally mlops.community) / Why and When to Use Kubeflow for MLOps // Ryan Russon // Coffee Sessions #107

Transcript

Why and When to Use Kubeflow for MLOps // Ryan Russon // Coffee Sessions #107 · Agentic Conversations (formally mlops.community)

0:00
Well, I'm actually not a coffee drinker.
0:02
So, but I do do enjoy a nice little iced chai.
0:06
But all right, so for those who do not know, we had Ryan Russo
0:13
on today, let me just make sure that I'm rousan.
0:17
I don't know if he pronounces it through, so I just made that up.
0:20
So Ryan is now a consultant he was in the Navy.
0:25
He got into ml, he got into statistics, he got into all
0:28
kinds of physics. All that good stuff.
0:31
And then he went and he worked as a data scientist, but has
0:34
always been an engineer at heart.
0:36
He told us and now he's in Consulting.
0:40
And the reason that I wanted to bring him on here is because
0:43
right from the get-go, when I started talking to him, he I was
0:45
like, hey, I'd love to have you on the podcast.
0:47
Maybe there's something we can talk about, he was like, I love
0:50
Cube. So we should talk about that.
0:52
It's like what? No way.
0:54
You're like one of the few. Well, one of the Brave and he
0:59
actually is, He literally he's bullish on cute though, and it's
1:03
awesome to hear why that is because I feel like it's a
1:06
contrarian position and so I wanted to give him the platform
1:10
to talk about why he loves it, what he thought about it, what
1:13
were some takeaways that you had, man?
1:15
Yeah, I mean, I think that it's really great to bring them on as
1:19
they, we spend a lot of time talking about cute flow and and
1:24
some of the downsides and some of the good sides and I think
1:27
that in his situation he was Was very level-headed.
1:31
I mean, even though he advertised himself as this very
1:35
staunch people Advocate and he does love the tulle.
1:38
I like that. He mentioned that which was a
1:41
very good Consulting practice, is it really depends right at
1:44
kubernetes and cute. Flow is not the solution for
1:46
every organization, every single team, and every size.
1:49
And there's so many factors that come into play and for many
1:53
organizations in might make more sense to pick an alternative
1:56
tool. So kubernetes is not for
1:58
everyone even though It does offer a lot of benefits, but any
2:05
team, any organization that's going to be listening to this.
2:08
The main takeaway should not be that.
2:09
Yes, you have to go all in on Cube flow.
2:12
You just, you need to do your research.
2:13
You need to think about where you are in your own progression,
2:16
machine learning and blocks, and consider the options.
2:19
Cute flow can be a very good tool for you, but make sure you
2:22
have the resources. Make sure you have the time.
2:24
Make sure you have the knowledge to really invest in a lot of
2:28
upfront Investments, make it work for you.
2:30
Yep. How mature is your team?
2:32
Exactly. And there was a great question
2:34
that you asked, I really like that.
2:36
You wanted to know around abstractions and like do we have
2:41
too many abstractions. If we put more abstractions on
2:44
it, ends up creating this leaky bucket type scenario.
2:47
I loved hearing you to look at things and just talk through
2:53
that whole challenge round abstractions.
2:56
Let's jump into this conversation.
2:58
The last thing that I will say Is, we are looking for people to
3:02
host or organize local, meetups mihael is organizing the local
3:07
meetups in San Francisco. He's running strong.
3:10
And we're looking for people in Austin, Texas, Denver, Colorado,
3:14
and Seattle Washington because I need an excuse to visit, all
3:19
these places and what better excuse than I gotta go?
3:23
Because there's the local meet up happening.
3:25
That's what we're thinking, and there's also great.
3:29
Talent and people doing a lot of cool stuff in these areas.
3:33
I really would like to have that out for the world to see and get
3:38
back in person. Let's let's do it.
3:41
So worldwide worldwide. Yeah, worldwide Global.
3:44
Dude. We are we are we've got an India
3:46
chapter. I'm talking with people in
3:48
Australia right now to do that. I mean, we've got Europe pretty
3:52
covered and Tel Aviv. There's a huge community in Tel.
3:57
Aviv that is wild. To I mean yeah, let's do it.
4:02
Viva La revolucion, that's it. Ml Ops all over baby.
4:07
All right, so that's all we got. Let's jump into this
4:10
conversation with Ryan and all about Q fo probably start.
4:21
I just kind of want to get into it which probably start with
4:24
like what are you thinking? How are you bullish on Cube
4:28
flow? I don't understand you at all.
4:30
Give me your reasons. Like defend yourself here
4:33
because this I feel like it's gonna be a long conversation on
4:36
me. Just going like this a lot.
4:39
And for those just listening, I'm shaking my hands like as if
4:42
I were Italian and Ryan defending too slow.
4:45
So please like break it down for us.
4:46
Give us a little context here. All right, here we go in the hot
4:50
seat for this one. So I think, you know, I'll start
4:54
off with this thinking that this is, this is a talk, I gave
4:58
recently to at machine learning meet up, but you know, cute flow
5:02
is not the right choice for everybody, right?
5:05
I'll just stay right there, like, and, and that's something
5:07
that I've thought a lot about in terms of when I'm working with
5:11
either clients or and I've worked at previous companies,
5:14
you know what is the right tool for the job and you flow works
5:17
well, Q flowers. Well, when you're already in a
5:21
kubernetes forward kind of team or your company is already
5:25
kubernetes forward. I think that there because it is
5:30
a kubernetes for it was built right for kubernetes.
5:34
I think that that is what makes it appealing in that
5:37
environment. I've seen it where small teams
5:43
have tried to implement it or have implemented it and it has
5:47
become that nightmare Why, why did I do this to myself and in
5:53
this? Because that's that kubernetes
5:55
just coming over coming over. That, hump, understanding that
5:59
learning curve of actually. Okay.
6:00
What a, why do I want to do this on kubernetes?
6:03
What does it provide for me? I don't know if that's kind of
6:08
what the thing that you guys have heard or what you've talked
6:10
around in the community. That's, that's the short answer.
6:14
We can definitely get more into that.
6:15
I get all out more kind of background there, but that's the
6:18
short answer. Answer really quickly for the
6:25
people that are tuning in and maybe don't have as much
6:27
familiarity with Cube flow. And you burn at ease.
6:30
Where does cute full kind of sit in the ecosystem of ml tooling?
6:34
And, you know why should, yeah, why should I practice or even
6:37
care. Right.
6:39
So that's, that's great question.
6:40
So like what Q flow brings is? Like, I was reviewing actually
6:46
for use in presentations yesterday with one Are your
6:49
consultants and talking about like where does q flow fit in
6:51
like so if you look at like vertex AI or Sage maker or as
6:57
your ml, right? These are vendor provided tools
7:00
for you on the cloud and the cloud to be able to have exploit
7:03
you know notebooks for exploration.
7:05
Have orchestration, tools, have serving tools, right?
7:09
Cube flow in a lot of ways fits that same mold, right?
7:13
It gives you tools for exploration.
7:15
Gives you tools for pipeline orchestration gives you tools
7:17
for Serving and it does it in an open source platform.
7:22
So the heat the big Advantage here is if I'm a company or I'm
7:26
a team and I don't want vendor lock-in I don't want to go all
7:29
in on vertex. I don't want to go all in on
7:30
stage maker, right? I want to do something.
7:33
I want something I want to be agile.
7:34
I want to be flexible. I want to be able to move from
7:37
one Cloud to the next and not have to necessarily change the
7:41
entire experience that my data scientists are used to write Q
7:45
flows. A good option there, right?
7:46
So if it's that same now granted Is it is mature.
7:50
Some of those tools. I mean you know it's an open
7:52
source project, right. It's still maturing is it's got
7:54
a lot of velocity though, right? The has come a long way, in a
7:57
very short time, even with like, 1.6 coming out later this month
8:01
at some point. So, it really just feels in my
8:04
mind. It fills that Gap, it's like
8:06
okay, it gives you the tools to be able to get off the ground
8:09
with you know, Exploration orchestration with serving in an
8:14
environment that is customizable to all right?
8:17
I don't have to, I don't know. Sorry I can hook it up to
8:19
whatever Cloud I want to. I can I can use S3 as my back
8:21
and I used UCS has my back and I can use it orchestrate jobs in
8:24
bigquery and and I can use it to run it.
8:27
You know. I can work with my local gpus
8:29
that I have on site if there, if that's the thing, right?
8:31
So you can it gives you just that flexibility to really kind
8:35
of connect a lot of your tools in a very in a way that I don't
8:39
have to be. I did a side is enough to be a
8:40
culinary expert per se to know, understand how to use it.
8:44
They'll use, you know, python DSL to write their Pipeline
8:47
orchestration and that'll make them, you know, they'll
8:49
hopefully feel comfortable with something like that.
8:52
You know, by the end of the day, it allows you to unlock that
8:53
power of kubernetes in a way that I think makes it more
8:55
accessible, that that's perfect. And I probably should have led
9:01
with that one and some people with the watches of switch, the
9:05
order of you are into it. You were excited.
9:09
I was a little trigger happy to be honest.
9:11
I got a little trigger-happy I just Really wanted to talk to
9:15
you. I mean, to be fair, this
9:17
conversation with you. I've been excited about like,
9:21
talking to you about this because I think cute flow.
9:23
What happened? When I first learned about it
9:26
back in, this is probably like, March or February 20, 20, and it
9:31
was before, like, 1.0 came out even and it was right when we
9:37
were starting the community. We had this guy, Josh bottom
9:41
from Rick do. Come on and talk about cute
9:43
though. And he just showed everything
9:47
and all of its capabilities what it kind of where they were
9:51
looking to go and what they were trying to do with it.
9:54
And I was blown away and I thought and I was working for a
9:57
company at that time, that was doing something similar in the
10:00
space, but without Cube flow and I just thought like, oh my God,
10:04
why would anybody buy us when they got this open source
10:07
project Cube flow? And then the next thing that I
10:11
heard about it because the company I was working for they
10:13
went Business and our CEO. Who's an incredible person, Luke
10:18
Marsden. He is very deep in the devops
10:21
field. He told me.
10:22
Yeah, I tried to install Cube flow on my computer and it
10:26
crashed twice. So then that kind of was the red
10:32
flag that made me realize like, oh, maybe it's not all as
10:35
flowery and tie-dye, and perfect as it might was made to seem by
10:41
this vendor who came on and obviously has Has a stake in
10:45
being able to sell managed Cube flow I think.
10:49
And so from there it just kind of has been something that
10:54
always gets a lot of attention in the community.
10:57
And Cube flow itself is being used by such mature data
11:02
organizations. I think all over the world and
11:06
then you have the other side of things where it's like people
11:11
come in and they talk about how painful it is and how Cube was
11:14
not working. Why is this not working?
11:16
What's going on? Or you have the what you were
11:18
saying, the small teams and they're trying to stand up Cube
11:22
flow and just struggling and having a really hard time
11:25
because maybe they're not kubernetes native, and they
11:28
haven't even played around enough when kubernetes to
11:30
recognize, like, y, Cube flow is throwing these errors, or is it
11:36
a problem with Cube flow? Is it a problem with my Jama
11:39
files? And, and is it the kubernetes
11:41
problem? So, all that being said, Said
11:45
let's put a fork in it and we'll get back to it in a minute.
11:48
You talked about you flow having velocity.
11:51
I've undoubtedly seen that just from what I was saying.
11:55
Two years ago when like 1.0 came out and that was a huge bang for
12:00
me. It almost feels like though that
12:03
it had a lot more velocity and a lot more hype 2 years ago than
12:08
it does now. But I want to hear what you
12:10
think and maybe that's just like the maturing in the stack Or
12:14
maybe it's new players have come out that make it easier.
12:17
Like, where is your head at around that?
12:21
Yeah, that's definitely true to. I agree with that.
12:23
Like Q flow was a lot more, you know, of a big deal.
12:27
I say, two years ago, right? Extra heard a lot more about it.
12:30
I wonder and I have my own theories of why that might be
12:34
right. And we have to understand kind
12:35
of understand where we're cute Flow came from, right?
12:37
So keep those kind of born out of Google trying to do, T it to
12:41
tensorflow Extended right on kubernetes.
12:43
That's kind of where Are that whole project kind of came from.
12:46
And so, I think that in a lot of ways, Google's been very bullish
12:49
On Cue flow, especially a couple years ago, but with the Advent
12:53
of like, the latest vertex, AI, which happened, you know, later
12:56
last year, I think that, you know, under the hood vertex is
13:02
using Hue, flu-like, especially useful, if you vertex pipelines
13:04
is using Q flow pipe lines under the hood, right?
13:07
To run their orchestration jobs. So, I think in a lot of ways,
13:09
there's kind of almost, I don't know if it's a, there's a split,
13:11
almost two, versions of coupons being developed right now.
13:14
The parallel, right? There's like the V2 version,
13:16
they call it, right, which is eventually coming to to kind of
13:19
the open source Community more, it's available now, but it's not
13:24
perfected yet, right? That's what's, that's kind of
13:26
the Google targeted and stay, and then there's, you know, the
13:30
classic version 10 call it. So, I think a lot of the
13:33
Velocity in terms of well, not the velocity of the, it's the
13:37
hype has been from Google kind of pulling away from the open
13:40
source side because they want obviously people to use their
13:43
their managed Offering right? They want you to go on vertex
13:46
and so now you've got companies like a rick toe like maybe even
13:49
canonical stepping in and other and even like AWS actually.
13:53
Now kind of bring it up more to the Forefront.
13:55
It's like, okay, now it's truly open source.
13:57
Let's try to like, let's bring it up a little bit immature, and
14:01
I think it's mature quite a bit. I think that's also to why maybe
14:03
some of the hype is gone is that it is now a mature product in
14:06
larger data. Organizations are actually
14:08
they're using it and they're seeing the benefits of it.
14:12
But that's, that's those are my kind of my theories.
14:14
What I've seen in the space. Yeah, I mean it's when I think
14:18
of the big debates in machine learning, you know, the ones
14:22
that can that can stop the music at any party.
14:25
There's a few that come to mind. One is sql-based workflows.
14:29
Like, should everything go shoot, you sequel to do machine
14:31
learning other. One is notebooks and production.
14:34
Is that something that we should be doing?
14:36
Moving forward? The third one is Q Plus 0 or not
14:39
to cube flow and and I feel like those even as the, the fields
14:43
matured Those those three and certainly the kubernetes
14:47
question have remained in people the Forefront of people's minds.
14:52
How do we make this on rest better?
14:54
I mean, if we assume that this is the future, right?
14:57
If we assume that that cute flows here to stay, how do we
15:00
make this better? How do we make it that more
15:02
people are in the yay Camp of kubernetes and Q flow rather
15:05
than the Nay. Yeah.
15:08
Well, I'm glad around talking about notebooks and production,
15:10
because I'm very opinionated about that one and probably,
15:13
anyway, that's always been a key subjects to me.
15:16
But Lon I'm definitely Camp of no to that.
15:19
But like, when it comes to like the honor and 4q flow, that's I
15:23
mean, that's great and I thought and features you did happen.
15:25
The question. Think I didn't quite answer is
15:26
like the other players in the space, right?
15:28
That's made a little easier and even come back that too because
15:30
there definitely are some good Alternatives my mind, right?
15:33
But it comes to making that on-ramp easier.
15:36
I think. I like the experience.
15:38
We said that your boss that or the CEO said he'd installed
15:41
coupon and crashed his computer, right?
15:43
I mean, that is true. If you try to stall full cube
15:46
flow on your laptop, it's just, it's just not going to work.
15:49
Unless you've got a pretty beefy laptop, right?
15:52
The good thing is, is, you know, in my, for my perspective, I've
15:55
tried to help make that a little bit easier for people who care
15:59
and want to start out, I've created a whole like GitHub repo
16:02
about how to, you know, get started with cute flow with eks,
16:05
or GCS, or even using canonical Chuckles micro case, right?
16:08
So you want to put my Croquettes on your boot to installation and
16:12
run it locally. There's these light versions
16:14
because you can get started right?
16:15
So you at least understand the basics and see if this is
16:18
something you want to do. So an honor at perspective, I
16:20
think that there's there's definitely better tools right
16:23
now that have ever existed. I mean, you can run kubernetes
16:27
locally now easier than ever right, whether it's for micro
16:31
case or even just using like Rancher desktop, right?
16:34
You can guess we get this list of careers going and get started
16:36
with some Thing to make the honor of easy though.
16:39
I think it's just that's that there's that kubernetes humps.
16:42
You've got to come over. So there's got to be someone on
16:45
your team who has an interest in moving their or is familiar with
16:49
it already have a platform team, the student of things that helps
16:52
the here's the kubernetes part of this, I get the data science
16:56
side of them, a lopsided of it. How do we work together to make
16:59
this work? I don't know if the honor is
17:00
going to be ever super easy, to be honest.
17:03
As long as this thing sits on kubernetes, it could be It's
17:07
getting better for sure, in terms of the features.
17:10
But I still think there's a lot of challenges that smaller teams
17:14
have to overcome security is a big one, right?
17:16
Depending on who you're working with or who you're working for.
17:20
There are definitely security challenges that have to be
17:23
overcome. And that that's a hard one.
17:28
Like networking equipment is right and making sure certain
17:31
that if you're going to run like cats have jobs right that the
17:34
side cars run and my side cars are talking about, you know, You
17:37
know containers the Run next to your main container that's part
17:40
of your job they'll have to might have to run his route.
17:42
Well maybe that's not allowed on your cluster so there's things
17:45
you have to, I don't know. So I really I answered your
17:47
question, okay? Like saying like it's actually
17:48
it is complicated, it's not easy.
17:50
The on-ramp is not straightforward, there's
17:52
definitely things to overcome. The benefits are huge once you
17:56
get there but there's definitely not an easy way.
18:00
There is easy ways to start a sort of get started, but they're
18:04
not going to be full production Worthy.
18:07
Ways to start in my opinion, I don't know what that feels like.
18:11
One of those things where you can see the promised land, if
18:15
you get there but you have to go over like Mordor, you know?
18:17
I mean you have to like the little cross.
18:19
Like the kids have fired. Yeah.
18:21
Yeah, you got attraction Mordor to get there.
18:23
Yeah exactly. So what are your thoughts on one
18:28
of the trends that has also become popular?
18:30
I think, as the as the kubernetes ecosystem that cute
18:34
blue consistently matured, he is this idea of Like Rappers.
18:37
Around the tooling right? Like, you know, maybe we don't
18:39
want to, I mean q flow in a sense is the wraparound
18:42
kubernetes, right? So there's already an
18:43
abstraction layer around that but then there's also
18:46
increasingly new tools that are also wrapping around, you know,
18:50
some variant of cute flow or maybe even just varying the
18:53
kubernetes at the wrapping around.
18:55
So what comes to mind is like, Selden as the ones, right?
18:58
He's kubernetes native types of platforms that are trying to
19:01
make the owner a little bit easier.
19:03
What are your thoughts on that? As a trend moving forward, I
19:06
mean it's I promise there is that is that may be one way
19:09
forward? I think so.
19:10
I mean, if you look at like, let's just take you for
19:12
pipelines, for example, right? At the end of the day, it's our
19:15
go work. Flows under that hood, right?
19:17
That's running. It's a rapper on top of already,
19:20
a tool that you can use impurities which again are go
19:23
workflows is designed to help you make your life easier for
19:25
any kind of like private or any kind of orchestration regardless
19:28
of what it is, right? Which also makes a great fit for
19:31
machine learning because you can really just run anything,
19:34
anything you containerize, you can do it and you can
19:36
orchestrate it. I think definitely is that is
19:38
going to be an important Trend. I think if you look kubernetes
19:41
itself as an abstraction, right? Like everything's like the
19:43
layers of abstraction our absurd, it's some degree and but
19:48
but it's kind of a necessary step.
19:50
I feel like to really enable technology, any kind of
19:52
technology to be accessible to any team.
19:57
And so, I do think it's going to still continue and I still think
19:59
we're gonna see these rappers upon rappers and it's definitely
20:02
prolific in the ml space. Like, how do we make this easier
20:05
for data scientists is some way? We've tried to it kind of feels
20:08
weird with somebody who's kind of try to dumb down at hate, to
20:11
use that word. But I kind of dumb down or water
20:12
down everything is much as we can.
20:14
So that a data scientist can use it, which seems kind of bizarre
20:18
in my mind. I feel like, you know, that
20:20
there's definitely when it comes to tooling.
20:22
If you're going to use the tools, you should try to
20:24
understand it at the best, you know.
20:25
It score with those tools, provide for you and how to how
20:27
they work in the whole entire stack even if there is
20:29
abstractions there. But IDs, I do think that from an
20:33
extract from like a making the on-ramp easier.
20:35
We're going to see more of these abstractions.
20:37
Um, kubernetes is not going to go away anytime soon and it's
20:40
definitely, I think I feel like it's here to stay in a lot of
20:42
ways, so there's just going to be more and more of these
20:44
layered abstractions. I feel like to to make that
20:47
transition to, whatever you're using easier.
20:49
Just one follow-up here to play Devil's Advocate because there
20:57
is the argument to make around. Yes, add abstraction layers and
21:00
water them down. As you go higher up, the
21:02
abstraction layers, there's another argument, which is that
21:05
you know, as you As you add more abstraction layers, Things
21:10
become leaky, right? There's likely abstractions.
21:12
And and in some sense they may not always actually help, right?
21:16
Because then you get some error, which is embedded three levels
21:20
down and so there's this weird Russian doll effect that's
21:23
happening here, right? One alternative universe is one
21:27
where we just say screw it. We're starting from scratch,
21:30
like, let's build. We're going all the way from the
21:32
beginning and we're no more seven layers, deep of
21:36
kubernetes. We're Going to try and build
21:37
everything. The way we should have done it
21:40
from the from the from the get-go.
21:42
I mean is that one possibility or are we just too to deepen at
21:45
this point? That is not the way we should
21:47
have done it on Coleridge and yeah I think about Fortran back
21:52
in for Chad we've gone too far. Already pythons out.
21:56
We're going right back to ya though.
21:57
I mean like yes I think that I think that mature organizations
22:03
were I've seen kind of step back to that even it's like okay.
22:07
Because too many abstractions you lose control and you're
22:10
leaking, right? And when you said that like the
22:13
air is I think oppai spark. There you go.
22:14
You know how many of you know exactly like all the job errors
22:17
you have to go troubleshoot just to work.
22:18
Making pie smart job work right? Because your anyway but yeah
22:21
like user gives to it to a point where you're too far.
22:25
You're too far away from what's actually happening to where
22:28
you're running. So you know you got this this
22:29
Gap here and it's problematic. So I think that it works again.
22:33
If your it was refract, light is going to work.
22:35
If you're doing small scale things, Using it for quick,
22:38
iterations Quicks, permutations things like that and Q slope
22:41
specifically in this case. But like it comes a point where
22:43
it's like, if I really want to actually use this as like a
22:46
production Building train system, or I want to actually
22:49
take K serve and actually I want it to be my production serving
22:53
system like every day's, I mean you have to, again that stack
22:57
the understanding. What's in that stack matters and
22:59
getting too far away from that is is problematic.
23:02
I think it because it you just lose control and you've lose
23:05
sight on what's Happening and you could have it.
23:07
Issues that you're not even aware of because you so I yeah.
23:11
Do you feel like there's that's probably the reason why I think
23:16
there's more of a trend of people going with our go
23:19
workflows instead of cube flow Pipelines.
23:24
I think so. I think that, if you look at the
23:27
community, right, like, let's see, who is the end user for
23:29
cube flow, right? A machine learning engineering
23:32
data scientist baby, right? That's, that's who's the
23:34
end-user here, who's working on the platform that that will sit
23:38
on? Well, these are the s.res.
23:40
These are devops people argues feels more comfortable already,
23:43
right? Argo already fits the moles, it
23:46
makes sense. So if I can tell my data
23:48
scientists now you got to orchestrate all your jobs on our
23:50
go, because that is how we do things.
23:53
Hear that, that's the influence there.
23:56
It says, hey, this is my platform.
23:57
You do it this way, right? And so, I think there is
24:01
definitely this kind of convincing that has to happen,
24:03
where we both kind of understand we have to work together at this
24:08
point to make this work and this is, that is ml Ops in a
24:11
nutshell, right? Like, how do all of these
24:13
disparate types of Talent come together to work on?
24:17
Something that everybody can understand and can use?
24:20
And it's not too hard for one or two hard for the other.
24:23
There's give and take, right? So I think that, you know,
24:26
devops either going to get up higher for these Ponies are go
24:28
because it's, that's, that's the, I understand it.
24:31
I get it. It makes sense.
24:32
I don't want to write things like a python, did you know?
24:34
I gotta want to have to compile. That's kind of a weird.
24:37
I think it's strange word fees that but we'll compile my, this
24:40
python code into essentially an Argo workflow that I can get
24:42
deployed, right? I think that, that's why you'll
24:46
see it, I think because platform heels up a lot of they carry a
24:50
lot of, you know, influence in what's You so that's probably
24:54
why. So there's this very almost.
24:57
There's an Enlightenment has to happen as to why why cute flow.
25:00
Why does it work? Where does it not work?
25:02
Or does it break down and how do we work together to make it work
25:06
for everybody? Yeah, I mean, especially one
25:09
thing that comes to mind is You mentioned, who is this tool
25:14
serving, right? In a sense, Machinery,
25:17
Engineers. Maybe you're even more
25:19
comfortable kubernetes than it than a citizen data scientist
25:22
would be. Yeah, it really feels like
25:23
there's still a pretty big gap to get those folks to a point
25:27
where they're super comfortable, orchestrating workflows,
25:30
orchestrating pipelines. And I wonder if people is really
25:34
the end state, for those types of folks, right?
25:37
Who don't want to deal with all this stuff, all the extra
25:40
overhead of Cube CTL and seeing what the pods are running, and
25:44
all these things, it's quite a bit.
25:46
I mean, when you, when you, when I first installed Cube flow and
25:48
I saw everything that came packaged it, I was like, holy
25:50
hell, what is all this stuff? I've where do I begin?
25:54
Yeah, like we get oh, it's just so much happening there.
25:57
Yeah, whereas I mean I think the general trend has been to try
26:01
and ease data scientists. The amount of touring the data
26:05
centers need to know, right? When you look at something like
26:07
meta flow, it's like oh yeah, just put a decorator on all the
26:10
functions you already writing and that's That's, that's all
26:12
you care about and and he right. You're right.
26:15
You're right. You're training, Loop,
26:17
independent of that. So I wonder if.
26:21
Did you think that that keep flow is, is the end state for
26:23
like a more citizen data scientists to use that?
26:27
I can use that term do. I definitely could questions do
26:31
I think it's the end state? Do we ever get to the end State?
26:36
As my question? I think it's like whenever will.
26:39
But do I feel like it Is a and state, it could be.
26:44
Again, there's like you said, like when you qu flow is not for
26:47
the faint of heart. But if you look at again, if you
26:49
look at the flip side of that, you look at how many things it
26:51
does install for you. That is all of the conveniences,
26:55
it's added for you. But the other way to look at
26:57
that, right? That's all the other stuff.
26:59
It's helping you manage that you didn't have to build in.
27:03
So yeah, meta flow. That's a great, great product,
27:05
they'll be there right with meta flow, kind of comparing,
27:08
contrasting from an orchestration perspective and
27:09
what is tracking for you. Right?
27:11
You Get a lot of the same thing meta flow gives you a lot.
27:13
What Q flow does from its artifacts, Infamous
27:15
orchestration. The in meta flow, just released
27:18
an ability to run this on Cuba and he's right with their with
27:20
their latest releases. So is that a better answer?
27:23
I don't know. It's so easy.
27:24
Our go under the hood. It's still an abstraction layer,
27:27
right. And it's not, it was never a
27:28
kubernetes first product, so, again, I go back to that
27:32
dichotomy is if your team and your company is kubernetes
27:35
forward. This is a good direction to go.
27:37
This is probably the right direction.
27:39
I'd say, if your team is not, if you just want to, Your small
27:42
team your Cloud native, maybe, but you want to make things a
27:46
little bit. You don't want to get into the
27:48
kubernetes world so much, then use the vendor.
27:50
Use the managed options. Use Metal Flow, use these other
27:53
tools that fit into that Paradigm.
27:55
But again, from the end State perspective, you float does
27:57
represent, I think one way of doing things and it represents a
28:00
way of doing things, that's flexible that I don't have to.
28:04
I'm not locked in, I go back to that.
28:06
I'm not, I don't how I can use to flow native u.s., I can use
28:09
it on Google Cloud. I can use on Azure.
28:11
I don't. It doesn't matter.
28:12
I can use it anywhere. And that does not necessarily
28:15
the case with some of these other, you know, with managed
28:18
offerings and I think even better flow, right?
28:19
I think it's still just a WS kind of directed and thinking
28:25
about like if Cube flow were to drop off the face of the Earth
28:31
and all memory of it is erased on like Men In Black.
28:36
Yeah, they just wipe your memory, love it.
28:40
What in your mind? And would you be using?
28:44
What are things that you think are in that end-to-end workflow?
28:50
That also look at kubernetes first or maybe not.
28:55
You just have had some experience with.
28:57
And you think these are very useful?
28:59
I understand like the easy way out here or the probably the
29:03
first choice would be those Sage maker vertex and as your ml are
29:09
there other ones that I'm not thinking about That's a great
29:14
question. Yeah, those are the easy
29:15
answers, right? I mean, what?
29:17
IBM Cloud? I'm not as familiar with that,
29:19
right. And so I won't speak too much
29:20
there, but you know, those are the big three players right
29:24
there. The in that case, from the AI
29:26
offerings you know, the a coupla was gone if we didn't have it,
29:31
right, what would he be doing it to me?
29:33
It's like okay. Now you're going back to I
29:36
probably use ml flow at this point right for tracking
29:40
artifacts and models and you know, kind of use that Suite of
29:43
Will's plus some poor castrator, right?
29:46
So that's that could be meta flow.
29:50
It could be air flow you know. But yeah those are that's a
29:54
great question. Is, is kubernetes native?
29:56
Then you're gonna be like okay? Well then that's why I go back
29:58
to our go. If that exists, right?
29:59
But hopefully that wasn't wiped out with the Q flow existence.
30:01
But, you know, you know, there's there's yeah, there's there's a
30:06
those are kind of my thoughts there, like q+ solves, a lot of
30:09
problems. And a stack perspective that I
30:12
don't have to think about. Out.
30:13
Maybe I've gotten complacent, some degree on what other things
30:17
I should always be thinking about just because again, all
30:19
the clients have been working with, it's just cute.
30:20
Flow has been kind of front center for them, so there's
30:24
definitely producers are plenty of them, but I'm like, thinking
30:28
to myself. It's like, which, what would it
30:31
be? I think, you know, there's
30:32
there's a lot of options. There's a ton.
30:34
Yeah. Even just watching the last meet
30:37
up, right, with Jakob? Yeah, that's As an example right
30:43
there, right of using it was using meta flow right to
30:49
orchestrate things prefect, there's that as well.
30:52
So, I mean, there's ways to there's definitely ways to do
30:54
it. You Flo just makes it easier.
30:56
Especially with the exploration piece, think the built-in
30:58
notebooks essentially, in the environment that you're also
31:00
going to be training on and things like that.
31:02
That makes it nice. But again.
31:04
Yeah. You gotta manage those
31:06
containers. You got to know some kubernetes
31:09
Cube cuddle becomes your friend. So yeah.
31:13
We all love that. Cute coat.
31:14
Oh, we love it. So did you have something Kyle?
31:19
Sorry man. Okay, no.
31:21
No, please please. It's all you.
31:23
Yeah. So I'm thinking about how I see
31:26
the ecosystem in Cube flow and especially when it comes to like
31:30
K serve and the maturity that K serve has reached and just the
31:36
amount of energy around that part of the project.
31:39
And then from what I hear around like Cube so Pipelines.
31:43
And around a lot of the politics where you've got Google involved
31:47
and they have their ways of wanting to do things or where
31:50
they want the product to go. Does that a ever like turn you
31:55
off and then be how do you feel about K serve and as you were
32:00
talking about it before like getting that to be your main
32:03
production serving platform. What do you see as like let's do
32:08
kind of going as a deep dive indicate serve because I for me
32:12
that's what I love that project out of the whole Cube so
32:16
ecosystem, that seems to be like the really cool project that is
32:20
got a lot of attention, a lot of love and they're doing things,
32:24
right? Yeah, I mean, first and
32:27
foremost, I'm glad to see that it kind of has broken off out of
32:30
KF serving to case Earth that has become its own.
32:35
Standalone project is still obviously part of the cube flow
32:37
umbrella, but it's now its own Standalone thing.
32:40
And I think a that is the right move.
32:43
Because when you look at serving, I mean, are you really
32:46
going to serve in the same environment that you're building
32:48
and training your models on with?
32:50
No you're not. You shouldn't be at least that's
32:52
a whole other problem in himself, right?
32:54
So that should be on a different cluster that should be separated
32:56
already. So I think that, you know, based
33:00
on that I'm glad to see cases of reach that maturity level and
33:04
the fact that it is recognized as it's a standalone product.
33:06
I think useful pipelines is also a standalone product, right?
33:09
Some degree. So you can you can pick and
33:11
choose out of the queue flow. Um, what you want to use, what
33:14
makes sense in case of makes it really easy on the serving front
33:18
to use some of these these great serving tools that have been,
33:21
you know, kind of wrapping around them and making life
33:23
easier. So I think that I am a huge fan
33:26
of that that direction that this we going.
33:29
And I think that's the right answer because it makes sense,
33:31
you're going to be serving in a different environment then when
33:33
you're orchestrating building a pipeline for training, right?
33:37
And so that's that's like my short answer to that.
33:41
You also just to go back to something you mentioned earlier,
33:44
you'd brought up a lot of the different orchestrators that one
33:47
can choose from like airflow professions that are up.
33:50
How do you think about that ecosystem and where Q Post-its
33:54
and all that and weighing the pros and cons?
33:57
I mean, is it is it interchangeable at this point or
33:59
does one really rise above the others or or does it depend on
34:03
the majority of the team, right? Certain size teams should be
34:06
using one because it is more small friendly or or As friendly
34:11
depending on which use case you have, that's a great question
34:15
and it's definitely something that I think a lot about often
34:19
in terms of like what does q flow pipeline provide for you.
34:22
Well, right, you are castration on kubernetes, what are the
34:26
advantages of kubernetes, right? Well, it's in very few places.
34:30
Can I stream together a desk job, plus a spark job, plus, any
34:35
gpu-based job and seamlessly is different components.
34:39
Orchestrate that all in the same Same environment.
34:42
So again for an advantage of this is a kubernetes Advantage.
34:44
I can run pretty much anything, right?
34:47
And look as long as there's an operator that's been created for
34:50
me or help you know to to actually stand my capabilities
34:53
on my cluster, I can piece together very complex, machine
34:58
learning pipelines. They're using very different
35:00
tooling using the right tools for each step in that pipeline
35:04
where that's not as easy, especially something like
35:05
airflow, right? That's not going to be that's
35:08
very much more kind of restricted to the Granted.
35:11
Now these these other orchestrators are coming out
35:14
with ways to do this on kubernetes.
35:15
So does it, does it help, you know, bring.
35:18
So then would argue, was just our go under the hood, that's
35:20
doing it. And that's true.
35:22
But these orchestrators are helping you abstract away.
35:23
A lot of the dependent, like, you know, if you ever looked at
35:27
the difference between what it takes to write a queue flow
35:29
pipeline forces that are go workflow.
35:32
The lines of yeah move code versus how much you know, python
35:36
code use rights, it's tremendously different.
35:38
So the advantages I think again from people of pipelines,
35:41
kubernetes native right out of the box allows you to run those
35:44
that crazy, maybe the very complex data pipelines when it
35:48
comes to other tooling. If you can, if you're not
35:50
cooperate as native, there's probably there's better options.
35:53
Definitely so. I think that, yeah, I think it's
35:58
what I use to come down to. That's that's the that's the
36:01
theme that keeps coming back to. That's where I draw the line.
36:03
Is there you full for the kubernetes or you are?
36:06
You just trying to find your way in the ml Ops World?
36:10
If you're not sure, kubernetes is right for your team, then it
36:13
stopped working through teams and I would say steer away from
36:15
Cube flow. Go some work go somewhere else,
36:18
there's there's better. There's better easier tools to
36:20
use. That's one you mentioned this
36:25
this idea of it. Our communities native then then
36:29
yeah. Go for it.
36:30
Otherwise, if you're up-and-coming then then maybe
36:34
start with something a little training wheels included.
36:37
Is there a point in the work that you've done?
36:39
Where you see? There's a natural transition
36:42
going from, something like a managed service or manage
36:46
provider to then saying, hey you guys got it like now it's time
36:50
to you know, now you're getting your green belt or your whatever
36:54
the next whatever comes after. Yeah.
36:57
I don't even know that. I don't know karate, but you
36:58
didn't say like you're going up the belts.
37:01
Kubernetes, even that yourself. Maybe.
37:02
Yeah. Is it yellow?
37:03
Okay, so the brain has a yellow belt.
37:06
You now have have been promoted to the yellow belt.
37:08
Now, you can set up your first Cube Cube flow pipeline.
37:11
Is that, is there a point where you see that happening in the
37:14
work you've done? Or do you just or once a team is
37:17
been indoctrinated in the native cloud provider?
37:22
Then you kind of say Edge like stick with this this is good
37:24
enough for you, use case. Yeah no that's a great.
37:27
Question. I think the, the short answer,
37:30
the good Consulting as it depends.
37:32
But like but like, no, honestly, like there is a point of
37:36
maturity that does happen. And it does come down to team
37:39
size, the demand for whatever it is, you're building, right?
37:44
Like, how much and then how much control.
37:47
That's like the those three things that I think of what it
37:50
comes to, what does it make sense to go from a managed
37:53
service to to, I'm going to run my kubernetes cluster.
37:57
Or on that cloud service. And so yeah, when it comes to
38:01
like team size again, do I have support?
38:04
Is there are capable platform to help me out that that's like.
38:07
Okay, there is actually, oh, there they are doing kubernetes
38:10
than that. Make it might make sense to do
38:12
this. Because long-term, if that's the
38:14
direction, your company is going to benefit, your team is going
38:16
to benefit from just getting over that hurdle.
38:19
Now, potentially, the other size right there is, is the man like
38:25
how when you're building, whether it's in the pipeline for
38:31
training, whether you're trying to serve something up, how much
38:34
demand is there from like a I'm trying to think the word here.
38:42
You know, how often is that model?
38:44
Being utilized? Like, is it being strained once
38:46
a week? Is it?
38:47
Trained daily? Is it trained?
38:50
You know, dynamically based on, you know, some observability
38:53
metrics that you have going on. So what?
38:57
So by kind of looking at that maturity of that, how mature is
39:01
my model? How much her is my pipeline?
39:03
How mature is my everything that we're doing there, does it
39:07
that's terrific Tipping Point where it's like, this is where
39:10
cure is might make sense because you want more Fine-tuned
39:12
control. You want to be able to, you want
39:14
more control of the scalability, you want more control over
39:17
where, and how it gets deployed versus.
39:19
I'm just going to serve this up to vertex and, and And let them
39:24
take care of it, right? But you still have control that
39:26
right, you still there, still control their still.
39:28
There's still a kind of, you know, I see her go back and
39:31
forth. So, is there a clear defined
39:32
line? Now, it really just comes down
39:33
again to the direction that as a whole, your team.
39:37
Your company is trying to make or whatever it is.
39:39
You're trying to do is making and that thing is where that
39:41
says makes sense. Okay, if you're trying to scale
39:43
this out largely some point Cooper is probably going to be
39:46
what you what you're going to be running on.
39:48
So just have that direction. But as far as the sir good entry
39:55
cost is the big part of that, right?
39:57
And so that's a really good one. So like as you think about cost
40:02
for kubernetes because if you just kind of let things go, I'm
40:04
just going to scale and add notes, notes notes, has to this,
40:07
to my cluster, whatever, right? There's cost.
40:10
Obviously, there's a cost Associated that.
40:11
So, there does come a time chance for.
40:12
It's like, okay. When you look at the managed
40:14
offering how What is my cost for that?
40:19
Total cost for that offering, right if I use vertex serving,
40:22
right? If I'm using the models for
40:24
serving their like, what's that cost versus if I just have my
40:27
kubernetes cluster, that's always on and it's taken care of
40:30
that serving, right? What so definitely costs a part
40:33
of this and that's a good call out.
40:36
There are some companies where cost is not an issue and it's
40:38
more about his it's because it's for critical that availability
40:42
in control is there versus, if you're very cost sensitive, if
40:46
you're very You know, cost trying to be very cost aware
40:49
which is important for everybody at argue, but maybe that's your
40:52
Forefront, then maybe it does make sense to to, maybe use a
40:56
managed offering in skill slowly that way again, just because
41:00
you're using the manage offerings, not have any saving
41:02
money, because there's lots of fees that come with those,
41:05
right? Because not only now, are you
41:07
getting charged for just the compute instances but the
41:10
service itself costs extra on top of that, right?
41:13
So it, it's kind of this, this weird But yes cost analysis
41:18
definitely comes into play. It's definitely part of the kind
41:21
of the recommendation process and the the end state of words
41:25
that make sense right now we're to make sense in five years or
41:28
so it makes sense forever if that makes sense right now.
41:31
And also I would be remiss if I did not say this.
41:35
I feel like a lot of people in the community.
41:38
Shout about it quite a bit, don't forget about the engineer
41:42
cost. And like the actual resources,
41:45
you need to pay people. Tim you do this or keep maintain
41:50
your Cube flow or just kubernetes?
41:53
So that is a whole other factor that you want to keep in mind.
41:57
Absolutely. That's real as part of that team
41:59
size part, right? Like if you don't have the to
42:01
support it don't go there because unfortunately someone's
42:04
going to become somebody's are going to become the dedicated,
42:09
you know, kubernetes people on that team and that may or may
42:12
not be good and then they leave if they leave.
42:15
That's, that's yeah. So it is Definitely, there's all
42:17
of that considered so for sure. Totally.
42:20
So I want to dive into and I swear, I'm not trying to make a
42:25
joke here, but it like sounds like a joke because we're gonna
42:30
do it. I built it up too much.
42:32
Now it's not going to be funny. But how have you seen Cube flow
42:36
being used with some of the other flow Brothers, like,
42:39
airflow and ml flow? Have you seen it being used?
42:43
I know that I've seen, like, airflow and to blow you talked
42:46
about It a minute ago. I've seen those used together,
42:49
it almost felt like there was too much complexity involved and
42:53
it wasn't totally necessary but, you know, everybody likes to do
42:57
their own thing and then m/l flow it almost like years ago,
43:02
ml flow used to be it was very clear like what its Niche was
43:06
and what it was doing. But as Mikail and I know because
43:09
we just interviewed some of the meadow ml flow team, they have
43:15
expanded quite a bit. And so now, it's not like so
43:18
clear. Would you use those two
43:20
together? Would you just choose one or the
43:22
other? Like how do you look at those
43:25
trade-offs? That that's a fantastic
43:28
question. Yeah, I airflow Cube flow that
43:30
seems like an odd mix to me. That seems like a lot of
43:32
complexity for no reason. Since you're using an
43:35
orchestrator with another orchestrator, I don't quite know
43:37
what you're trying to do there. In fact, you know, because Q
43:41
flow could also run airflow tags as well again.
43:45
But why, why would you do that? I don't know, I don't know,
43:48
maybe there is a good reason so someone can enlighten me
43:50
hopefully of and why that's a really good idea.
43:52
Because if you seen it, there might be a really good reason
43:54
why to do that. And they'll flow seems like a
43:57
good fit in my opinion mainly because of the maturity level of
44:00
Q flow still even though there is artifact tracking built-in
44:04
naturally, cute. Floated keeps track of
44:06
essentially, your whole lineage as off your artifacts as you go
44:08
through a pipeline just naturally.
44:11
Ml flow adds a lot up that game quite a bit, I think.
44:16
And so it's a in my opinion it's a pretty good counterpart or not
44:21
account for a co-product 2q flow just because of like with
44:25
because you're right you there's all the tracking part especially
44:28
the model registry. I mean if you're using the
44:31
previous KF surfing case serve you've kind of got your own
44:35
model registry and Cube flow. Now it kind of it behaves like
44:38
that some degree right? Has that and that's but it is
44:42
kind of dependent on with the models deployed or not.
44:45
So I still That ml flow has a good Niche, you can kind of fill
44:49
in some of the gaps that Q footage is quite hasn't mature
44:53
to yet because there's a lot of pieces to this puzzle.
44:57
That's the one I would think that makes sense as, as far as
45:02
other things go, like, you talk about prefect or meta flow.
45:04
I don't think you should mix and match Cube flow with that.
45:06
That just doesn't against like the airflow thing.
45:08
It just doesn't, there's, there's too much overlap and I
45:12
don't, I don't see the advantage of either, or again, if you need
45:16
something that metal, Bill provides for you.
45:19
Go for it, right? That's then that's the way the
45:20
GUI is really great. Right?
45:22
The, what benefit it provides you from an artifact tracking as
45:24
well as really, really awesome out of the box.
45:26
So, yeah, so that's the natural fit for me has always been an ml
45:30
flow plus Q flow. That's that's a natural can fit
45:33
that can work well together, obviously tensorflow, extend, if
45:36
you're actually doing that kind of stuff, haven't dealt a ton of
45:39
people who are obviously that fits naturally in the queue
45:42
Philistines, that's where the whole thing started.
45:43
Anyway, but Those are the two that I come to mind, though.
45:48
The flow Brothers ml flow Q, float?
45:50
Those, those work. Well, that cute better flow.
45:52
Could warm LML flow can work with a lot of different
45:54
orchestras. They could, it can enhance
45:56
pretty much all of them. So, yeah, those are cousins,
46:00
cousins, Brothers, exactly. Maybe not, always, welcome to
46:05
the family reunions either tonight.
46:07
Welcome. Yeah.
46:07
They're they're animated Theory. Yeah, I'd love to maybe take a
46:14
step back or step up. And and ask a more broad
46:18
question because we've spent a lot of time talking about
46:21
kubernetes and cute flow and obviously very important and
46:24
something I'm glad we're disgusting.
46:26
But you also do just Consulting in general for a lot of
46:29
different clients and so I'm wondering is is kubernetes and I
46:33
mean maybe this is still crazy people question.
46:34
But is this something? Is this what you see, most often
46:38
that companies need help with, right?
46:40
Is this like the biggest Emma laughs challenge that the
46:42
clients that you work with, think about, and ask for your
46:45
guidance on or their other? Other challenges that also
46:48
surfaced repeatedly that you have to think about and help
46:51
people with, you know, it's gonna make sure I understand too
46:54
kind of the questions to like you're talking more about like
46:58
is Q flow but we're helping more with or kubernetes in general.
47:01
Or is it just like from the MLS landscape like?
47:04
Yeah. From the mops landscape.
47:05
I think it's what other challenges people deal with or
47:08
is cute poker buddies. I mean we spent so much time
47:10
talking about is that because that is really what people are
47:12
most often struggling with, right?
47:14
Or is this just what was the other stuff?
47:16
People think about yeah, it definitely comes down to yeah,
47:19
no, great question. So it comes down to the maturity
47:22
level of where people were companies and teams are at in
47:25
their ml Ops Journey. There's definitely like a
47:27
maturity level of this where you're at.
47:31
So in a lot of cases, Emma, laughs the Hot Topic right now,
47:33
right? In terms of lot of company
47:35
because a lot of people, they've hired tons of data scientists.
47:36
They have these machine learning Engineers, but they're still
47:38
struggling to make it all Jive well, and it's going to be, it's
47:43
always gonna be the challenge, right?
47:46
So that Emma laughs a Hot Topic. It really comes down to.
47:48
So when you're looking at like, how do I help business get from
47:51
A to B, maybe they want to get to d.c.
47:53
They want to get way down the road here.
47:54
So what is the best way to get from A to B first?
47:57
What are the baby steps? I got to take right to actually
47:59
get to where I want to be Cooper days?
48:01
Is definitely part of that, right?
48:03
There's a lot of people using kubernetes, it's definitely
48:05
kubernetes. Definitely shows up more on kind
48:06
of like the app side right now. I think I think still we're
48:09
seeing this transition to companies using it for machine
48:12
learning and for their data science practices.
48:16
I do think though that there's more work that standing right
48:18
around just helping people transition still from Legacy
48:23
on-prem stuff to the like stuff this breakdown of through, right
48:28
to the cloud, right? So all of their on-prem, all of
48:31
their whatever it is. They're running right there.
48:35
All their old infrastructure. How do I get?
48:37
This is the clouds. That's like babysit one.
48:39
How do I go from here to here and get the most out of it?
48:41
So in that case, this Cube flow make sense, maybe maybe just
48:46
Whines. Maybe I just have a standalone
48:47
into the pipelines for orchestration, but there's other
48:48
options for that, right? So maybe that's not the right
48:51
answer, so I think, you know, in that case, you're just trying to
48:53
get them onto the cloud. How do I get my machine learning
48:55
training on to the cloud? And how do I do that in a way
48:58
that's Cloud native? So that's like one step and then
49:02
you're also seeing a lot of. Okay, now that I've done that,
49:04
how do I separate different environments for production and
49:08
for training and building like how do I break these things out?
49:10
What's the right kind of thing? That's, that's a huge challenge
49:12
to that. Only people think a lot about is
49:14
you're going to have Like how do you divide up?
49:17
Your different environments on Google Cloud, right?
49:19
Is it by projects? You have a production project,
49:21
you have your training project. So there's a lot of how do I
49:24
move my model that I've built here and training in an
49:28
automated? And you know, repeatable way
49:31
into maybe a QA environment so they can go through another
49:34
series of tests. And the whole before, then the
49:37
model actually gets thrown over the fence to a serving, you
49:40
know, production server, for production cluster, whatever it
49:43
might be for, for actual use. So that's a huge challenge to
49:47
that. We do a lot with those are the
49:50
big ones that we might go ahead and what are some best practices
49:53
on that because I know that's come up quite a bit to in the
49:58
community. And like, how do you even just
50:01
have your file structure? What does that look like?
50:05
When you're building your models, like these are what you
50:09
would think I guess in software? It's kind of a solved problem.
50:13
But for machine learning it's not So clear anymore.
50:17
So have you found best practices around that?
50:20
That's yeah that's great. I still feel like every case is
50:23
different to some degree, right? Because you have to kind of mold
50:25
to what has already been done and try to make it.
50:27
But yes, there's definitely. I mean your death were always
50:30
borrowing from the software. The devil's practice is right.
50:33
What is the right way to structure?
50:35
My training code that I've created how, and how do you know
50:40
how to merge is? How does you know whether it's
50:42
like, trunk based development? How does that fit into my to my
50:45
prom? Us to my flow.
50:48
So I definitely think you know from a best you definitely
50:51
should be trying to use see ICD, that should be part of your
50:55
process. There are still a lot of people
50:56
to understand because there's a lot of red flags.
50:58
There's a lot of red tape to jump through, but there's still
51:00
a lot of teams that are manually moving things across the board
51:04
right to wherever that might be and that works to some degree.
51:07
But at some point, it becomes unmanageable.
51:10
So I think just going back and looking at.
51:12
Okay, what are the cic practices that I should be implementing?
51:15
Am I getting To use a cloud build sites, another scenario,
51:19
right? We're going to do some Cloud
51:20
native. Am I going to use GitHub
51:22
actions? What am I going to?
51:23
You know what, what makes sense from a security perspective from
51:27
Team development perspective? On my team do I use?
51:30
Are you know, do I use Argos CD right?
51:32
Like what is it like you know but there's definitely is her
51:35
best practices. Well you definitely guess
51:37
there's some simple ones of you should have a repository for
51:40
your code. It should be version controlled.
51:42
You should be implementing trunk based design as much as you can,
51:44
right? You should so that, it's Faster.
51:46
Especially, I mean, it may be, that there's definitely do that,
51:49
right? You could, but like, you should
51:51
be implementing the things that makes sense from a software
51:53
perspective, because they definitely pay dividends on the
51:57
long term for technical debt from the machine learning
51:59
perspective. So that's, there's my big
52:03
blanket answer. The specifics always change
52:05
based on what the company or team is done previously and
52:08
where they want to be. But yes, there are, we should
52:11
try to be borrowing from the software practice as much as you
52:14
can. When question that at that.
52:16
I also have again more of a high level one.
52:20
So you've been in data science and ml ml opts for a number of
52:23
years. What has changed over the years
52:27
as you've experienced and what can we expect?
52:31
Moving forward. That's a good one.
52:34
The prophecy question. So we're gonna hold you hold
52:38
this, is it like this is a record we go.
52:40
Well, see ya Ryan says, he already has things that have
52:45
changed, I think? A lot is there's just it's
52:48
almost, it's overwhelming the number of tools that exist.
52:51
I don't think a day goes by that.
52:53
Somebody doesn't land in my LinkedIn.
52:55
Mailbox or someone wanted me to check out the latest something
52:57
they've built right and there's a lot of oh the latest flow.
53:00
I got this, love that. Yeah we got to stop naming
53:02
things flow though. I mean that's not an end at this
53:04
point. There's too many of them to keep
53:06
track of this but I think just the number of tools.
53:10
How like if I look at like how we did things when I was working
53:15
for a summer. Tractor manufacturer like how we
53:17
trained models like you know, image classification models
53:20
locally and just it's such a short time.
53:23
It was like it always it made sense to just immediately
53:25
transition to the cloud and start using those because we're
53:28
cost perspective, right? You go back to that.
53:30
If you're maintaining things on site, there is a cost.
53:33
I just with the product or the, the the actual Hardware, but the
53:36
persons or person the person or persons that have to manage that
53:39
Hardware. Does it make sense to roll those
53:41
costs up? So I think that just the
53:43
accessibility of Do very complicated things.
53:47
His tremendously changed like it's been gotten so much better.
53:50
Like the fact that you can do things that I don't, I think,
53:53
you know, for five years ago, I'd only like dream about doing
53:56
because I needn't have the right compute, environment or DeWitt,
53:59
you know, I can't. I can't build that home or I
54:01
can't afford to build that at home, right?
54:04
This is now, just totally accessible.
54:05
That's the biggest thing I think so.
54:07
I think that's, that's the cloud.
54:08
That's the, hyper scalar, effect of, you know, Cloud vendors,
54:11
David really made that accessible.
54:12
That's the in my opinion is one of the biggest, the biggest
54:16
Ages. So, looking out in the future,
54:18
that's not going to change, right?
54:19
They're still going to be that, you know, machine learning or
54:24
just clot, you know, computers, a service type scenarios where
54:28
it's going to always be there. The where things are headed, I
54:31
think there has to be some kind of a some point.
54:34
There's going to be some kind of breakdown of all the tools that
54:36
exist there is the going to be like, there's going to be a,
54:39
there's going to be some clear winners that stand out and stand
54:43
the test of time when it comes to what has been Create what is
54:46
being created right now and where things are at?
54:49
Who those are? I don't know.
54:51
Do I want to throw a cube flew in there?
54:52
Yeah, it will. It will see, right.
54:54
I don't, I don't know. I'm not going to say that this
54:56
again. They visit the end state, right?
54:59
I like you flow a lot, but there's I think there's still
55:02
there's lots of gaps and all these tools and they're all,
55:05
they all fit a stay. All kind of try to fill a
55:07
specific part of your ml Ops Journey, your, you know, your
55:11
data science life cycle but they none of them do it perfectly.
55:16
There's always very large gaps that are still yet to feel.
55:18
So it's going to be. I think the most interesting
55:20
things to see platforms that come out and I hate use the word
55:24
just platforms. It's just so broad.
55:26
But, you know, tools that come out that really actually make,
55:31
you know, a full ml Ops platform.
55:34
The cloud vendors are trying to do it too and they're doing a
55:36
pretty good job but they're still holes, they're still holes
55:40
and what's being done and there's still a lot of it's
55:43
complex and so I don't know. I think again the what Where's
55:46
things going? I think there's going to be a
55:47
consolidation of clear tools and winners.
55:51
That's yeah, it's funny. You say that too because when we
55:56
did the apply conference, I don't know how many like two
55:59
months ago. Now, Matt Tay from data bricks,
56:03
who I imagine. Most people listening to this
56:05
know he's one of the creators of data bricks.
56:09
He was talking about how he doesn't see like his vision.
56:14
Is that people aren't going to be Using more than two tools in
56:17
their machine, learning workflows.
56:19
And it was the first time that I realized, because I up until
56:24
that point thought, you know what, it's going to be
56:26
best-of-breed. There's definitely not going to
56:28
be one tool to rule them all. That is like a horrible concept,
56:33
that I don't think is going to succeed.
56:36
But then when he's like, yeah, we will be using two tools at
56:40
most, and that's just because of its more pragmatic.
56:45
And I Of see where he's going with that.
56:48
And I kind of feel like yeah, he might be on to something there
56:52
and the dude, I mean, of course it's easy to jump on the
56:54
bandwagon of Matt's a and be like.
56:56
Yeah, I'll just follow what that guy says.
56:59
He's hit a few home runs already.
57:00
So yeah, can't really miss. But it makes sense to me.
57:07
He also it's easy for him to say as well, right?
57:09
Yeah, it's his best interest is in beta bricks which already has
57:13
a little bit of a head. Start to be the one that we all
57:16
consolidate around right now. So true.
57:18
So true that these the other piece that is the other piece.
57:21
Yeah. Data bricks lesson things to
57:23
work out to they do not but they definitely got some good
57:25
momentum. Yeah that's I see it like that.
57:29
It is pragmatic. You don't want to have.
57:32
Here's my 10 12, 20 ml Ops tools.
57:35
I use a my that's That's absurd, right?
57:38
It just, it doesn't make sense and it's not going to make
57:42
things easier. So yeah, consolidation down to
57:45
tools, you know, or three, maybe that that does sound like what
57:50
things will look like here in. Hmm, maybe they're not to
57:53
develop a little bit. When we talk about prophecy, I
57:56
thought, I've been really funny if you had not thrown in Cube
57:58
flow as part of your prophecy because they would have been,
58:01
like, we spend an hour. Why are we talking about?
58:03
Yeah, but it's actually not gonna win.
58:05
I buy property. Execute low is definitely not
58:09
going to be the future. So thanks for the our joy will
58:13
it last. Yeah.
58:14
Enjoy. The only gods.
58:15
Don't ya know. That's, yeah, that's a is a
58:19
question I think of any tool right like where does it have
58:21
again I think from let's go back to it from a flexibility
58:27
standpoint. Pupil offers you a lot of
58:28
things, you know, it gives you a lot to do and it gives you that
58:32
cloud vendor field without having to lock into a cloud
58:36
vendor. The first day.
58:38
So that's the that's the that's the power it holds right now in
58:42
my mind and it's got it done. Hopefully, the community keeps
58:46
going with it because it's fantastic.
58:48
I think it's fantastic but