Latest / Agentic Conversations (formally mlops.community) / ML Platforms, Where to Start? // Olalekan Elesin // Coffee Sessions #118

Transcript

ML Platforms, Where to Start? // Olalekan Elesin // Coffee Sessions #118 · Agentic Conversations (formally mlops.community)

0:00
Like on. This was cool man.
0:02
I mean I feel like we always are saying wow what a great
0:05
conversation but it's true. We kind of have pretty good
0:08
conversations all the time. So what do you think about
0:11
pretty good at this shit Demetrius good at this shit.
0:15
What can I say underrated? That's why people say.
0:18
Yeah. In the podcast Niche, the a IML
0:23
podcast Niche. We are the most underrated or
0:26
sleeper. People are sleeping on this
0:27
podcast right here, completely Lee, which is why?
0:30
If you're listening, you should ask your friends.
0:33
But today we had on Lake on Ellison, who is currently the
0:38
director of data platform at HRS group and he joined us today to
0:44
talk about his experience previously, building a machine
0:47
learning platform at Scout 24. I really enjoyed this
0:51
conversation and I'll tell you why dimitrios because this was
0:54
the best demonstration and explanation of the role of
0:59
product. Men in ml, technical product
1:01
management, ml that we've had on this podcast, right?
1:05
A lot of times what we talk about is the totality of what's
1:08
required to put machine learning into production and just like
1:12
the way that lacan went from an engineer to a data engineer into
1:17
a product manager Charters as journey and made that, you know,
1:21
applied the lessons. I thought it was like an
1:23
incredible description and then discussion.
1:27
Yeah I mean he also called out what But the main metrics they
1:31
were looking at were and how they didn't have infinite
1:34
resources. So they had to focus on two
1:36
major. Like I wouldn't say low-hanging
1:39
fruit but like the most impactful things that they could
1:42
do for the data scientists and I really appreciate how he broke
1:46
down, how they figured that out, what the most impacting pieces
1:49
of the platform they could work on were and I mean, just
1:53
all-around. Great guy.
1:54
The dude is what did he say? At one point I'm paraphrasing
1:58
here but it's a, it's a Quote, it's like be number 2 or be
2:02
number one or number two in something, or get out.
2:05
And so that. Yeah, that's exactly.
2:07
It is. The said I'm trying to be number
2:10
one or be number two in your career or get out.
2:12
That's his attitude. This dude does not mess around,
2:15
so let's get into it. If you all enjoy these podcast,
2:19
please like Vishnu was saying, tell a friend, or if you don't
2:22
enjoy them but you want them to be better.
2:24
And you think, you know how they can be better?
2:27
Hit us up. We always are looking for
2:29
feedback. Feedback and we've got pretty
2:31
thick skin by now. I don't know if you've seen any
2:34
of the YouTube comments of me playing guitar for the meetups
2:39
that we do, they're not nice. So, if I can handle that, I
2:41
could probably handle your feedback.
2:43
All right. Let's hear it.
2:45
If you want to tell us what you think we should be doing better
2:48
and let's get into. It started out with your
2:55
experience. That Scout 24.
2:58
I know that there's all Stuff that you can talk about as far
3:01
as technical leadership and building platform teams and I
3:05
hope that we can get into that. But first, maybe you can give us
3:09
a bit of background on what it was like at Scout 24.
3:13
What was the situation? Okay, so It's very interesting
3:19
because any time I think about it, it helps me to reflect more.
3:23
I think it was more of the and the beginning when I joined, I
3:26
joined Scout 24 hours a day, the landscape engineer, right?
3:29
And then there were, we have this transformation to set of
3:31
platforms and platform teams internally, and there was this.
3:35
And I think back then we had these three pillars.
3:36
So we came up the transformation team came up with this
3:40
application platform, they did a platform which I was part of us,
3:43
did a landscape engineer and then there was this third
3:45
pillar, which was the AI platform.
3:47
I was just there nobody was listening picking it but it was
3:50
quite Vivid on the on the pillars.
3:53
And then I think the CTO back then I forgotten his name is
3:57
name is first. Name is Richard, by forgotten
3:59
his last name, there was really keen on having this AI platform
4:02
team. So I walked up to my manager
4:04
back then and said I'm interested in this.
4:06
I have a bit of product sense and I might be able to make this
4:10
happen. So we started the process of, oh
4:13
yeah, you need to have product management skills.
4:15
You need to go for product running workshops and all that.
4:17
All that just to build my product knowledge because I was
4:20
coming from technical, really technical background.
4:22
I am from software engineer to full stack engineer to front-end
4:26
engineer, to data engineer data science, engineering.
4:28
That's a lot. Did a landscape engineer, but I
4:31
had this Nick for product. So what happened was, I think I
4:35
was on a training for about 6 months or like an apprenticeship
4:40
on the technical product manager for data platform back then,
4:43
which is who is my mentor, by the way and then evolve Into the
4:48
technical problem management of the AI platform.
4:51
So basically started did with one it I think about engineers
4:54
that were basically machine learning Engineers for a couple
4:58
months. I think I did as well as Central
4:59
data science team back. Then I think for me that was the
5:03
starting point and interestingly I attended reinvent this was
5:08
2019 and one of the things that caught my attention was a
5:12
technical product manager there who was for the think the step.
5:17
Functions SDK for sale for data science SDK, which later became
5:22
or evolved into Sage make a pipelines and I was quite
5:25
intrigued by how he was able to break down a couple of things.
5:28
I was able to even write some write code.
5:31
So I then went back and said yeah this is it, this is what
5:34
I'm going to do. I'm going to bring ml to live as
5:37
a platform at Scout 24 bugs. And I'm was what about six
5:41
months there? About boy was an interesting
5:43
Journey. So initially we didn't know
5:45
where to start and this is Definite, all platform teams,
5:49
right? We didn't know how to start.
5:50
We had clashes with the data science team because they felt
5:53
were trying to take their jobs away.
5:55
But interesting, it was just us figuring out the what we wanted
5:59
to build rather than just going on.
6:01
Yeah, we have a platform and then we have this idea about a
6:03
platform. Let's just go build.
6:04
It was basically all stranded in the found in the beginning
6:07
phases, really? Doing a lot of Discovery work,
6:10
which I think most startups actually do if they want to if
6:12
they want to succeed, right? Which is we want.
6:15
We have a we know there is a problem.
6:16
What where Is the solution. How do we solve that problem?
6:20
Not necessarily going from an idea to the solution space which
6:23
is just walking through Discovery phases.
6:25
And then eventually, To the problem and then scaling the
6:30
solution. I think that was that was
6:32
basically it in the initial phase was a bit rough.
6:35
We had retrospective with data sciences and it was more.
6:38
Oh yeah. You're trying to do this, but
6:39
this is how people to engage us and blah, blah, blah, things
6:42
like that. Eventually, we figured out how
6:44
to solve, which is solving for the data scientists first.
6:47
Then, also from that, then scaling to the other day or the
6:50
other other parts of the organization or that engineers.
6:54
And then what we realized was at least what we We learned and
6:58
what I learned was wondering, what else were able to solve for
7:00
data scientists? That were willing to work with
7:02
us if only it was painful for them.
7:04
In the beginning was a really huge change from their existing
7:08
workflow to a new workflow. I think one metric does took
7:11
that's stuck with me that I took away was when we interviewed one
7:16
of the data scientists and the beginning, each take them out at
7:19
least four weeks to move into production, right?
7:22
And then we turn it around to maximum of three weeks.
7:27
And then we had one engineer was really Keen about what we built.
7:31
We went from building the model into production in, I think less
7:35
than 24 hours. So I mean that's the ultimate
7:38
metric for a platform team rate, time to production exactly.
7:42
So I thought that was a really great overall landscape view of
7:47
the platform Journey that you guys went through at Scout 24.
7:53
I want to start sort of by backing up and for Asking what
7:57
is a data? Landscape engineered a
7:59
landscape. Architect that what was the role
8:01
that you were in just prior to shifting into primary product
8:04
management? Okay, was more.
8:07
Let me think about that because that's an intelligent question,
8:09
right? I've never thought about what it
8:11
is, but I know what I was doing, sure.
8:15
I think what it is was really making sure that was an engineer
8:22
was responsible, or was part of a team responsible for making
8:26
sure everyone could do analytics, easily.
8:30
Okay. And yeah.
8:31
So from so yeah, go ahead. So basically, like, You guys
8:37
had, I'm assuming a lot of analysts dashboarding
8:41
professionals, maybe even analytics, engineers, and you
8:44
were sort of responsible for the data model and infrastructure.
8:47
That those professionals interface to it.
8:48
Is that right? So whereas fumbles were
8:51
responsible for one piece of it which is the data infrastructure
8:55
so that was made of major responsibilities even though we
8:58
had some data models that were responsible for which also
9:02
helped us to know what we're building on how to use what
9:04
we're building where was more of the infrastructure.
9:06
So we have this notebook environment before stage makeup
9:09
became popular. We had this notebook environment
9:12
that we were managing. Which was based on elastic.
9:16
Mapreduce, it'll be a game. Our yeah, but then what we then
9:20
observed, which was one of the things that was a huge learning
9:23
for us. At least for me, when building
9:25
the the AI platform was that data scientists because it took
9:29
us has started. It took the infrastructure,
9:31
which was a Mr. But then it took about 10 to 15 minutes to
9:34
launch, right data scientists, didn't need a lot of spark.
9:38
They all they needed was on ec2 instance to come up in less than
9:42
five minutes, and they wanted Do something on Jupiter did not
9:45
every of their work hard to do with spark but for analysts the
9:48
data analysis was quite different because they were
9:51
munching huge amounts of data. So the data scientist just were
9:54
clever enough to then really clever people to then check out
9:58
some easy to machine connected with their local through SSH
10:02
tunneling. And for them, it was normal
10:05
right? Was.
10:06
Oh yeah, it takes me. 10 minutes or 5 minutes or seven minutes to
10:09
start this up. So why should I try something
10:11
new? And this was one of the first
10:13
First things that we saw, which is, we know that you're doing
10:16
this already. If you go by this approach,
10:18
which you don't have to think of so many scripts, SSH, tunneling
10:21
and all that, just a CLI to that says, AI platform notebook
10:25
start, and it's not stopping no book in ready in less than 30
10:29
minutes done, I think five or three minutes and you don't have
10:33
to think about copying things or back and forth from your
10:36
computer to your remote server. Would you try that unusually was
10:40
a lot of no I wouldn't try it because I prefer what I was
10:43
doing. Every other customer, right?
10:46
But what we then did was to show them how it looked like.
10:48
So I start with them in real customer interviews, run into
10:52
growing three cost. My interview and saying walk me
10:55
through your daily experience and then they walk me through
10:57
and then I show them this huge. This interesting simple thing to
11:00
do and I like, wow, can I use this right now?
11:04
My yes show, just log in to AWS account that you can use it and
11:08
then can I copy my notebooks from?
11:10
I thought. Yeah, you can do that and then
11:12
okay. Yeah, that's another next
11:14
morning you. And then the next one you get I
11:17
tried this on his super interesting.
11:20
Can I deploy model right away? I'm like sure.
11:24
Yeah, that was out. That was how we went got it.
11:27
Now, I love that. I it's it's so clear.
11:30
How you think as like, both a product professional and an
11:33
engineer from the standpoint that you completely understand.
11:37
This is actually where I think a lot of sometimes this technical
11:40
product stuff can break down where it's like there's a real
11:42
work flow component that The tools of the trade are one of
11:46
the most defining aspects of an engineer, a data scientist,
11:49
like, whatever. Technical professional your
11:51
tools. Make you a technician, right?
11:54
And I think, I think a lot of product professionals walking in
11:59
may not necessarily understand the emotional connection to your
12:03
tools that every technical professional has right.
12:07
You know, there's a reason why I like there are certain like old
12:09
car mechanics that, you know, do things do exist, the way that
12:12
they've been doing for like decades.
12:14
Right. You know you're tied to your
12:15
tools and I think it's cool to hear in your in your depiction
12:18
of the events. How intimately you understood
12:22
like, why the different technical professionals at your
12:24
company work the way they did and then we're able to create
12:27
that customer Delight experience, you know, on their
12:30
behalf. The question of.
12:33
Yeah, go ahead. Go ahead.
12:35
I think you're totally right and this was not easy for me because
12:38
I'm an engineer. I'm an engineer by trade I
12:40
studied computer science think really In from the cosmic
12:45
perspective was all of it was a bit of a challenge.
12:48
I would say, also for the engineer's that was working with
12:51
back then but it was something that I had to learn and I give
12:55
you an example, even right now, more building on AI platform as
12:58
well. Currently I'm building data
12:59
platform and then leading some cross functional data science
13:01
teams. This particular problem that I
13:03
just highlighted the other people that you mentioned right,
13:06
people being used to what they're comfortable with.
13:09
We had a internal customer like a roadmap discussion that To
13:14
view session. And one of the data scientist in
13:16
my current company said, we've been working with this
13:19
technology for at least five years, and we're used to it.
13:24
You need to convince us, why? And which is for me, it's
13:27
normal, right? I guess for people new to build
13:31
an internal platform teams. The assumption is, yeah, we are
13:34
Nintendo platform teams. You are a colleague, so you must
13:36
use what with build, what it doesn't work.
13:38
Usually doesn't. Most times it doesn't work that
13:41
way. You have to work backwards from
13:42
that emotional connection and making it a lot better for them.
13:47
I agree with you completely. Yeah, I were on the same page
13:51
there. I'm, I love it.
13:53
I I wanted to kind of take a step back here and you know,
13:56
you've you've narrated us through the various actions that
14:01
you took sort of as a product / technical professionals to
14:03
really drive a change in mindset, amongst the data tanks
14:07
team and the Delight that you inspired from Scout 24
14:11
standpoint. You know, I'm actually a fast
14:13
growing startup. I think many of our guests have
14:14
been you know, making a platform investment is A complex
14:20
Endeavor, right? You know, you're investing in a
14:23
long-range project when your business needs in the short term
14:26
are changing very quickly, right?
14:29
And you know, I'm not necessarily as familiar with
14:33
Scout 24s maturity and the size and the revenue size of the
14:36
business. But can you give us a sense of
14:38
like why the company felt like? It was the right time to take a
14:42
take, the leap and invest in a platform approach and you know
14:46
what made them want to make that investment in you and your team.
14:51
Ah, that's a really deep one, you know.
14:54
So I remember back then we I was taken into a discussion so we
14:58
just I think it was quite clear that it was taking us time to
15:02
move from ideas, who 4ml ideas into production, not because the
15:07
data scientists were not Good, to be honest.
15:11
I think they're really good. I would, if I could, I would
15:14
have hired all of them to join me at my current company right
15:17
now. They were really really good but
15:20
it was just the tools of the trade, right.
15:21
And it wasn't that the data landscape team or the data
15:24
platform team, which I was back then which I was part of back
15:27
then was wasn't really didn't really care about what they were
15:30
doing. Was just that they were so many
15:32
things at some point. And this is to product
15:34
understanding right, which is at at every stage in product, you
15:38
need to start looking at the long tail effect.
15:40
I'm not looking for how can you cater for that, long tail and
15:44
most times. It's that's where the, that's
15:46
what that, that's that long tail is what gave birth to now, we
15:49
need to get her for this long tail and then give birth to
15:53
something that helps us to speed time to market for, for it, with
15:58
AI product. So just so I understand that
16:00
long tail piece. They're basically what you're
16:02
saying is you need to understand.
16:05
I guess you could say almost the exceptions, right?
16:09
The places in, on the bell curve where like there is an
16:13
exceptional need or an exceptional sort of opportunity
16:16
that is perhaps numerically small, but speaks to, you know,
16:22
I don't know, the most intense need is that what I'm trying to
16:24
understand their yes definitely. And I give you an example,
16:28
right? So when you two before you two
16:30
started content, creation was more from the say, the big He's
16:35
in Hollywood and then what YouTube did was to kill her for
16:39
that long tail. And let's say, they are people
16:41
that want to create content on their own, that they are self.
16:45
Create self, create their creators.
16:46
And I want to create content. And then YouTube started
16:49
catering for that long tail of people that could create content
16:52
on their own. And this is for me because even
16:55
building a technical platform what I did was to research how
16:57
people build Platforms in general.
17:01
That was my starting point of building a platform in any kind
17:03
of platform. Even what I do right now is
17:05
researching our people build platforms and if you take that
17:08
and take what YouTube was doing and then take to itch on top of
17:12
that twitch looked at a really long tail of Gamers who have
17:15
also creators right? And then build a platform around
17:17
that and for me that for me is looking at it from that
17:21
perspective which is the long tail of analytics at that point
17:24
in time and how that how that long tail was significant to the
17:29
competitive Advantage cause And if I wanted to have, which is
17:31
differentiating with data science and machine learning,
17:35
and how we could leverage that and speed that up as quickly as
17:38
possible. And then building on top of that
17:41
long-term what were some of the long tail things that you had to
17:45
keep in mind, like what were some specific pieces?
17:49
I mentioned one already, which was The Notebook experience I
17:53
think for every data scientist is the first thing is Neil that
17:56
notebook experience. We are that Allah pointed their
17:59
platform team was we had a an internal environment where was
18:03
notebook built on top of EMR and it was taking a bit of time to
18:07
launch. It was maybe 10 to 15 minutes.
18:09
Not that was bad, but it was a lot of things big into it.
18:12
That was General for everybody that wanted to do analytics.
18:15
And data scientists were really clever to hike out something and
18:18
say, yeah, Let's launch something on ec2.
18:21
So the first part was to really simplify that notebook
18:24
experience. Once we did that.
18:26
The next thing was how can we deploy models and production?
18:30
Which the data platform team at that point in time was not
18:33
really catering for even though the data scientists,
18:35
interestingly David had about Sage maker and the things they
18:38
had about Sage make about then was it was really expensive.
18:41
It's complex to use and all that and all that quite a number of
18:45
things. What we then and it was
18:47
difficult to put things together build a pipeline Line it
18:50
together. I was doing, let's say a
18:52
training model training pipeline or let's say back in France
18:55
pipeline was really complex and we didn't want to do it.
18:58
So what we did was The Notebook experience and then and now
19:02
we're going to the into the more technical into the into the
19:06
nitty-gritty of it shortly and notebook experience and then the
19:09
model deployment experience and then the model inference
19:11
experience and how we got to this was.
19:14
I'm not sure I'm allowed to mention this on.
19:15
There was a book I read. It's called.
19:18
Can I mention it? Yeah.
19:20
Please do, it's called outcome outcome driven Innovation,
19:23
right, forgotten tell right off the books.
19:26
Really good book about how to identify needs identify needs.
19:31
And we did it couple of Statistics to figure out what
19:34
was the most pressing problem for people that wanted to go
19:37
from idea to production for machine learning, when we spoke
19:41
to people at random, the what we had was, yeah, we need to
19:45
improve data quality, and this is what people complain about,
19:47
in machine land. They said, oh yeah, it did
19:49
occur. Oddities but I spoke to my
19:50
fellow Engineers back then we were like yeah, you need to
19:52
improve the data. Quality need to improve the data
19:54
quality. So what we did was we had a
19:56
survey. The and this is now the tactics
19:58
of building a platform from scratch ml.
20:03
Ml platform data platform application platform, whatever
20:05
platform. So what we did was to identify
20:08
talk, let's say topical ideas that people would comment
20:11
telling us, we run is just talking to about 10 people.
20:16
Within the company which were our colleagues.
20:18
But if someone is building a start-up from, from a plow,
20:21
let's say, from a platform idea, he stops at least 30 people.
20:25
That's my, that's my, that's my view.
20:27
But internally, we spoke about 10 people and then they came up
20:30
with all you need to improve data quality.
20:32
You need to simplify however, the time it takes to find nude
20:35
ourselves data assets need to, simplify the time it takes to
20:38
build up a model, you need to simplify the time.
20:40
It takes to do x y z. So we created a survey from that
20:43
and then run that solely by 40 people.
20:46
And what we did was from that. So we identified three things
20:49
that we most important, it was number one, simplify the time,
20:53
it takes to get an environment up number to simplify the time.
20:56
It takes to Big pulley machine. Learning model in production.
20:59
I think I forgot in the third and then the third one was
21:02
increasing the knowledge about building machine learning models
21:06
normal of those three. And the, I think the fifth one,
21:09
which was the least on the what people voted was Data quality.
21:16
Fascinating normal. And what we did was to, because
21:19
we knew that with the team was quite small.
21:21
Like, then, I think, one engineering Berlin with me, and
21:23
another engineer Munich. And then we have, we had half
21:26
the time of one engineering Berlin.
21:28
We have to focus on two things and it was really easy for us to
21:32
identify priority because we run surveys across both locations
21:36
and people gave us feedback we collected that.
21:37
Now we saw number one is to simplify the amount of time.
21:40
It takes to get machine learning, modern production on
21:42
the infrastructure and then increasing knowledge about
21:44
machine. And, and that was all that we
21:46
focused on in that six months. And what that did was the metric
21:51
I gave, which is at least four weeks to at most three weeks.
21:54
And then we saw an exception where an engineer way from idea
21:57
to production in I think in 24 hours or days or so.
22:02
And the metrics with what were the other metrics that you were
22:06
looking at? Because there is that one but I
22:08
imagine spinning up a notebook is also another metric.
22:12
You were looking at. And how did that change?
22:14
Yeah. That like I said it was 10
22:16
minutes for the jet engines generate analytics and Romans
22:19
built on EMR and then what the engineer, what the data
22:23
scientists were doing was about five to seven minutes.
22:26
We reach, I think that we reduce it to three minutes if not less
22:30
right? And then what that gave us an
22:31
opportunity. We then started.
22:33
I think we started with this. These are the tactics for
22:36
Building Products, right? We first started with a cop
22:39
doing with PowerPoint, just for the engine, they did a
22:41
scientist, get a feel of what we're talking about and then we
22:43
built a CLI script and then we converted our CLI skills into is
22:48
a python python Jean command line application, where they
22:51
just said, AI platform notebook, start.
22:53
And then, if they wanted to shut it down, they said a iCloud from
22:56
notebook, shut down. And then we realize that they
22:59
will also interview them. And then they said they don't
23:01
want to think about shutting it down, okay?
23:03
Fine, we add automatic shutdown at the end of the day or when
23:06
your notebook is I do. After one hour, we added that
23:09
and they just started notebook. And in the morning they were
23:12
they had to do when to launch. If they were at loan during
23:16
lunch, time with the notebook was idle for about 1 hour is
23:18
shut down. Save costs, and then they came
23:20
back, they started it and they went home at the end of the day.
23:23
The notebook was shut down, and then notebook is all the
23:26
notebooks were automatically saved and then they, they are
23:30
work. They resume their work at, where
23:32
they stopped on top of the second eye, which is the second
23:35
part, which is increasing the knowledge about machine
23:37
learning. Right?
23:38
Is Sage we were using say quick a notebook instances back then
23:41
and Sage maker notebook. Has this part where most people
23:44
don't really know about is that they ask this piece.
23:47
I think there's this hidden piece.
23:49
Where they kind of stone notebooks.
23:53
They collect the think notebooks from GitHub, and then they make
23:55
it available as examples. We saw how we are to go into the
23:59
code and see how that was done. So, we then build, internal
24:01
examples, at Scout and made it available through that.
24:04
So when people were looking at the safe make examples, they
24:07
could also look at our whole, the company examples as well,
24:11
which helped them to speed up the time to machine learning.
24:13
Also, that's so cool. There's one thing that I wanted
24:17
to just ask before Vishnu jumps into it is you mentioned you
24:22
were you were going to get into the nitty-gritty about how you
24:25
did this can you explain that to real quick and it's okay.
24:29
Yeah. So from the product side which
24:32
is that those interviews, right? And then the into the nitty
24:35
gritties, interviewing the data scientists showing them.
24:41
Let's say that's and this is more, the product work that most
24:44
people don't do what the rotisserie it's.
24:47
It's the real success is to Define Meaning factor for your
24:50
success as building an in an email platform or any other
24:53
platform is having interviews with your users internally.
24:57
And then testing ideas out really, really rapidly because I
25:01
was technical, right? I could prototype ideas as
25:03
really quickly as possible. And then, I had one, one of the,
25:06
I think one engineer with me, went during the interviews,
25:09
which made them. See at least, get a feel of what
25:11
data scientist was talking about.
25:13
Also help the engineers to develop empathy and really
25:16
thinking about how we designed this so that it's really easy
25:19
for If what he did a scientist and really getting into explore
25:23
testing, a lot of things out there, still a lot of things,
25:25
I'll try to tweak, Sage make a notebook seeing if this will
25:28
work. And then this led to this was
25:31
how this was the precursor for me becoming.
25:34
I think a machine-learning hero for AWS, which is as I like
25:37
creating content machine learning content internally for
25:40
our data scientists. We won't build Sage maker
25:42
containers to really speed up, things that people were doing.
25:46
And this was most of those kind that I wrote, internally was,
25:49
then what I could publish the most I could publish.
25:52
And then publish them to help the broader community of around,
25:56
at least the public Community, around the machine, learning for
25:59
interviews. Hey, there.
26:01
I'm a Satanist and ever, and I have a product, a teachable hub,
26:04
and if you're into envelopes and want to hear more than just
26:08
buzzwords, subscribe to the envelopes Community podcast and
26:11
listen to the best practitioners in the field, sharing their
26:14
wisdom, where I wanted to To jump in.
26:17
And I really do want to dive into you being a
26:19
machine-learning hero for, for AWS, maybe later on in the
26:23
conversation. But what what got me going was
26:28
really that metric that you had around educating people and you
26:33
know, how that was a part of your definition of success?
26:36
Because what I think are many conversations over the course of
26:40
this podcast or on machine learning platforms, has taught
26:42
me is that number one process Excellence is just as much of
26:48
engineering as technical Excellence, right?
26:51
Your proficiency with your tools and your proficiency with
26:54
process and the reason platforms exist is to encode and scale
26:59
process efficiency, right? Within tools and systems.
27:04
And so I think hearing you describe the way that your
27:08
metrics map to that. You know, I think it's actually
27:11
like a great set of metrics for any platform team to adopt.
27:14
And yeah. You know, I just kind of just a
27:17
thought that I'm having as well. I'm a building out the internal
27:20
data platform at my company, you know, just like time to value is
27:24
the highest, you know, endorsement of a team's effort.
27:29
Yeah, definitely, definitely, I think for me, it's when I talk
27:33
to people that are building internal platforms, usually the
27:37
because most times, right when people build and I think I can
27:42
also mention this person, right? So before I started on that
27:45
role, I had to send an email. I reached out to Marty Kagan.
27:49
Who is like the core? The golden gai?
27:53
Yeah. It is look at really product and
27:55
forever, right? Yes.
27:57
It's gonna boil it and then he responded to me.
27:59
Then explained in detail and also sends me a blog post to go
28:03
through, right? I think most times when I talk
28:08
to team people do want to build internal platform teams, ml
28:11
application, whatever data, whatever they want to build,
28:15
they miss out on the essential. Understanding of you need to get
28:19
the product side, right? First before the plot, before,
28:22
building the platform because you need to really experience
28:25
people that can understand that. These are my colleagues but then
28:29
Building something for them or this is fun building that the
28:32
startup. It's the product side that most
28:34
important what you see, 80 to 70 percent of the time when people
28:39
are building internal platform or ml platforms, is that they
28:42
put really good individual contributors together.
28:46
Yeah. And then those individual
28:47
individual contributors assume that they know what the people
28:51
that are building for need which is individual contributors being
28:54
ml Engineers that are maybe staff or lead but then they
28:58
misunderstood On the one thing you mentioned before, which was
29:01
the emotion between the person that's doing the work and the
29:04
tool the person is used. Yeah, for us for us, it was more
29:09
important to crack that first. Once we knew where that emotion
29:13
light, whether emotion was we fit into with it, so many things
29:18
in there. And then it was Varla, I
29:21
remember, we had one engagement with one at one of our data
29:24
science. Things were building a complex
29:28
problem back then. And this was one of the things I
29:31
really wish I do not Scout which was I built.
29:34
We built it. Sage make a container for
29:37
snorkel, which is this data label in automatic labeling
29:42
library, right? I need it changed the trajectory
29:46
from I think training time when they were building on stage
29:48
makeup because what we're doing before from about six hours to
29:51
30 minutes, which was really, really cool.
29:54
Wow, even this this came as a not like we planned it, but it
29:57
was just something that oh yeah, this is something that Doing
29:59
let's enable them to do this better.
30:02
What, what, what pain me, what? What I really feel is that I
30:04
could not open source because I'd forgotten about internally,
30:08
it was when I left Garden but oh yeah we built this back then.
30:12
Mmm. But I never had time to build
30:14
another version of it. You know, I actually really want
30:18
to jump into talking a little bit about AWS and some of your
30:22
work outside of work. I think one of the things I've
30:26
been doing most about this conversation is hearing like,
30:28
how intense Finally, and thoughtfully you have been
30:31
upscaling. Whether it's like at work or in,
30:34
you know, you know, outside of work, you know, you really
30:37
invested in mastering the fundamentals of the product
30:40
discipline. You have had tons of experience
30:43
as an engineer and in addition you also hold this title of
30:46
machine learning hero from AWS. I want to start by asking you
30:51
how do you think about your career growth and spending time
30:55
intentionally to grow your career?
30:59
That's a very tricky question. I'm very intelligent one.
31:04
So let me think about how best answer this because I don't
31:08
really don't know. I think for me.
31:11
So when I look back, anytime I get asked that question is, when
31:15
I started my career in Nigeria, was before moving to Germany?
31:19
I made up my mind. I think it was in 2014 that I
31:22
wanted to by 2020. I wanted to be the number one
31:26
big data professional in. As a whole and I put this on my
31:30
my stack Overflow profile but this was in 2014, I checked it
31:35
in 2020, I was I was like, shocked like, oh yeah, I think I
31:37
tried and this ties back to when I what I learned quite early
31:42
which is be number one or number two in your field or get get
31:46
out. And this is this for me is the
31:50
driving Factor but then right it's more from a conscious
31:55
decision to always bring value in any way that I can.
31:58
And this is for right now. I'm spend a lot of time learning
32:02
management. Spend a lot of time at the same
32:05
time in the morning, I wake up. I read new life python libraries
32:09
about two weeks ago, up until now, I find myself reading this
32:13
conference this AI Academia conference Rex is oh yeah,
32:19
possible riddance. Yeah, yeah, I realize I was
32:21
going through their videos. I was going through some of the
32:23
right up and I think to be honest I think any machine
32:26
learning practitioner right, either engineer or a leader,
32:29
should be spending time there. There's this other one NLP.
32:34
Conference also have forgotten the number confit.
32:36
I also I'm also reading through some of their papers and Amazon
32:42
science. I spent some time recently also
32:46
going through their papers just to see what what direction
32:48
people are going, what direction the industry is going, the
32:51
collaboration between Academia and Industry.
32:53
And this is how I this for me this how I build my career.
32:57
And then I look at What universities are doing.
33:01
So aside outside work. I took a course last year from
33:05
insead about fintech. The year before then I took a
33:09
course from Wharton about AI for business.
33:14
And this is just for me, really been.
33:16
I want to be an all-around person it with regards to my
33:18
career. Basically that's just don't know
33:21
how to put that in in a very concise way.
33:25
No I mean I think I think that was it's awesome to hear what
33:28
you do, right more than I think I summary or or vision
33:32
statement. Sometimes I think for the
33:34
practitioners that listen to our podcast hearing like the
33:37
Tactical steps of what you do is extremely helpful.
33:39
And I totally think that that was that was helpful.
33:42
What stands out to me is Complete the total thirst for
33:45
learning, right? I desire for Mastery of a lot of
33:48
different aspects of machine learning, which sometimes is,
33:52
you know, sometimes is neglected, right?
33:54
There's a lot that goes into making machine learning
33:57
successful and we talked about that on this podcast and you're
33:59
clearly interested in learning all those different aspects.
34:02
And I got to say, if you ever want to, you know, go for a job
34:05
interview, like don't even bother with interviewing just
34:07
send them this clip from your from this podcast because
34:12
they'll just realize and this guy's trying to win Demetrius.
34:17
I think you have something there, right?
34:19
Yeah. I really like that you said
34:22
this, which speaks deeply to me because I also feel the same way
34:26
it's like anything that I do. I want to bring immense value to
34:30
that and so that it that's super cool to hear.
34:34
I mean following up though and going deep diving deeper into
34:38
AWS and what you're doing in that ecosystem.
34:42
Why did you feel Like you wanted to invest so deeply into a WS.
34:47
Specifically is it because you had the experience at Scout 24
34:51
and you figured well, I kind of already know this.
34:53
Let me help others know this. Hmm.
34:56
Okay, that's cool slightly but that's a very intelligent
34:59
question to, right? So when I joined Scout, I was
35:02
that was my first time. Get it been into it obvious and
35:04
what really fascinated me about AWS, but then was cloud
35:07
formation and which is service that most people don't talk
35:10
about what? It's crazy, it's crazy, it's
35:13
once you get it, it's yeah, mind blown.
35:17
Remember speaking with one of the senior engineers and I asked
35:21
him, so why would I describe, right some llamo template?
35:24
And then it spins of And I spent weeks really trying to
35:28
understand how that worked and immediately.
35:31
I got it. I just was just a light bulb in
35:33
my head, so I can do this and describe infrastructure and this
35:36
was in 2017. I can describe infrastructure as
35:40
a llamo and then, it does this. So what else is possible on this
35:43
on this Cloud on the cloud? And that was how I began
35:48
exploring and exploring. In fact, I invested some of my
35:50
own money to even learn this because I was really fascinated
35:53
at what could happen. And then to the point of what,
35:56
how, how did I start sharing knowledge?
36:00
I wrote a lot of a lot of internal blog post back.
36:03
The not Scout for data scientists and I realize I went
36:06
to visit a stack, Overflow one day and said let me move, see
36:10
what people are asking about and Sage maker.
36:14
And then I've got some of the things that I wrote about with
36:17
actually could be useful for people outside.
36:20
So I walked up to my manager and said is it possible for me to
36:23
share some of this content and said yeah you just remove a
36:27
couple of things that are it's a internal and then feel free to
36:30
share it. So what I did was to change a
36:32
couple of things in there and then made it available for
36:35
people and the feedback I've been getting on.
36:37
Oh yeah, this is really good. And also when I think about my
36:40
own problems that I want to solve, I remember I started CD
36:44
for auto email was basically me reading McKinsey in McKinsey.
36:51
I think, was a publication by McKinsey or garden.
36:53
I can remember he was McKinsey. That says by 2025, 40% of ml
36:59
models in production will be using Auto email and I was like,
37:02
okay, if by 2025 this will be, is going to be 14%, why not?
37:08
Let's bring 2025 to 2020 and that was where I started.
37:12
CD4 also a man and I'm a great fun.
37:13
Of also email to be honest, I'm not the best data scientist
37:17
which I was still telling some of my team members today.
37:19
I'm throwing my teams and I'm not the best at a scientist but
37:21
I what I would tell you is that how to think about solving
37:25
problem the problem with data science, I want to tell you the
37:27
best model to choose. What I will tell you how to
37:30
figure out the real problem that you're trying to solve.
37:34
And for me was Bring in 2025 into 2020 and that making that
37:39
really really making making that available.
37:42
So that was how I started. And that's how I go about.
37:45
Most of the things I'm trying to do with what I do with it.
37:47
Obvious, what are people talking about about the future?
37:50
Can we bring in the Conquering it into now?
37:53
I love that problem-solving approach.
37:54
That you take, right? Which is to say like, hey,
37:57
here's an ideal and state can we get there, faster?
38:00
Right, can we get there better? Can we get there faster?
38:02
Like how can I make that happen? Both, you know, personally and
38:05
professionally as part of a team?
38:07
One of the things that I want to ask you about before we jump
38:09
into the auto ml piece, which I think I would really love to
38:12
dive into, you know. You know, it's obviously meant
38:19
but we're in 2022 now, right? And it's like AWS has really
38:24
expanded its product portfolio. I think it's got like so many
38:28
different products. Now in any situation where, you
38:32
know, there's a sort of popular open source project, AWS is
38:35
basically forking it and bringing it into its platform.
38:37
Right? And it's smart.
38:40
It makes sense. Yeah because at the end of the
38:42
day it'll be a solves. A lot of problems for places
38:45
where Billing and, you know efficiency are the main
38:50
problems, right? Not necessarily technical
38:52
Innovation, but there's in the machine learning World,
38:55
particularly there is a heavy emphasis on best-of-breed tools,
39:00
right? So, around monitoring you'll
39:02
have a point solution tool. That are says, we're the best at
39:04
monitoring. I compute optimization, you
39:06
know, will say we're the best at compute optimization training
39:09
with the best training, right? How do you think about the
39:13
difference between the integrated ecosystem?
39:15
That either? Sufferers versus the sort of
39:18
best-of-breed model that the machine learning world has sort
39:21
of started to move towards like how do you compare and contrast
39:24
and think about strength and differences.
39:27
Okay, so this is my personal view first, and then I tell you
39:31
what, I think from the industry standpoint, my personal view is
39:33
that some of this Let's say focused tools.
39:39
Sometimes, sometimes I think personally and I'm, this might,
39:42
this might be wrong, right? I think they're overrated.
39:46
And my view is that if you talk to most companies most companies
39:50
are even still struggling to put a linear regression model in
39:53
production. And this is, this is, this is
39:58
the market. I mean, I mean I mean Germany.
40:00
I mean Europe, the and I can speak for that.
40:01
I'm not so sure about the US market but most companies still
40:04
struggle with putting linear regression.
40:06
Introduction and seeing how that feel, what that looks like.
40:09
And then on the other side, we have lots of companies.
40:12
They know we can help you monitor XYZ, we put layer this
40:15
on top of it, and all that, and all that.
40:18
That's my personal view. Not from an industry's
40:20
perspective. Right now is if I'm making a
40:22
decision between build or buy or leverage AWS or use something
40:26
that exists. First question, I ask myself
40:29
which is what I do every day for myself.
40:31
Personally, is this the best use of our time would if we Build
40:36
this built this on our own. Does this make us a different?
40:40
Does this differentiate of Competency in a competitive way?
40:43
If it doesn't, why spell waste time building it and then we
40:48
then go into those AWS solve it, or do we go into something
40:51
that's really. This is what they do.
40:52
Actually my and my first disposition is always evaluate.
40:57
If it'll be solved it and then do it compiles.
41:00
And if someone else does it better, and if someone else does
41:03
it better definitely, why not? I think when I went with 11 and
41:08
start the data API call for months, that's called back.
41:11
Then we have unlimited email flu.
41:15
And Emma flow was. I think when I attended Sparks
41:19
armies in 2018, hold when ml flow started got introduced on
41:22
all that hype around it and all that conversation around it.
41:25
And in one of the engineers at all, let's, let's evaluate Mo
41:27
flow and for me was okay, fine to get Engineers motivated from
41:30
other product from a program manager standpoint, you'd only
41:33
also need to give, they give them that guided autonomy rights
41:36
in here, let's evaluate this. Try this out and see and then
41:39
another, another set of Engineers.
41:40
Just I think, one of the engineer tried on stage makeup,
41:43
we just compared and realize that Take us a lot of time to
41:46
really get this integrated into the tooling that we have a
41:50
existence back then. So if we use Sage maker, we move
41:54
faster, even though we might not have as much fun seen, as always
41:58
much capability as Mo flow that point in time, but integrated
42:02
better into our ecosystem and this is what I look out for, is
42:05
the integration into the existing ecosystem, which lots
42:08
of people shy away from say, oh yeah it's Legacy.
42:10
We don't want to think about it. But that Legacy is what paid the
42:14
bills and we Still might be paying the bills to a reasonable
42:17
extent. So we really need to think about
42:19
how this integrates into Legacy Legacy couldn't cope, right?
42:23
And then evolved from there and I've spoken to some companies
42:27
where they say, oh yeah, we don't like, we have all our data
42:31
in either in need of us. But we prefer to use Google
42:35
Cloud because or Google vertex, because Google vertex is the
42:39
next big thing. There's a lot of hype around it
42:42
and what they don't think about is that in my You learning
42:46
because when you're building machine learning, if we are
42:48
designed machine learning algorithms.
42:50
The reason why magellanic machine learning algorithms are
42:53
not so suitable, accessing relational systems because they
42:56
consistently access relational systems Administration latency.
43:01
And when I see teams and only are, we will move our machine
43:05
learning infrastructure to Google Cloud, but then Allah
43:09
applications, I in an AWS and S3 and like, have you really
43:14
thought about the cross cloud data transfer costs, and what
43:19
that would what that means for your applications or other means
43:22
for what you want to design. Most times, I don't think about
43:25
it, just oh yeah, we need to go to Google Cloud, but you need to
43:27
really think about it from a strategic point of view, which
43:29
is we want to go multi-cloud as it Right.
43:32
What does this mean for us from an application design
43:35
perspective? What does this mean for us from
43:38
machine learning and other aspects of data AI on other
43:41
things that people put into consideration?
43:43
And this is how I think about is that integration into the
43:46
existing ecosystem. So she said something there that
43:49
I think you kind of just skipped over but I caught it and it was
43:55
to get buy-in from the different stakeholders for when you were
44:01
I'm planning out how to or which platform to use your thinking.
44:07
Well should we use ml flow? Should we use Sage maker and
44:10
someone wanted to test out Mo flow?
44:12
You said yeah, go test it out, go have your fun with it because
44:16
ultimately we're going to see but it was a little bit like
44:19
that's a nice little Jedi mind trick to get people stoked about
44:23
whatever the choice is at the end of the day because they felt
44:27
like they were part of it. So yeah, definitely definitely.
44:32
So I like that, that's a nice. Nice one for me to remember to
44:36
now let's dive into this continuous development for auto
44:40
amount because I want to know what it is.
44:42
First of all, Okay. Why did he thought so what it
44:47
is, is basically how it started. I think it started from that
44:50
Mackenzie article. I came about and then I just
44:54
start back and just told myself how many Auto email use cases
44:57
are in production. Most times you see and no
45:01
offense to data scientists. Right?
45:03
Is that they feel that oh Otto ml is going to take our jobs
45:07
away. To be honest.
45:10
It's not taking your jobs away. It's only making you more
45:14
efficient and I will I'm not sure if I still have that, I
45:17
don't have it anymore. I think it's on my slide here.
45:20
This particular diagram but I what I do to my internal
45:23
stakeholders back then or take out a scout.
45:25
But I said if you're building an AI platform, that AI Plus for me
45:29
is getting for two markets, which are two internal markets
45:32
or to internal personas, which is the data scientist, which you
45:35
want to enable them to build. Real different business
45:39
differentiator with analytics or Advanced analytics and you want
45:43
to look at another long tail, which is this Engineers that are
45:47
really a male and fuzzy as that are really interested in ml and
45:51
what he can do and really speed up their own time to Market.
45:54
And that's what I think. That's where I see that auto
45:58
email comes in, which is that those ml enthusiasts are not.
46:01
They're not, they don't really did it.
46:03
They didn't really do a lot of statistics and school, they're
46:05
not. Super crazy data scientists,
46:09
they are just Engineers like me. That want to leverage machine
46:13
learning to do stuff and then around that is most times.
46:19
Like every other machine learning initiative, right?
46:21
Is he still the model works on my computer?
46:25
So how can we take Auto email and put it in production
46:28
relatively easily? So there's the fact that you can
46:30
specify the data sets and there is the fact that you can specify
46:33
the features that you want to use.
46:35
You don't need to think about feature engineering which I
46:37
think where this is where data scientists really really Excel
46:40
when combined, with domain knowledge, I don't have, I just
46:44
know that I know domain knowledge because I know my
46:45
application but I know what I want to predict.
46:48
I About the techniques and feature engineering, pandas,
46:51
numpy, whatever I need to use in between.
46:54
Can I do something? And that's where I started from.
46:56
Okay? I tried a couple of libraries
46:58
and I realized thinks he's making autopilot was really
47:01
interesting, so that's the first part.
47:03
I said, oh yeah, let me try this one out and then I saw oh wow,
47:06
they are curious is really good. Okay, it might solve a problem.
47:11
Okay, so as an engineer but to cloudformation experience, how
47:16
can I spin off infrastructure that would do this right?
47:18
Nativity easy and this was basically how it started.
47:22
And I remember I just had my son back then and he was always
47:25
crying out. Sometimes I won't have to carry
47:27
in while still building out this framework.
47:31
I think about that was how it started was.
47:33
Simply I wanted to simplify my own experience with putting able
47:36
to email into production. So cool.
47:40
So we've got a few last rapid fire questions for you.
47:44
The first one that I want to ask is what was the last book you
47:48
read. The last book, I read is
47:51
empowered by Marty Kagan. I read multiple books at the
47:55
same time. As it's the last one is in part
47:59
by Marty Kagan and there's this forgotten.
48:01
His name is, I think is a Stanford Professor.
48:04
He talks about power dynamics in the workplace.
48:08
Mmm. That's nice.
48:10
Fat is named Jeffrey pfeffer. Yes okay.
48:15
Cool. Yeah, yeah.
48:17
Next two books. That the last two weeks, our
48:20
next question that I had for you was what was the last bug that
48:25
you smashed? That was one recently.
48:30
So most times right now, I don't write look called because I lead
48:33
teams are write code. I think the last book that I
48:38
smashed was about DBT. So we're introducing DBT.
48:41
The did a platform and then, I some issues with it, but it was
48:47
more working with the engineers to solve it.
48:49
Let me solving it. But yeah, I think it was really
48:52
that to DB T and D BT. DBT or glue.
48:55
Yeah. Nice.
48:58
All right. What?
48:58
One piece of technology you're bullish on.
49:00
That would surprise people. So not the ottoman Mel because I
49:03
don't think oh it's surprise anyone.
49:06
What's something that Surprise people.
49:10
Hmm, interesting. Hmm.
49:22
This is really good. This is really good.
49:24
I think, right? I think that if it obvious and
49:29
I've been very critical of them internally and even I would say
49:32
here, if they are really serious about AWS glue, I think it can
49:36
be a technology that would surprise and everyone I give you
49:39
an example. So when it builds, it obvious,
49:41
Glory was just oh yeah, this could do a couple of things and
49:44
then they built ETL on top of it.
49:46
And then they build interactive sessions, which was quite Mila
49:49
to data bricks there are Explorer star Derek's posture.
49:53
So, I did a test, right? I started I didn't mix cluster
49:55
which was the Community Edition, and took about three to four
49:58
minutes to start a cluster. I started the same something
50:01
quite similar on glue with include interactive sessions.
50:05
And it was, it took about two minutes to run to get started.
50:10
And I think if they are really serious about it, right?
50:12
Which I've told them quite a number of times, it would change
50:16
the game. And most people don't know,
50:18
because, right now we build It's something with DBT close glue
50:23
and it's the experience was written.
50:25
It's someone was telling me in the AWS Euro communities, that
50:31
if it obvious clue Community was really, it'll be as glue
50:34
internet was really serious about what they're doing and
50:37
Community engagement. Most of most people would not
50:40
need to use things like snowflake.
50:45
I actually think that is that is such an interesting choice, I
50:48
gotta say dimitra's. That's the best answer.
50:50
The question that we've gotten so far because people usually
50:53
pick something like flying cars or stuff like that.
50:57
But that is actually a really interesting hot take in our
51:00
industry because I have used. Aw, just glue.
51:03
I have been frustrated by it obvious clue, but I totally see
51:06
the potential now that you mention it.
51:08
Yeah, last one for. Yeah, I got is how do you want
51:12
to be remembered? That's a really good.
51:18
I want to be very rich. I want to be known as very rich,
51:23
that's one. And then It's not solved.
51:27
A lot of problems that the environmental problems, right?
51:31
Sometimes I'm really politically inclined, you know, if my
51:34
country, I'm from Nigeria and I want to see change and I want to
51:38
be remembered for even though I'm a technical person.
51:41
But then really remembered for some really huge, a
51:46
environmental impact. And this was for me, people talk
51:49
about this green stuff. Oh yeah, we need to go green.
51:52
But then I picked up the book by Bill Gates on climate.
51:56
Think about last year and really struck me how Simple Solutions
52:01
can have lasting impact on the world.
52:04
And that's, that's how I want to remember, Simple Solutions, that
52:08
can have lasting impact on the, on the world.
52:10
And then, at some point we see visit Mars.
52:13
Also, there you go. Shoot for the stars van.
52:16
I love it. Yeah, we'll just go through the
52:19
infrared Rays from the James Webb satellites and go 13
52:24
billion years back in time. Mind under a spot back in time.
52:29
Yeah, those are so cool, man. I love that.
52:33
Well, this has been really special and I appreciate you
52:35
coming on here. I appreciate you being very
52:37
candid with us. I learned a ton.
52:41
Thank you, thank you. Thank you a lot.
52:44
Thanks a lot. You ask me quite a number of
52:45
questions that most people have not heard.
52:47
Had people ask me in a while, I'm not even hurt them.
52:50
And I also made me think a lot, I'm very happy to hear that.
52:54
Like I have to say this was One of the conversations I've most
52:58
enjoyed. So thank you for joining us.