Latest / Agentic Conversations (formally mlops.community) / Managing Machine Learning Projects // Simon Thompson // MLOps Coffee Sessions #128

Transcript

Managing Machine Learning Projects // Simon Thompson // MLOps Coffee Sessions #128 · Agentic Conversations (formally mlops.community)

0:00
Okay so I'm Simon Thompson. I'm head of data science for gft
0:05
Technologies in the UK where Financial technology specialist
0:09
consultancy at 10,000 people. We work in all the big
0:12
investment Banks and I lead the machine learning practice there
0:17
and I like my coffee black and strong Small Talk.
0:23
Small talk stuff. Yeah, we get right to the big
0:30
Don't lie. This is not a conversation of
0:34
small talk this one. I mean Abby, it's great to have
0:37
you. Here you are a new first time, a
0:41
first-timer co-host and so it's probably only right if we let
0:46
people know who you are, what you're doing here with us today.
0:49
Hey everyone. I'm happy.
0:50
I am. She's having internet and I'm
0:53
also helping out the podcast at and Bob's Community.
0:58
It's fantastic to be able to talk Talk to everybody, prior to
1:01
this. I is a routine.
1:03
I think researcher as that UCLA and a couple of other things.
1:09
So, being at the field for sort of, I don't know about six seven
1:13
years, amazing. And today, we talked with Simon
1:17
Thompson, who just came out with a new book.
1:19
We talked all about that and so much more.
1:23
I mean, we got into just like you can tell that he's been in.
1:30
In learning and especially like machine learning in the industry
1:33
for a long time because his thought process was so clear on
1:38
the question that you asked him about, what do you recommend
1:43
when you're starting a team and how do you build out the team?
1:46
Do you get data Engineers to get data scientists?
1:49
And he said, no, no no first figure out how much money the
1:52
project is going to make and then figure out the rest which I
1:56
felt like was this is like the voice of reason.
1:59
It's wisdom. Adam experience all bundled into
2:02
one. I loved it because that's that's
2:04
usually something which is missing and the community.
2:07
There are there people talking about things from an engineering
2:10
perspective. And yes, there are people who
2:12
say we should talk about business problems and as such,
2:16
but it goes more. So towards understanding the
2:19
customer understanding the scope of the problem instead of just
2:22
understanding how much finances the financing is this project
2:27
going to require and not just that but how much Are we going
2:30
to make out of this project? Yeah.
2:32
And how all of that will then dictate the size of the team?
2:37
You need the type of Team you need.
2:40
Is it going to be? Are we thinking that we need
2:43
something that is going to be highly available online?
2:49
Then we're going to look at it like this but that highly
2:52
available online thing. It needs to be very tied to
2:57
business objectives and the ability to make money.
3:00
You. So that was one huge one.
3:01
It always stands out to me when we get guests on here, that are
3:06
very focused on that because it shows that they understand.
3:11
They've seen it too many times. I think we're a project will get
3:15
started but then it kind of Peters out and they've probably
3:19
heard too many times that cliche quote of like, oh, 80% of
3:23
models, never make it into production and so they don't
3:27
want that to happen. He also, Is really cool when we
3:30
got into the idea of how it differs between researchers and
3:36
Industry on the motivations. That each one has so many things
3:41
that on on the talk was also the part very, he is started talking
3:46
about the differences between the healthcare industry versus
3:50
the finance industry and how much does the accuracy met
3:55
important things? That one should be looking for?
3:57
Because that's not often talked about They sort of just gives up
4:01
one example in space, trust is reading point, but when it comes
4:05
to somebody, there's far too many Industries and there's new
4:09
one, dictates one rule that dictate sort of every single
4:13
industry. So while yes that's important.
4:17
But then there are so many other Industries where the focus is
4:19
different. So, little background on Simon,
4:22
before we jump into this conversation, he's been doing
4:26
machine learning since 1994 tease.
4:30
Read wrong. All kinds of different projects
4:32
when the EU and the UK with the government you can government
4:36
funded research projects and right.
4:38
Now as you mentioned he is working at a technology
4:44
consultancy company called gft and it mainly Works within the
4:50
capital markets in the financial sector.
4:52
Let's get into this full conversation and if you are not
4:55
subscribed to the email newsletter that we send out,
4:59
we've got three Different email, newsletters, go jump on it.
5:02
Will leave a link to that in the description in case you want to
5:05
join us and hear more from the MLS Community.
5:08
Also joining, this lack also, join the slack.
5:12
Also join the, well, the slack gets a ton of people in it every
5:16
day so maybe not, we don't need anybody to join the slack, but
5:21
if you want to give us a review on iTunes or Spotify or wherever
5:25
you listen to your podcast, that would be super helpful for us to
5:29
reviews and Ours, and all the best.
5:32
So thanks everyone for listening.
5:33
Let's jump into this conversation.
5:40
Simon. Good to have you here man.
5:43
Nice, nice to come. Nice to be here.
5:45
I know it's been a long time coming.
5:47
I know that I reached out to you like probably like six months
5:50
ago, when I saw the book that you are writing or have written
5:54
was coming out. And so wanted to get something
5:57
in the books, it's written, it's out, and it's all it's
6:00
available. It's in the Early Access program
6:03
right now. So awesome.
6:04
We're it should be printed before the end of this year, but
6:10
you can get it. Electronically because getting
6:12
things printed takes a while. And so we're going to talk all
6:15
about that and we are going to get much more into the book, but
6:20
we should probably start at the beginning and you have one of
6:23
the coolest things that you can say, on your bio that I have
6:27
seen in a long time. And that is you've been doing
6:29
machine learning since 1994, when I let's see, I was like,
6:35
seven years old in 1994 and 1998.
6:42
And I was I was 20 something. And I won't say exactly how many
6:47
20s because it's embarrassing and by no means calling you all
6:52
well, you know, you have to face facts.
6:54
It's just, it's just life, is hard, is it happens to you and
6:58
then, you think, wow, yeah, that's when I started.
7:00
My PhD was January 1994 and I had actually done some machine
7:05
learning before that I did some machine learning.
7:08
Probably started 18 months before that.
7:10
I did a wildly ambitious underground project which was to
7:15
try to learn prolog Clauses using a thing called genetic
7:21
programming and search. That was popular, then which
7:25
didn't work, because obviously, it's probably still be on the
7:29
state of the art, but I was a kid and I thought, oh, I know, I
7:33
know what would be cool if we could, if we could solve some
7:36
problems by just specifying them?
7:39
And then learning the Lucien, I mean, talk to us about growing
7:43
up and how you had this inclination, towards computers
7:47
and engineering? And all of that, not so much
7:50
towards towards engineering more.
7:52
I'm was more interested in art, actually, when I was younger and
7:56
maths, I was always very interested in maths.
7:59
But then, what happened was, I was very lucky, very, very
8:02
lucky. My dad went on a course of work,
8:06
and on the course, they had a computer and they had Sinclair
8:11
zx81. And they showed all the people
8:15
on the course, you know, what computers can do.
8:17
Because it's kind of like, you know, this is going to be the
8:19
future in those days and it tells you, you know, when people
8:23
come on the Telly and they say, this is the future, you should
8:26
listen because it was and he came, he came home, and he said,
8:31
we've got to get a computer and that then we did.
8:36
And, and of course, it was just so interesting.
8:40
Ting. Because we did not have Netflix
8:44
then and it was so interesting to have a zx81 and to be able to
8:51
write programs on it and it did things, you could make it do
8:56
things. And that was that was the thing
8:58
that really fascinated me. What was the first thing you
9:01
actually wrote on that computer has a like writing is Colonel
9:04
because I know a large Computing, your start out
9:07
working on basic so had it. Had a basic interpreter, or It's
9:13
featured. So, it had one k of memory when
9:16
it came one kilobyte. And then, for my Christmas
9:18
present, take got a ram pack, which you stuck on the back of
9:23
it, and it expanded it to 16k, and 16k, you could do a lot in
9:28
16k. Wow.
9:31
Yeah. And it and it had, but it had a
9:33
basic interpreter. So you could programming it with
9:37
bytecode and I did a bit but not very Rematch because programming
9:42
in bytecode is incredibly tedious and error-prone.
9:45
It's really painful. As anybody's done, it will tell
9:47
you. The thing I wrote, first of all,
9:49
the first thing was hello world, you know obviously just and then
9:53
put a loop around it. So you had I will go to town and
9:56
it fills the screen with with hello world and that was
9:58
amazing. And then started to write some
10:01
games and it was really, it was a social thing to try and make
10:04
friends. What was the plan?
10:07
So you did your betters? Two masters, PhD and everything
10:12
for you. I went straight to PhD.
10:16
I was very lucky. And well, I was lucky and
10:18
unlucky. I got accepted into a master's
10:20
program, that was Masters and PhD.
10:22
And then they had a funding crisis, and they unaccepted me
10:27
and I was I was quite miffed and I spent some time looking for a
10:32
new, a new gig, and I tried out for a bunch of jobs and I got a,
10:37
I got a job, but before I could start, I got accepted on to a
10:41
Ph.D program and it was basically, I just I think I went
10:47
to the interview and I wasn't fat frightened about it.
10:52
It wasn't so important because I already had another job I was
10:55
going to go off and be a software engineer down the road,
10:57
and I just thought I'd go along and see if it was interesting.
11:01
It was so then you've obviously seen a lot of changes especially
11:08
in the machine learning field. Is there anything that jumps out
11:13
at you when it comes to unexpected changes?
11:19
Everything, everything is unexpected to me.
11:22
Maybe, I'm just like a Taoist or something, just constantly
11:28
surprised by every Innovation and it's because it's because
11:32
everything. So out of scale, now, you know,
11:35
the, the the idea of 200 billion parameter models is it just
11:42
doesn't make any sense at all, and it's just meaningless words
11:47
to me. And I can't, I can't imagine
11:50
What that really is like, although I use them and and just
11:54
the resources that are put into creating models is
11:59
extraordinary, you know, the thousands of gpus to create a
12:03
model. It's just bizarre unimaginable
12:06
to somebody from my era, I think.
12:12
Yeah, but so completion Ali. You've seen a lot as an how
12:17
computers have served grown in terms of Memory in terms of
12:21
computability and everything, just the ram as well.
12:24
How have things changed on an infrastructure level according
12:28
to you when it comes to AI, Well, I mean, you know, I when I
12:32
did my PhD, I wrote use that and compile it on my machine and
12:38
that, you know, that, that, that maybe that gives you a flavor of
12:40
how different things were, you know, it was.
12:43
But maybe, maybe probably The Cutting Edge.
12:45
Ain't if you're doing a PhD in machine learning.
12:48
Now, I guess that you probably are still coding away in in
12:52
Cuda, right? Probably you you're having to
12:57
Wrangle really unpleasant problems to do with gpus.
13:00
Talking to each other or something.
13:02
I stay away from that. Now, as much as I can, I
13:07
suppose, I suppose what's happened, is it scaled right
13:10
there? We had almost impossible,
13:13
infrastructure problems which were at the level of putting
13:16
together. 5 workstations over an Ethernet Network and making them
13:21
exchange messages to do. Distributed computation.
13:25
And now people have got these Indescribable problems of
13:28
putting together, you know, 500. Gpus over a Data Center and to
13:34
make them work together to create a trillion trillion
13:38
parameter scale model or something.
13:41
How does things are used in terms of Open Source?
13:43
Because right now, if you're looking at the field, we have so
13:46
many open-source library in so many so many communities, sorry
13:50
for working together solving the bigger picture problems were
13:53
think. Sometimes we were Distributing
13:55
back in the day and visit sort of obvious that there will be
13:58
such massive communities companies.
14:00
Didn't open source programs when it comes to pushing the free to
14:03
forward it. Well, I mean, Ross Quinlan.
14:08
I mean, it wasn't open source. He sold his book with a disc in
14:11
it, right? But I mean I wrote to him a few
14:14
times and he obviously didn't care what I did with this code.
14:17
Really I think so long as I didn't make a million dollars
14:20
out of it but I wrote him and said, oh I'm using it in my PhD
14:23
and he was like yeah, great you know just go for it as I
14:27
remember a probably now, sue me but I think he still I don't
14:30
know if He's still alive, actually, but I benefited from
14:33
that availability and I suppose I was on the, the beginning of
14:38
that movement that people were, you know, in those days there
14:43
was going to and Richard stallman had started doing
14:48
things and people were quite open.
14:52
I think that probably I was very lucky that people were extremely
14:56
accessible that whenever I asked somebody at dumb, Jen, they had
15:02
time for me and they would, you know, people were really
15:04
surprised that that I was interested that there was
15:06
somebody else working on what they were doing and they wanted
15:10
to help and people were really generous to me with their time
15:14
and openness and and helping me solve problems and I wouldn't, I
15:21
wouldn't have been able to get anywhere without that from a
15:24
community, but it was probably more informal and probably less
15:27
stress than a days. Because I think that the problem
15:29
now is People do something in open source and either nobody
15:33
uses it. Or people expect it to have
15:36
commercial level of support. And I ran an open source
15:39
Community for years and years and towards the end of it, it
15:43
got distressingly difficult, helping people dealing with the
15:48
request because the expectations just went through the roof.
15:51
What, what people should be able to get from you.
15:53
Yeah, and, and, and sometimes I do sympathize with a few of the
15:59
different. All behaviors, you see?
16:03
Because I think somehow people end up depending on a piece of
16:08
Open Source software and then they're under tremendous
16:11
pressure because of that then, and and it's awful for them.
16:15
And I really empathize with that.
16:17
And I, the other one is just the difficulty of maintaining a
16:26
thread with an open-source maintainer to resolve something
16:30
really. To the end which I think is
16:31
extremely frustrating. Where, you know, sometimes you
16:35
report a problem, you reproduce it, you show the code or
16:39
whatever, but then you get pulled away to do something
16:42
else. So let's change gears a little
16:44
bit. Here you mentioned before that,
16:47
choosing and running the algorithms when it comes to
16:50
machine learning is only a small piece of it.
16:54
And I think that's very clear and it is something that we see
16:57
over and over again with that machine.
17:00
Learning the high-interest credit card, debt of machine
17:03
learning paper when that came out and it showed the different
17:05
boxes and how the models are just a little sliver of that.
17:10
And then you have all the infrastructure around it and you
17:13
also mentioned like setting up an organizing the project which
17:18
doesn't really get put into that infrastructure piece that came
17:22
out of that Google paper. What exactly did you mean by
17:26
setting up these projects? Well obviously you've got to get
17:34
the right team for your project and that's a big issue.
17:39
Almost nothing anymore. Gets done by one person.
17:43
It's pretty unusual in my experience of the minutes anyway
17:47
and and it's too dangerous in a commercial context, right?
17:50
You can't really have if somebody's got one person
17:53
working on something. It's you're pretty well doomed
17:58
because We'll have holidays, you know, get Ill.
18:04
Get a new job, right? And, and you're just not going
18:07
to deliver. So you've got to have a team.
18:10
It's a group of people working together.
18:12
It's got to be the right team. So finding them is a priority.
18:16
I think, breaking down the project understanding what it's
18:22
for. I had a good lesson today.
18:25
Collaborator was taking a team that I'm managing right now.
18:29
Now and they we hadn't got to that even though I have
18:33
obviously worked very hard to get this project all set up,
18:36
there was something that we hadn't got to that point and I
18:39
think being able to get under the skin of the project, and
18:41
really have a shared understanding of everybody about
18:44
what it's about what we're doing here and why is really crucial?
18:48
I think, getting at the data is a hell of an important thing,
18:53
and, you know, nothing about them L project until you seen
18:56
the data and nothing at all, and You know, you're not going to
19:01
get it very easily, you know? And and even when you do get
19:05
technical access to the data actually it's a big process to
19:08
get into it and to understand it and delve into it properly.
19:10
And again you know today I was in a meeting.
19:12
I was talking about why it was that.
19:16
I didn't think that we could really say we had sign off about
19:22
the data, the the data fields we were using in a modeling project
19:26
because the reality is that when the Idols are done.
19:30
We may well, make a whole load of discoveries, about the data
19:35
fields were working with that. We can say.
19:37
Okay, you know, this looks like the right set of data fields.
19:41
So I think things like that and I think also agreeing a way of
19:45
working with your team. So creating a culture in the
19:48
project is something that people miss out, and that's a hard
19:52
process. But you have to get to a set of
19:56
behaviors. A set of expectations.
19:59
Set of greed outcomes, agreed ways of taking on bits of work
20:04
and a lot of its standard, you know, will pick up a ticket from
20:07
the, the ticketing system, whatever you use, God forbid,
20:11
jira, but, you know, maybe maybe it's jira and you pick up a
20:15
ticket and you have to sign it off.
20:18
And, you know, what does the documentation that like, what
20:20
does it mean? Have you done a code review?
20:23
What does a code review mean in this project?
20:25
What are our expectations during a code review?
20:29
When you say, You've done some testing, what does that mean?
20:32
You know what are our expectations around using the
20:35
test data? Because you can consume it and
20:39
then it's no good because we've, we've optimized everything on
20:42
the test data, you know, we've overused it.
20:46
So you have to figure that all out and it's different from
20:48
Project to project because the Project's themselves are
20:50
different. Right?
20:52
So that's setting up a mod a project I think.
20:56
Yeah. So let's say there's a
20:59
hypothetical organization that is just starting out to build
21:03
their designs team. How do you think they should go
21:06
by? Which is how many people should
21:08
they start out with? Is that a decision you make?
21:11
Ideally after you've already looked into the data which is
21:15
what kind of skill set to you actually require before.
21:17
You start thinking about all of those things, I think you need
21:21
to think about how much money you're going to make first,
21:24
right? So I think I think there So many
21:27
data science project start off with a challenge that isn't
21:32
really connected to the nuts and bolts of the business.
21:35
And it's always going to be disastrous for everybody
21:37
concerned because, okay, your sponsor has a certain amount of
21:42
rope that they've got because the CEO loves them, and they've
21:45
been so successful and they're doing all these other things.
21:48
But your sponsor is going to move on at some point, but that
21:51
that has to be that you have to you have to get to that answer.
21:56
Where is It that the target is what are we going to do?
22:01
And and then that will tell you, how many people what kind of
22:05
people because you know, if it's going to be a massive fishing
22:09
Expedition, through a vast amount of data, we need data
22:12
Engineers. Right?
22:13
With we don't need too many of those data scientists.
22:16
Maybe we'll need one or two. Well, I won't need too many of
22:19
them. Will we need many machine
22:21
learning Engineers. Will maybe not at first right?
22:24
But then again, it can be the opposite.
22:27
It can be that we know that there's you know we're going to
22:29
be producing 2,000 models here. There's a well-known models of
22:34
people you know we've had the consultants in their bills and
22:37
things before. We can see that there's 2,000
22:40
models, we need to get rid of our regulation risk or whatever,
22:44
okay. You're going to need a lot of
22:45
machine learning Engineers because you're going to have a
22:48
massive production bottlenecks. Oh it's that's what drives it
22:53
definitely the business drives it You've seen a lot of data
22:58
science, teams in Industry, as well as Equity be aware of the
23:02
big challenges that I faced when it comes to the management side
23:06
in both of those teams and what's the code difference?
23:12
Is definitely motivation. I would say, I think in Academia
23:22
I think that people are very self-starting and their
23:24
motivations very intrinsic. And I think that in Industry as
23:30
much more extrinsic motivation in the team, the team are
23:33
looking to expand their skills. Their expect looking to get
23:37
personal growth. I think that's one of the most
23:40
Things, I see in teams people want to go into a team, we're
23:43
going to have personal growth that and, and if they feel like
23:47
I've not got personal growth, they will leave.
23:49
And, and they'll go to the next place that gives some personal
23:51
growth, and you have to accept that.
23:53
So, thinking about, the data teams and and the skills that
24:01
you need to develop along the way and that growth that you're
24:04
talking about, how do you recommend people go about not
24:10
only growing in the way that you're saying or finding teams
24:13
where they have that ability to grow, but also what you said
24:19
beforehand and making sure that the data scientist or the
24:24
machine learning engineering, Understands the fundamental
24:28
value of how to get to the money and how to prove out there, use
24:33
case and prove out the benefits and the ROI of the use case.
24:41
So on the on the second bit, I think that it's a long process,
24:45
it takes, it takes quite a lot of work for four people in teams
24:50
to be able to understand for a particular business, where the
24:54
real value is going to be because you have to really get
24:57
under the skin of it and it's sometimes sometimes I find
25:02
people don't really want to do that.
25:03
Work will look after that so people have got to be open and
25:09
inquisitive, right? And and inquisitive over a long
25:13
period of time. Humble about what the smes are
25:18
telling you, you know, why is it that they don't think it will
25:22
work and you should try and find the thing that they think.
25:24
Yeah. Maybe that one will work right?
25:27
Because yeah. That often then there's
25:29
something they can't articulate which which is the blocker,
25:32
right? That the data is bullshit in it
25:36
in it in a deepen and particular way, that can't really be
25:39
articulated. It.
25:40
But when you get to that understanding of why the model
25:44
isn't working, suddenly they'll pop up and they'll articulate it
25:49
perfectly. And so I told you this at the
25:50
beginning of the project, right? And you know, they've been
25:55
looking for that articulation for 3 months, but they have, you
25:59
know, it's really hard for them to do it until it's put on the
26:02
screen and they say, that's what I said, you know, that's what we
26:05
were trying to tell you. So yeah, I think I think that
26:09
inquisitive Nature. And and curiosity about the
26:13
business you're working with to try to pin down, what is really
26:18
going on? What matters to the people who
26:20
are going to be using your model.
26:22
And I think that's really important.
26:24
You know, it's something that's coming out more and more in the
26:28
wider world and machine learning.
26:30
I think obviously, which is it. The people who are using your
26:34
model, the people who are going to be impacted by your model,
26:37
are the most important people in the mix.
26:40
And that, That's because your model might chew them up or it
26:43
might make them miserable and you know there are two ways to
26:49
be very unethical. You can be very unethical by
26:52
knowing that you're going to chew people up with your model
26:54
and not caring and it just building it and say I don't
26:57
care, I got paid that's pretty bad right?
27:01
But what often people don't understand is that not being
27:05
curious about the people that you're doing this too and not
27:09
inviting them into the He says, I think he's equally equally and
27:14
ethical, right? I love that you point out the
27:17
conversation towards ethics because then it brings to the
27:21
entire thing about monitoring deployment and thinking skills,
27:26
thinking, or building a product with scalability in mind when
27:30
idea science team is starting out, how does it really balance
27:34
between shipping words, the skill ability?
27:36
Which is addressing that? There's a big demographic that
27:39
we're dealing with It and not be.
27:41
So I had I think I think there are two things that are
27:45
conflated there, right? So well maybe maybe you, maybe
27:49
you push back in a second and tell me I'm wrong but I'm
27:52
thinking there's a bit of a conflation because there's the
27:54
thing about monitoring and governability and management for
27:58
your models and and the system, you've integrated the models
28:01
into and then this is scalability, right?
28:05
And I think that the scalability brings a whole bag of issues
28:10
with it. That people love to deal with
28:13
and to focus on and to talk about and then they never get
28:18
used right there because your model is just you know, never
28:25
gets hit with the imaginary requirement that you got.
28:29
I mean obviously if you know your model is going to get run
28:32
10,000 times a day, you must make a model that can be run
28:36
10,000 times a day and it must not cost $50 to run each time.
28:40
Time. Because if it does, you're going
28:42
to go bust, you can't have that. You can't spend half a million
28:44
pounds a day on 10,000 customers, forget it, right?
28:47
That that's ridiculous. So, obviously you have to think
28:51
things through is like that but I would say deal with those
28:53
requirements in the moment and then when when you have the
28:57
first world problem, where people come to you and say it's
28:59
not scalable well then you can spend the money then to try and
29:02
make the thing more scalable, right?
29:04
But then on the other side is the the manageability and the
29:08
governability and and the Usability of the model, the
29:12
transparency of it, they explain ability.
29:15
And if you don't put those things in it's useless, right?
29:17
I mean you might, you might have it in production, you might be
29:19
running it all that. You're doing is building up a
29:22
huge, a huge debt for the future and that, that can we seen it?
29:27
It comes do very suddenly and very nastily, you know, people
29:30
arrived at the door and say you have committed an offense and,
29:36
you know, we want we want reparations.
29:38
Now that thing has to be turned off Off, right?
29:41
And there's a huge amount of damage as a result and that
29:45
that's no good, right? So I always think you've got to
29:50
have those levers which allow you to manage the thing in
29:55
production and to be able to show people not only how it
30:00
works but why it works in that way, what would the decisions
30:05
that went into making it work? That way?
30:08
I think that's fair. That's it.
30:11
I was remember Twitter, right? So, there's the Twitter's,
30:14
probably the most scaled thing in the world.
30:16
But when I think it was a guy called Blaine cook was the first
30:21
sysadmin at Twitter, the first developer and he built it all on
30:25
mice ql, right? So, it was just a mySQL database
30:30
for quite a long time, and I think it wasn't until quite a
30:34
long way into 2009 that they really had to rip it all out and
30:40
Old much more specialist infrastructure to deal with the
30:44
traffic they got and that, that didn't kill Twitter, definitely
30:50
not. And it reminds me of what we
30:52
were talking about before with Uber and how boober was built by
30:58
Dev shop in Mexico. And the story goes, I don't
31:01
know. I've never looked into the Uber
31:03
code base, but I've heard that you can find some comments in
31:06
Spanish still and so I really like that idea.
31:10
Idea of you want to be pragmatic about the traffic that you're
31:17
actually going to get. And how much do you need to keep
31:23
that? Scaling question in mind and
31:26
keep that ability to scale in mind and how much of a priority
31:30
is because it reminds me of a friend of mine who talked about
31:35
how at every conference basically in any talk you're
31:39
always Hear someone ask? Yeah.
31:42
But how does it scale? That's like the go-to question
31:46
that you'll hear in the conferences.
31:47
And so sometimes that is a very valid question but a lot of
31:52
times it's just the cop out question that every engineer is
31:56
going to ask. Yeah.
32:00
And and you know, you may never need it.
32:03
Let's just be open to the idea. I think he'll piece of powerful
32:08
idea, but when it comes to to accountability and manageability
32:12
and governability, you are going to need it.
32:14
If it's going to production, it has to be manageable, you can't
32:18
inflate unmanaged and monitor software on anyone.
32:23
So let's let's go into that a little bit.
32:27
Are there certain things that you feel like with machine
32:31
learning? There's the must-haves and then,
32:33
there's the nice-to-haves, yeah, health.
32:36
So fancy algorithms are definitely the nice-to-haves,
32:39
right? So, that last and the scent of
32:44
accuracy is a nice to have often, you know, your business
32:48
sometimes for a business drivers are different from that, right?
32:50
So obviously if you if you You, if you're not want, not point
32:54
one percent of accuracy equals fifty million dollars a year,
32:59
right? Then you are going to spend a
33:01
lot of money on that note point, one percent of accuracy, right?
33:04
That that that's for sure often though.
33:06
That's illusory. It's not, it's not actually
33:09
true, it's not going to matter. There's a certain level of
33:13
performance, which is going to be successful for you if if it's
33:19
low-hanging fruit and you can just pull it way.
33:22
Ask that 99% or something terrific, but often people,
33:28
people optimized for quote, unquote accuracy, and do far too
33:33
much on that. And it's and I think I think
33:38
that it's it's just a symptom of not really understanding how to,
33:43
how to evaluate your products in the context of the business that
33:49
you're delivering it to. And I think that's The need to
33:53
have, which is the deep understanding of how they should
33:59
be evaluated and what is valuable and what is not
34:02
valuable about this product. And that's often a thing that
34:06
just gets left left behind and people are saying, oh, it's F
34:09
what it's got a great F1 school, right?
34:12
And and sometimes it can have a great F1 score, that's been a
34:15
miracle of technology and it's just not good enough to use,
34:19
right? And it was never going to be
34:20
good enough to use because You know as soon as you figure that
34:24
out you realize there is no model in the world that's going
34:27
to get me to that level of performance.
34:29
You know we need 90. You know I mean there's the
34:31
medical cases a pick one you know there are lots of cases in
34:35
medicine, where you've got to be almost absolutely Flawless to
34:42
dare to employ this thing in the field because he will kill
34:46
people, right? And After 10 minutes of looking
34:51
at the problem you realize yeah, I could do a really good model
34:53
on this, but it will not be almost Flawless, right?
34:57
It's not going to be as good as the consultant, he'll be good.
35:01
I could probably get a conference paper out of it but,
35:05
you know, I'm not I'm not going to participate in a project
35:08
where somebody's going to pick it up and try and use it on
35:11
patients. Right.
35:11
No way. And I think I think a lot of
35:14
time that that's the big thing that we we need to do more on
35:20
You know, work, I think Health Care is often one of the most
35:25
quoted scenarios that we discuss, especially when it
35:29
comes to accuracy or talking about the trust ability of the
35:32
models. How does it change when it comes
35:35
to different markets? So, for example, you're working
35:37
capital markets where the challenges are.
35:40
Do the same challenges transfer in other domains as well.
35:43
And equally as much, I think it's very different.
35:47
I mean, the classic one. Is people thinking that a model
35:54
that predicts price brilliantly is going to be really useful
35:58
right? When actually, what you wanted
36:01
to model the tells you when to buy or sell it because you you
36:06
and and in fact, I guess that's somewhat the same for medicine,
36:09
right? You don't people often say I
36:11
want something that's really good at Diagnostics, right?
36:14
And it's really good at saying when somebody's got cancer or
36:18
not, right? The reality is whenever I've
36:22
taught to oncologists, they said to me, almost every day.
36:29
I see people walking through my office door and I know what's
36:33
wrong with them before they sat down.
36:35
You know, I can just see it and and certainly, you know, five
36:39
minutes into the conversation. They're pointing at where it
36:41
hurts. I know, I know what's wrong at
36:44
that point. You know, and after that
36:46
everything is just in the in the margins of them checking and
36:49
Making sure and not mal practicing by assuming something
36:53
but they they're good at Diagnostics.
36:55
What they want is something that tells them how to treat the
36:57
patient, what what the what the course of treatment should be.
37:00
I mean that seems to be the consensus ayat same in in
37:03
finance. You know, it's very often not
37:06
you know the Traders are like well we know what price roughly
37:09
real prices is going to close out but should we sell it?
37:11
You know what will it be doing tomorrow?
37:13
What's you know? Is it a good idea to sell it?
37:15
Now that those are the kind of questions?
37:17
I think I think the other These, you know, the cost of its
37:23
misbehavior. So if we're doing models that
37:26
are looking to see is somebody misbehaving, you know, some sort
37:30
of surveillance thing. Well, if we say that Traders
37:34
doing something a bit suspicious and it's not true, it's okay,
37:39
right? They get checked out somebody
37:41
reviews, the case they say, no, it's normal as normal behavior,
37:45
stupid model and it, you know, it's irritating and I'd rather
37:50
that it didn't produce that false positive but it's not a
37:52
big deal. If we miss the guy who's really
37:56
doing bad things that is a big deal so we can't do that.
38:01
And you know especially if it's obvious right?
38:03
We can't we can't have a model that just doesn't check the
38:06
particular field that it should be checking because oh when we
38:10
did the F1 score you know that didn't matter right?
38:13
It's got to be it's got to be accountable.
38:16
We got to be able to say no it did check everything and so
38:19
Those those are the differences, I think.
38:21
Yeah, the bowls are highly regulated Industries, right?
38:26
And so, it makes sense that there is some crossover, but
38:32
then there's also a little bit of nuance there.
38:34
I want to change gears and jump into your book a little bit
38:38
because I know that we were talking about it beforehand and
38:43
we're going to tell everyone what we're going to do, but the
38:46
fine details are not quite turned out because I just
38:51
proposed this to sign them beforehand.
38:53
I said we'll give out some of his books.
38:56
We don't know how many books were going to give out yet.
38:58
The first people to comment, we're going to be giving you
39:01
free books and we want to get these out there.
39:05
For those who do not know the book is called managing machine
39:09
learning projects from design to deployment.
39:13
Let's jump into this. What was the inspiration behind
39:15
it? So what happened was That I did
39:21
a year as a visiting fellow at somewhere called the cheering
39:25
Centre in London, the Turing Institute and it was when it was
39:30
starting up and I got to talk to an awful lot of people.
39:34
I wouldn't otherwise have talked to about what they were doing
39:37
with machine learning projects in a bunch of different contexts
39:40
and industries. You know, people from
39:42
engineering, civil engineering and dot-coms and defense and
39:48
intelligence all sorts of interesting people were doing
39:51
stuff and I realized that the that some people had access to a
39:56
community of practice or have had access to a community of
39:59
practice for a long time. And there are some places like
40:02
the big, the big tech companies, right?
40:04
That there's a huge community of practice and there are people in
40:09
those companies who have spent many years figuring out all of
40:14
these wrinkles and issues and talking about it to everybody in
40:18
their Community there Lots of other people who aren't are
40:22
excluded from those communities of practice who have not ever
40:26
been in that community of practice and never will be able
40:28
to get into that Community Practice, because that's not
40:30
what they do. And I felt really bad for them
40:35
because talking to them, they were a whole load of stuff.
40:37
They things they were doing, which would just fatal, right?
40:41
Just just blunders and, you know, the kind of things we've
40:44
been talking about, I not understanding really the problem
40:48
and thinking that That, you know, certain things would be
40:52
easier to solve when they wouldn't be solved for, not
40:54
having any infrastructure to support their project or not
40:57
having a team, or having a weird construct of a team.
41:01
I could sit here and be smug about it and say, you know, I'm
41:04
a, I know how to do it and why don't you come, pay me some
41:08
money, but it seems a bit churlish.
41:12
And so I thought, well what, what would be appropriate and I
41:16
thought, well, try and write down what?
41:19
It is, you think would help people.
41:22
And that's where the idea for the book came from, was just to
41:25
say, you know, I've been in a community of practice for a long
41:30
time. I was very lucky about having
41:31
the people I had around me. I had a lot of good experiences
41:35
where people helped me and told me things at all.
41:40
Sorts of amazing institutions, and you should write it down and
41:46
put it there for people to talk about what are some of the key.
41:49
Cool conversations that you expect to come out of the book.
41:55
I think just people saying oh, that's not how we do.
41:59
We do it a different way or why did you think that's the right
42:03
way of doing it? That's not the right way of
42:04
doing it. You should do it a different
42:05
way. So that's those, those are the
42:09
cool, the cool kind of conversation side.
42:11
One. I mean, the point, the point is
42:12
to try to have that so that we do get the good practice out
42:17
there that we have, you know, I would like the to be a widely
42:21
agreed accessible. Point of good practice for ML ml
42:27
people that they could actually say this is this is the these
42:32
are the right sort of things to do.
42:34
And I think we're getting to the time now, when, when it's
42:36
starting to be possible to have that, because, you know, there
42:40
is, there is this this available technology now?
42:45
That's such a great point. This is the time that it feels
42:50
like more information is out there, there's more maturity.
42:55
There are communities that have sprung up and the good practices
43:01
should be codified and should at least be out there for the
43:05
masses. Well, Simon, we're going to cut
43:08
it here for everyone listening, if you want, one of his books,
43:13
if you want. The downloadable version comment
43:17
and we will send you one over maybe if you're lucky, that's it
43:23
for now, it will be free pokes. There will be how many I can't
43:27
say for sure exactly. So if anyone wants to time, if
43:33
you are already listening to this conversation there is a big
43:37
program but mapping which allows all of the chapters to be
43:41
available to everybody online. So it's a fantastic idea.
43:44
Just looking and know for sure. Hey, I want the book because
43:49
like that's something I do a lot of times which is all read the
43:52
book a few chapters and then I'll buy the book because I feel
43:54
like it's so important to be able to reference back to.
43:58
Yeah. The book essentially, and the
44:00
whole book is available now. And the point, I think the point
44:04
of having a book is to have, you know, everything brought
44:07
together to try and bring things together as and have just one
44:10
article about one thing but to try to bring things together.
44:13
So you know, I think, I think the fact that as of last week
44:17
checked Temple into the MEAP and, you know, it's all there
44:22
for people to have a look at. It's awesome to see.
44:26
So, thanks so much, so even if anyone wants to reach out to
44:29
you, what's the best way to connect email Simon?
44:33
Don't cheat. Simon dot-to-dot Thompson at
44:35
gmail.com. Yeah, there's number two so it's
44:38
not a hard one. Is it?
44:40
And it's time some of the P or on Twitter.
44:43
I am AI. Simon Thompson, you should be
44:45
able to find me, send me messages and I'll reply because
44:49
it is fun about the book. Not about my hair styles.
44:53
I'm excited man.